chore: import upstream snapshot with attribution
This commit is contained in:
@@ -0,0 +1,22 @@
|
||||
# Copy to showcase/.env and fill in.
|
||||
# Picked up by docker-compose.local.yml via env_file for every showcase package container.
|
||||
# Not committed to git.
|
||||
|
||||
OPENAI_API_KEY=
|
||||
ANTHROPIC_API_KEY=
|
||||
LANGSMITH_API_KEY=
|
||||
|
||||
# Optional: route LLM traffic through local aimock on the compose network.
|
||||
# Most packages consume OPENAI_BASE_URL / ANTHROPIC_BASE_URL directly.
|
||||
# Spring-AI splits base-url (host) + completions-path (/v1/chat/completions),
|
||||
# so it needs the host without /v1 via SPRING_AI_OPENAI_BASE_URL.
|
||||
# OPENAI_BASE_URL=http://aimock:4010/v1
|
||||
# ANTHROPIC_BASE_URL=http://aimock:4010
|
||||
# SPRING_AI_OPENAI_BASE_URL=http://aimock:4010
|
||||
|
||||
# Package-specific (only required for the packages that use them).
|
||||
# ms-agent-dotnet: GitHub model provider token.
|
||||
# Use any non-empty value for local dev (aimock doesn't validate tokens).
|
||||
GitHubToken=gh-mock-local-dev
|
||||
# google-adk / gemini-based demos.
|
||||
GOOGLE_API_KEY=
|
||||
@@ -0,0 +1,27 @@
|
||||
shared_python/
|
||||
shared_frontend/
|
||||
shared_typescript/
|
||||
|
||||
# Incremental TypeScript build state — regenerated on every tsc
|
||||
# invocation and not useful to track.
|
||||
**/tsconfig.tsbuildinfo
|
||||
|
||||
# Test fixture generated by showcase/scripts/__tests__/create-integration.test.ts.
|
||||
# If the test is killed before cleanup, the directory leaks into the worktree;
|
||||
# this entry prevents accidental `git add`.
|
||||
integrations/test-integration-tmp/
|
||||
|
||||
# Generated data files — produced by showcase/scripts/ generators at build
|
||||
# time and dev startup. Every build path (Docker, CI, npm run build, npm
|
||||
# run dev) regenerates them, so they don't need to be committed. Tracking
|
||||
# them caused constant git noise from embedded timestamps.
|
||||
shell/src/data/*.json
|
||||
shell-dojo/src/data/*.json
|
||||
shell-docs/src/data/*.json
|
||||
shell-dashboard/src/data/*.json
|
||||
|
||||
# Eval local artifacts
|
||||
.eval-results/
|
||||
.eval-baseline.json
|
||||
|
||||
next-env.d.ts
|
||||
@@ -0,0 +1 @@
|
||||
2026-05-01T05:11:09.810Z
|
||||
@@ -0,0 +1,802 @@
|
||||
# Showcase Local Debugging Playbook
|
||||
|
||||
Tagline: per-failure-mode debugging strategies, the debugging loop, integration
|
||||
patterns, anti-patterns, and production-side ops (harness probe logging,
|
||||
triggering probes manually, isolated-stack ops).
|
||||
|
||||
For the canonical cell red→green SOP and the `bin/showcase test` invocation
|
||||
table (control-plane vs `--direct`), see
|
||||
[`TESTING.md`](./TESTING.md#sop-turning-a-cell-red--green). This document
|
||||
covers the per-failure-mode investigation strategies that complement the SOP,
|
||||
plus production / harness ops (probe triggering, Railway log access, isolated
|
||||
stack cleanup).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
See [README.md](README.md) for Docker/Colima/OrbStack setup, API key configuration, and the general layout of the showcase directory. This document assumes you have a working Docker engine and a populated `.env` file.
|
||||
|
||||
## CLI Reference
|
||||
|
||||
The unified CLI is at `bin/showcase`. It wraps Docker Compose and adds debugging-specific commands. All commands can be run from any directory -- paths are resolved relative to the script itself.
|
||||
|
||||
```sh
|
||||
# From repo root:
|
||||
./showcase/bin/showcase <command> [args...]
|
||||
|
||||
# Or from within showcase/:
|
||||
./bin/showcase <command> [args...]
|
||||
```
|
||||
|
||||
### Core Commands
|
||||
|
||||
| Command | Description |
|
||||
| -------------------------- | --------------------------------------------------------------------------------------------------- |
|
||||
| `showcase up [slug...]` | Start containers (rebuilds if source changed). No args = infra only (aimock, pocketbase, dashboard) |
|
||||
| `showcase down [slug...]` | Stop containers. No args = stop everything |
|
||||
| `showcase build [slug...]` | Build Docker images without starting containers |
|
||||
| `showcase ps` | Show running containers and their status |
|
||||
| `showcase ports` | Print slug-to-host-port mapping (from `shared/local-ports.json`) |
|
||||
| `showcase logs <slug>` | Follow container logs (supports `--grep`, `--since`, `-n`, `--no-follow`) |
|
||||
|
||||
### Debugging Commands
|
||||
|
||||
| Command | Description |
|
||||
| ---------------------------- | ---------------------------------------------------------------------------------------------------- |
|
||||
| `showcase aimock-rebuild` | Rebuild local aimock from a source checkout and redeploy the container |
|
||||
| `showcase recreate <slug>` | Force-recreate a service (picks up a newly built image) |
|
||||
| `showcase test <slug>` | Run probe tests against a running service |
|
||||
| `showcase fixtures validate` | Check fixture JSON files for structural errors, duplicates, and common mistakes |
|
||||
| `showcase doctor` | Diagnose common local stack issues (Docker engine, Depot interception, stale images, port conflicts) |
|
||||
| `showcase diff-logs <slug>` | Show log output for a specific time window, filtering out noise from before your change |
|
||||
|
||||
## The Debugging Loop
|
||||
|
||||
This is the iterative process for going from a red probe to green. Each section below is a phase you will cycle through. Most debugging sessions follow the same pattern: establish baseline, trace what aimock sees, fix, re-test.
|
||||
|
||||
### Phase 1: Establish the Red Baseline
|
||||
|
||||
Run the failing test in an isolated stack to see the exact error. This is the
|
||||
production-equivalent path — control-plane pipeline, per-slug rebuild only,
|
||||
no interference with the shared `showcase-*` stack:
|
||||
|
||||
```sh
|
||||
showcase test mastra:<demo> --d5 --isolate --verbose
|
||||
```
|
||||
|
||||
For the older shared-stack workflow (e.g. if you already have the stack up and
|
||||
want to iterate without a fresh build), the equivalent two-step is:
|
||||
|
||||
```sh
|
||||
showcase up aimock mastra
|
||||
showcase test mastra --d5 --verbose
|
||||
```
|
||||
|
||||
But `--isolate` is the canonical SOP. See
|
||||
[`TESTING.md`](./TESTING.md#sop-turning-a-cell-red--green).
|
||||
|
||||
What to look for in the output -- the specific probe name and error message. Common patterns:
|
||||
|
||||
- **"chained reply missing fragments after 30000ms"** -- fixture matching issue. The agent is sending requests that don't match any fixture, so aimock returns 404 and the agent stalls.
|
||||
- **"timeout"** -- container not responding. The service may not have started, may be crash-looping, or may be listening on the wrong port. Check logs first.
|
||||
- **"404"** -- endpoint not found. The demo route doesn't exist, or the agent backend path is wrong. Check the integration's routing config.
|
||||
- **"JS error on page"** -- a frontend error is crashing the demo. Use Playwright UI mode (`npx playwright test --ui`) from the integration directory to see the browser and open DevTools.
|
||||
|
||||
### Phase 2: Trace Fixture Matching
|
||||
|
||||
When an agent loops, stalls, or produces wrong output, the answer is almost always in what aimock is matching (or failing to match):
|
||||
|
||||
```sh
|
||||
showcase logs aimock --grep "fixture|match|NO match|404"
|
||||
```
|
||||
|
||||
What the log lines mean:
|
||||
|
||||
- **"matched fixture X"** (with the chosen discriminator — `turnIndex`,
|
||||
`hasToolResult`, `toolCallId`, `sequenceIndex`, or just `userMessage`) --
|
||||
aimock found a fixture for this request and is returning it. If the same
|
||||
discriminator repeats turn-after-turn, the agent is stuck in a loop
|
||||
(re-sending the same request and getting the same canned response).
|
||||
- **"NO match"** -- the request pattern doesn't match any fixture. This means
|
||||
either the fixture is missing, or the request shape has changed (different
|
||||
model, different system prompt, different tool definitions), or the chosen
|
||||
discriminator (often `toolCallId` strict-equality) silently misses against a
|
||||
backend that rewrites IDs.
|
||||
- **Repeated matches at the same discriminator value** -- the agent's
|
||||
retry/loop logic is firing because it didn't get the response it expected
|
||||
from the previous turn. The fixture chain is broken somewhere upstream.
|
||||
|
||||
### Phase 3: The aimock Edit-Build-Deploy-Test Cycle
|
||||
|
||||
This is the most repeated cycle during debugging. When you need to change aimock's behavior (response format, fixture matching logic, streaming behavior), the `aimock-rebuild` command automates the full rebuild-and-redeploy:
|
||||
|
||||
```sh
|
||||
# 1. Edit aimock source (e.g., src/responses.ts, src/fixture-matcher.ts)
|
||||
|
||||
# 2. Rebuild and redeploy:
|
||||
showcase aimock-rebuild --from /path/to/aimock
|
||||
|
||||
# 3. Run the test:
|
||||
showcase test mastra --d5 --verbose --cycle
|
||||
```
|
||||
|
||||
The `--cycle` flag on `test` automatically dumps aimock's log delta on failure, saving you from running a separate `logs` command after each failed attempt.
|
||||
|
||||
**Without the CLI** (the manual equivalent, for understanding what the commands do under the hood):
|
||||
|
||||
```sh
|
||||
cd /path/to/aimock && npm run build
|
||||
DEPOT_DISABLE=1 docker buildx build --builder desktop-linux --load -t aimock:local .
|
||||
docker compose -f tests/docker-compose.integrations.yml up -d --force-recreate aimock
|
||||
sleep 5 && docker logs showcase-aimock 2>&1 | tail -3
|
||||
```
|
||||
|
||||
The CLI version handles the Depot bypass, builder selection, compose file path, and container readiness check automatically.
|
||||
|
||||
### Phase 4: Integration Code Fixes
|
||||
|
||||
When aimock is behaving correctly but the integration itself has bugs (wrong tool definitions, broken agent wiring, frontend rendering issues):
|
||||
|
||||
```sh
|
||||
# Edit integration source (integrations/<slug>/src/...)
|
||||
showcase build <slug> # rebuild the Docker image
|
||||
showcase recreate <slug> # pick up the new image
|
||||
showcase test <slug> --d5 --verbose
|
||||
```
|
||||
|
||||
Or combine build + recreate in one step:
|
||||
|
||||
```sh
|
||||
showcase recreate <slug> --build
|
||||
```
|
||||
|
||||
Use the Playwright UI mode directly (`npx playwright test --ui` from the integration directory) for interactive debugging of frontend issues where the DOM isn't rendering what the probe expects.
|
||||
|
||||
### Phase 5: Fixture Iteration
|
||||
|
||||
When adding or modifying aimock fixtures (the JSON files that define canned responses):
|
||||
|
||||
```sh
|
||||
# 1. Edit fixture JSON (showcase/aimock/*.json)
|
||||
|
||||
# 2. Validate fixtures for common errors (malformed JSON, duplicate keys,
|
||||
# missing required fields, turnIndex gaps):
|
||||
showcase fixtures validate
|
||||
|
||||
# 3. Recreate aimock to pick up the changed fixture files:
|
||||
showcase recreate aimock
|
||||
|
||||
# 4. Test:
|
||||
showcase test <slug> --d5 --verbose
|
||||
```
|
||||
|
||||
Fixtures are baked into the aimock Docker image at build time AND cached in
|
||||
memory at container startup. Simply editing the JSON file on disk does nothing
|
||||
until the container is recreated (or restarted in the case of a volume-mounted
|
||||
isolated stack). This is the most common "why isn't my fix working?" mistake.
|
||||
|
||||
In an `--isolate` stack, aimock reads its fixtures from a volume mount, so a
|
||||
fresh isolated slot picks up edits at startup automatically. But within a
|
||||
warm slot, edits require an explicit
|
||||
`docker restart showcase-iso<N>-aimock`. See
|
||||
[`GOTCHAS.md`](./GOTCHAS.md#-isolate--aimock-operational-edge-cases) for the
|
||||
operational details.
|
||||
|
||||
### Phase 6: Verify Green
|
||||
|
||||
Once you believe the fix is in, run the full probe suite for the slug to confirm everything passes:
|
||||
|
||||
```sh
|
||||
showcase test <slug> --d5 --verbose
|
||||
# Expected: all probes pass, no timeouts, no fixture mismatches
|
||||
```
|
||||
|
||||
If you want extra confidence, run the test command multiple times to check for flakes.
|
||||
|
||||
## Gotchas and Common Mistakes
|
||||
|
||||
### restart vs recreate
|
||||
|
||||
`docker compose restart` reuses the existing container and image. If you have rebuilt an image, the restarted container still runs the OLD image. This is the single most common source of "I rebuilt but nothing changed."
|
||||
|
||||
Always use `showcase recreate` (which runs `docker compose up --force-recreate`) when you need the new image:
|
||||
|
||||
```sh
|
||||
# WRONG -- still uses old image:
|
||||
docker compose restart mastra
|
||||
|
||||
# RIGHT -- picks up new image:
|
||||
showcase recreate mastra
|
||||
```
|
||||
|
||||
### Depot intercepts Docker builds
|
||||
|
||||
On machines with Depot CLI installed, `docker build` is silently proxied through Depot's remote builders. This causes two problems:
|
||||
|
||||
1. The `--load` flag may not work as expected (the image stays on the remote builder instead of being loaded locally).
|
||||
2. Build caching behaves differently, and local filesystem mounts may not resolve.
|
||||
|
||||
The `aimock-rebuild` command handles this automatically by setting `DEPOT_DISABLE=1` and using `--builder desktop-linux`.
|
||||
|
||||
If you are building manually:
|
||||
|
||||
```sh
|
||||
DEPOT_DISABLE=1 docker buildx build --builder desktop-linux --load -t myimage:local .
|
||||
```
|
||||
|
||||
Run `showcase doctor` to check if Depot is intercepting your builds. The doctor command tests for the Depot shim and warns you if it is active.
|
||||
|
||||
### Aimock is stateless; the Responses API is not
|
||||
|
||||
The OpenAI Responses API uses `item_reference` to point to previous response items by ID. Aimock does not track conversation state across requests, so it cannot resolve these references dynamically. The synthetic assistant message fix handles this for `turnIndex`-based matching, but other stateful API features (e.g., conversation branching, response chaining by ID) may surface similar issues.
|
||||
|
||||
If you see errors about unresolvable item references, the fix is usually to ensure the fixture chain includes all necessary prior-turn context in each response, rather than relying on aimock to remember previous turns.
|
||||
|
||||
### Sub-agent calls are independent LLM requests
|
||||
|
||||
Each framework's sub-agent (e.g., Mastra's `Agent.generate()`, CrewAI's crew member, LangGraph's tool-calling node) hits aimock as a completely separate HTTP request with a different system prompt and user message. Fixtures for sub-agents must be added explicitly -- they do not inherit from the supervisor's fixture chain.
|
||||
|
||||
When debugging multi-agent flows:
|
||||
|
||||
1. Use `showcase logs aimock --grep "match"` to see ALL requests, not just the top-level one.
|
||||
2. Each sub-agent request needs its own fixture with the correct system prompt pattern and turnIndex.
|
||||
3. The order of sub-agent calls may not be deterministic -- fixtures should be robust to reordering.
|
||||
|
||||
### Production vs local aimock behavior
|
||||
|
||||
Production aimock uses `--proxy-only`, which silently forwards unmatched requests to real OpenAI. Local aimock returns 404 for unmatched requests. This difference matters:
|
||||
|
||||
- **Production can mask missing fixtures** -- the real LLM fills in, and the test may pass by coincidence. You won't know the fixture is incomplete until something changes in the LLM's behavior.
|
||||
- **Local surfaces fixture gaps immediately** -- 404 errors make missing fixtures obvious. This is a feature, not a bug.
|
||||
- **Local is the better environment for catching fixture gaps early.** If your test passes locally with all requests matched by fixtures, it will pass in production. The reverse is not guaranteed.
|
||||
|
||||
## Workflows by Use Case
|
||||
|
||||
### "A D5 probe is failing on CI"
|
||||
|
||||
This is the most common debugging scenario. The goal is to reproduce the failure locally, where you have full access to logs and can iterate quickly.
|
||||
|
||||
1. `showcase doctor` -- verify your local stack is healthy before chasing red herrings.
|
||||
2. `showcase up aimock <slug>` -- start the failing integration plus aimock.
|
||||
3. `showcase test <slug> --d5 --verbose --cycle` -- reproduce the failure locally. The `--cycle` flag dumps aimock logs on failure.
|
||||
4. `showcase logs aimock --grep "fixture|match"` -- trace what aimock is seeing. Is the fixture matched? Is it the right turnIndex?
|
||||
5. Fix the issue (fixture, aimock source, or integration code), then:
|
||||
- Fixture change: `showcase recreate aimock` then re-test.
|
||||
- Aimock source change: `showcase aimock-rebuild --from ~/proj/cpk/aimock` then re-test.
|
||||
- Integration code change: `showcase recreate <slug> --build` then re-test.
|
||||
|
||||
### "I changed aimock source and need to test it"
|
||||
|
||||
The `aimock-rebuild` command handles the full cycle: build the npm package, build the Docker image with Depot bypass, force-recreate the aimock container, and wait for readiness.
|
||||
|
||||
```sh
|
||||
showcase aimock-rebuild --from ~/proj/cpk/aimock
|
||||
showcase test <slug> --d5 --verbose
|
||||
```
|
||||
|
||||
### "I added a new fixture and it is not being matched"
|
||||
|
||||
Fixture matching issues are the most subtle to debug. Work through this checklist:
|
||||
|
||||
```sh
|
||||
# Check for JSON syntax errors, duplicate turnIndex values, missing fields:
|
||||
showcase fixtures validate
|
||||
|
||||
# Recreate aimock to pick up the new fixture file:
|
||||
showcase recreate aimock
|
||||
|
||||
# Watch what aimock is actually matching in real-time:
|
||||
showcase logs aimock --grep "match"
|
||||
|
||||
# Run the test with log dump on failure:
|
||||
showcase test <slug> --d5 --cycle
|
||||
```
|
||||
|
||||
If the fixture validates and aimock still says "NO match", the request pattern has diverged from what the fixture expects. Compare the logged request (system prompt, model, tools) against the fixture's match criteria.
|
||||
|
||||
### "A container is running but returning errors"
|
||||
|
||||
```sh
|
||||
# Check for stale images, port conflicts, missing env vars:
|
||||
showcase doctor
|
||||
|
||||
# Look at recent logs only (skip startup noise):
|
||||
showcase diff-logs <slug> --since 5m
|
||||
|
||||
# Try a fresh container (sometimes state gets corrupted):
|
||||
showcase recreate <slug>
|
||||
```
|
||||
|
||||
### "I want to see only logs from my last test run"
|
||||
|
||||
The test command writes a timestamp marker, and `diff-logs` can use it:
|
||||
|
||||
```sh
|
||||
showcase test <slug> --d5 --verbose # this writes .last-test-ts
|
||||
showcase diff-logs aimock --since last-test --grep "fixture"
|
||||
```
|
||||
|
||||
This filters out all log output from before your test started, showing only what happened during the test run itself.
|
||||
|
||||
## Isolated Verification Runs (`--isolate`)
|
||||
|
||||
`bin/showcase test <slug> --d6 --isolate <name>` is the canonical, default way
|
||||
to verify a slug's D6 state. It brings up a fully isolated stack — its own
|
||||
aimock, PocketBase, dashboard, integration, and harness control-plane +
|
||||
pool-worker — on offset ports in its own docker compose project, then runs the
|
||||
canonical `harness/src/probes/drivers/d6-all-pills.ts` driver: it enqueues
|
||||
per-pill jobs, the isolated worker claims them, and asserts per-pill. This is
|
||||
**identical to the non-isolate path** — same driver, same per-pill assertions —
|
||||
just namespaced so it never disturbs the shared long-lived `showcase-*` stack.
|
||||
|
||||
```sh
|
||||
showcase test <slug> --d6 --isolate <name>
|
||||
```
|
||||
|
||||
Verify with this flow rather than hand-driving the browser. The point of the
|
||||
isolated driver is the identical-tests invariant: the same assertions run the
|
||||
same way for every integration, so a result is comparable across integrations
|
||||
and to production. Manual clicking is non-reproducible and tests something
|
||||
subtly different each run.
|
||||
|
||||
### How it works
|
||||
|
||||
1. **`<name>` and slot/offset**: `<name>` names the isolated compose project and
|
||||
must start with a lowercase letter or digit, then lowercase letters, digits,
|
||||
`-` or `_` (`[a-z0-9][a-z0-9_-]*`, a docker compose project-name constraint;
|
||||
uppercase is normalized to lowercase with a warning). The name `showcase` is
|
||||
reserved — it is the default stack's own compose project name, so the CLI
|
||||
refuses it. Use a distinct name per run. The slot and port
|
||||
offset are **auto-assigned** — each run atomically claims a slot via `mkdir`
|
||||
under `${XDG_STATE_HOME:-$HOME/.local/state}/copilotkit/showcase/slots/N`
|
||||
and derives its offset as `(slot + 1) * 200`
|
||||
(slot 0 → +200, slot 1 → +400, ...). Up to 46 concurrent runs are supported.
|
||||
Do not assign slots manually.
|
||||
|
||||
2. **Full isolated stack, shared stack untouched**: every service runs under the
|
||||
`<name>` compose project on the offset ports, so containers, networks, and
|
||||
volumes are fully namespaced. The default/long-lived `showcase-*` project is
|
||||
left completely alone — an isolated run is safe to launch alongside it (or
|
||||
alongside other isolated runs).
|
||||
|
||||
3. **PocketBase authenticates out of the box**: the host CLI's default
|
||||
PocketBase superuser (`admin@example.com` / `showcase-local-dev`) matches the
|
||||
`POCKETBASE_SUPERUSER_EMAIL` the compose stack seeds, so a fresh isolated PB
|
||||
authenticates with no manual setup. A mismatch here is what previously 400'd
|
||||
on pb-auth and left the d6 control-plane enqueuing zero jobs.
|
||||
|
||||
4. **Scratch overlay + cleanup**: `apply_isolation` writes offset copies of
|
||||
`docker-compose.local.yml` and `shared/local-ports.json` into a per-run
|
||||
scratch dir at
|
||||
`${XDG_STATE_HOME:-$HOME/.local/state}/copilotkit/showcase/runs/<name>/`
|
||||
(originals are never touched).
|
||||
`restore_isolation`, registered via `trap EXIT` before any mutation, tears
|
||||
the stack down and frees the slot on exit — even on crashes or `Ctrl-C` —
|
||||
unless `--keep` is set (see Cleanup below).
|
||||
|
||||
### Interpreting results
|
||||
|
||||
Per-pill FAILs reflect real demo/feature issues for that integration, not
|
||||
artifacts of isolation. Because the driver asserts per-pill identically across
|
||||
integrations, a FAIL is the same signal you'd get from the shared stack or
|
||||
production for that pill.
|
||||
|
||||
### Cleanup
|
||||
|
||||
By default the isolated stack tears down on exit and frees its slot
|
||||
automatically — the normal case needs no cleanup, and each fresh run gets a
|
||||
clean PocketBase volume (which is what keeps pb-auth deterministic).
|
||||
|
||||
With `--keep`, the isolated stack survives the run (success or failure): the
|
||||
stack is left standing, and the per-run scratch dir and slot are preserved
|
||||
(live containers keep the slot from being reaped). That protection applies
|
||||
only while the containers are RUNNING — if they stop (manual `docker stop`,
|
||||
daemon restart, host reboot), the next isolate run's sweep reclaims the slot,
|
||||
composing the stopped containers and named volumes down and removing the
|
||||
scratch dir. Inspect a kept stack before stopping it; it does not survive a
|
||||
reboot. At exit a survival notice
|
||||
prints the stack's host ports (aimock, dashboard, PocketBase) plus the manual
|
||||
teardown command
|
||||
(`docker compose -p <name> down --remove-orphans --volumes && rm -rf <run-dir> <slot-dir>`).
|
||||
The teardown includes `--volumes`: isolated stacks are ephemeral, so the
|
||||
project-scoped named volumes (e.g. `<name>_showcase-pb-data`) are removed along
|
||||
with the containers and networks — the same flags the automatic (non-`--keep`)
|
||||
teardown uses, so a kept stack leaves nothing behind once torn down. The
|
||||
`rm -rf` clears the per-run scratch dir and the slot reservation (the notice
|
||||
prints the real paths).
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
- **Wrong ports / stale compose state**: If you see `.iso-bak` files next to `docker-compose.local.yml` or `shared/local-ports.json`, those are leftovers from the OLD isolate behavior (which mutated files in place). Remove them and restore originals:
|
||||
|
||||
```sh
|
||||
git checkout showcase/docker-compose.local.yml showcase/shared/local-ports.json
|
||||
rm -f showcase/docker-compose.local.yml.iso-bak showcase/shared/local-ports.json.iso-bak
|
||||
```
|
||||
|
||||
The current isolate behavior auto-detects and cleans up these stale backups on startup, but a manual restore is the safest fix if things look wrong.
|
||||
|
||||
- **Scratch files**: Per-run overlay directories live at `${XDG_STATE_HOME:-$HOME/.local/state}/copilotkit/showcase/runs/<name>/`. They are cleaned up on normal exit; if a run was killed with `SIGKILL`, the directory may linger. Safe to remove manually.
|
||||
|
||||
- **Slot directories**: Located at `${XDG_STATE_HOME:-$HOME/.local/state}/copilotkit/showcase/slots/`. Each numbered subdirectory is a claimed slot. Slots from killed processes are auto-reaped on the next isolate run (a slot whose compose project has no live containers is reclaimed); to clean them manually, run `rm -rf "${XDG_STATE_HOME:-$HOME/.local/state}/copilotkit/showcase/slots"/*`.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Purpose | Default |
|
||||
| ------------------- | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
|
||||
| `AIMOCK_SRC` | Path to local aimock checkout for `aimock-rebuild` | `../../aimock` relative to `showcase/` (sibling of repo root), then `../aimock` |
|
||||
| `SHOWCASE_LOCAL` | Use localhost ports instead of Railway URLs in the shell app | unset |
|
||||
| `DEPOT_DISABLE` | Bypass Depot CLI for local Docker builds | unset (set to `1` to disable) |
|
||||
| `OPENAI_API_KEY` | Required for all integrations (even with aimock, some init code validates the key) | none |
|
||||
| `ANTHROPIC_API_KEY` | Required for Claude Agent SDK demos | none |
|
||||
|
||||
## D5 Debugging Strategies
|
||||
|
||||
These are investigation techniques — ways to LEARN what's wrong. Each was earned by a real debugging session.
|
||||
|
||||
### Strategy 1: Binary classification before single-feature debugging
|
||||
|
||||
**When**: Multiple features fail and you don't know why.
|
||||
|
||||
**Do**: Run ALL features for the integration and categorize pass/fail. Look for the axis that separates them. Don't debug any single feature until you've found the pattern.
|
||||
|
||||
```sh
|
||||
showcase test <slug> --d5 --verbose 2>&1 | grep "feature-complete"
|
||||
```
|
||||
|
||||
Sort the results into pass/fail. Ask: what do all passing features have in common? What do all failing features share? The spring-ai breakthrough came from noticing: text-only features pass, tool features fail. That one observation eliminated 90% of the search space.
|
||||
|
||||
### Strategy 2: Same-endpoint differential
|
||||
|
||||
**When**: Two features use the same backend endpoint but one passes and one fails.
|
||||
|
||||
**Do**: Find the MOST similar passing feature to your failing one. Diff their request/response flows. The difference IS the bug.
|
||||
|
||||
Example: `agentic-chat` and `tool-rendering` both hit the same Java `StreamingToolAgent`. One passes, one fails. The only difference is tool calls. That tells you the bug is in tool event emission, not streaming, not fixture matching, not routing.
|
||||
|
||||
### Strategy 3: Bypass the frontend — curl the backend directly
|
||||
|
||||
**When**: You can't tell if the problem is frontend rendering or backend response.
|
||||
|
||||
**Do**: Hit the backend API directly with the exact request the runtime would send. Compare the raw SSE stream with langgraph-python's.
|
||||
|
||||
```sh
|
||||
# From inside the Docker network:
|
||||
docker exec showcase-<slug> curl -X POST http://localhost:8000/ \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"messages":[{"role":"user","content":"weather in Tokyo"}]}'
|
||||
```
|
||||
|
||||
If the raw backend response is correct but the probe fails, the bug is in the frontend/runtime layer. If the backend response is wrong, debug the agent code. This eliminates an entire half of the stack in one command.
|
||||
|
||||
### Strategy 4: Message count N→0 means duplicate IDs
|
||||
|
||||
**When**: The probe reports `baseline=0, current=0` but you can see the assistant responded (via backend logs or aimock fixture match).
|
||||
|
||||
**Do**: Check for duplicate `messageId` values in the AG-UI event stream. React's `deduplicateMessages()` uses a `Map<id, message>`. If a tool result event reuses the assistant message's ID, the map overwrites the assistant message with the tool message — it vanishes from the DOM.
|
||||
|
||||
Look at the agent's event emission code. Every event that creates a message needs a unique ID. `UUID.randomUUID()` for each event, never reuse the parent message's ID.
|
||||
|
||||
### Strategy 5: Test the gold standard first
|
||||
|
||||
**When**: A feature fails and you're about to blame the framework.
|
||||
|
||||
**Do**: Run `showcase test langgraph-python:<demo> --d5 --isolate` in the SAME
|
||||
environment first. If langgraph-python also fails on the same cell, the
|
||||
problem is infrastructure (stale aimock, broken fixtures, Docker state, probe
|
||||
bug), not the framework. This saves hours of framework-specific debugging that
|
||||
turns out to be a shared issue.
|
||||
|
||||
```sh
|
||||
showcase test langgraph-python:<demo> --d5 --isolate # gold standard check
|
||||
showcase test <slug>:<demo> --d5 --isolate # then your target
|
||||
```
|
||||
|
||||
### Strategy 6: Check custom renderers for missing testids
|
||||
|
||||
**When**: `baseline=0, current=0` but the backend is correct AND the SSE stream has correct events.
|
||||
|
||||
**Do**: Check if the demo page uses a custom message renderer (`messageView={{ assistantMessage: CustomComponent }}`). Custom renderers that replace `CopilotChatAssistantMessage` must include `data-testid="copilot-assistant-message"` or the probe's selector cascade finds zero messages.
|
||||
|
||||
```sh
|
||||
grep -r "messageView\|assistantMessage" showcase/integrations/<slug>/src/app/demos/
|
||||
```
|
||||
|
||||
### Strategy 7: Trace the full event chain for tool features
|
||||
|
||||
**When**: Tool-involving features fail but text-only features pass.
|
||||
|
||||
**Do**: Trace each hop in order. Stop at the first broken link.
|
||||
|
||||
1. **Aimock** → did the fixture match? `docker logs showcase-aimock | grep "Fixture matched"`
|
||||
2. **Backend** → did the agent emit correct SSE events? Curl the backend directly (Strategy 3)
|
||||
3. **Runtime** → did the Next.js server process events without error? `docker logs showcase-<slug> | grep "Error\|ZodError"`
|
||||
4. **Frontend** → did React render the message? Check for duplicate messageId (Strategy 4) or missing testid (Strategy 6)
|
||||
|
||||
Don't skip hops. Don't start at the frontend. Work from the data source (aimock) forward.
|
||||
|
||||
### Strategy 8: Production parity — check env vars first
|
||||
|
||||
**When**: Feature passes locally but fails in production.
|
||||
|
||||
**Do**: Before investigating anything else, verify ALL provider base URLs are set on Railway:
|
||||
|
||||
```sh
|
||||
RAILWAY_PROJECT_ID=6f8c6bff-a80d-4f8f-b78d-50b32bcf4479 \
|
||||
railway variables --service showcase-<slug> --json | \
|
||||
python3 -c "import json,sys; d=json.load(sys.stdin); \
|
||||
[print(f'{k}={v[:40]}') for k,v in sorted(d.items()) if 'BASE_URL' in k or 'AIMOCK' in k]"
|
||||
```
|
||||
|
||||
Missing `ANTHROPIC_BASE_URL` or `GOOGLE_GEMINI_BASE_URL` means the service bypasses aimock and hits the real API. This produces non-deterministic results that look like flapping. Local docker-compose sets ALL providers to aimock — production must match.
|
||||
|
||||
### Strategy 9: Pull PocketBase D5 history to distinguish bugs from flapping
|
||||
|
||||
**When**: Production shows blues (D4) but everything passes locally. You need to know if a feature is genuinely broken or just flapping from transient production conditions.
|
||||
|
||||
**Do**: Pull the current D5 status records from PocketBase in one request and categorize the errors. PocketBase stores ONE record per `d5:<slug>/<featureType>` key (upsert), with `fail_count` tracking consecutive failures.
|
||||
|
||||
```sh
|
||||
# Get ALL currently-red D5 records with error details:
|
||||
curl -s 'https://showcase-pocketbase-production.up.railway.app/api/collections/status/records?sort=-updated&perPage=200&filter=key~%22d5:%22%20%26%26%20state!=%22green%22' | \
|
||||
python3 -c "
|
||||
import json, sys
|
||||
data = json.load(sys.stdin)
|
||||
items = data.get('items', [])
|
||||
print(f'{len(items)} RED D5 records')
|
||||
for item in items:
|
||||
key = item.get('key', '?').replace('d5:showcase-','')
|
||||
fc = item.get('fail_count', 0)
|
||||
signal = item.get('signal', {})
|
||||
error = signal.get('errorDesc', '') if isinstance(signal, dict) else ''
|
||||
print(f'{key:<50} fc={fc:<3} {error[:80]}')
|
||||
"
|
||||
```
|
||||
|
||||
**How to read it:**
|
||||
|
||||
- `fail_count=1` → just flipped red on the last cycle (likely flapper)
|
||||
- `fail_count>=3` → consistently failing (likely a real bug)
|
||||
- `fail_count=0` with `state!=green` → transitional state, check again next cycle
|
||||
|
||||
**Categorize errors** to find the root cause pattern:
|
||||
|
||||
- `page.fill: Timeout` → React hydration too slow (probe infrastructure issue)
|
||||
- `assistant did not respond within 30000ms` → backend or aimock not returning
|
||||
- `chat input not found` → selector cascade failed (page didn't render)
|
||||
- `keyword missing` → aimock returned wrong fixture
|
||||
- `auth: after clicking sign-out` → auth probe timing race
|
||||
|
||||
**Cross-reference with deploy history** to identify deploy churn:
|
||||
|
||||
```sh
|
||||
gh run list --branch main --limit 10 -R CopilotKit/CopilotKit --workflow "Showcase: Build & Push"
|
||||
```
|
||||
|
||||
If a feature flipped red right when a deploy happened and `fail_count=1`, it's deploy churn — the Railway service restarted mid-probe. Wait one cycle and re-check.
|
||||
|
||||
**Get per-service flapping rates** from the harness API (last 10 probe runs):
|
||||
|
||||
```sh
|
||||
curl -s 'https://showcase-harness-production.up.railway.app/api/probes/probe:d6-all-pills-e2e' | \
|
||||
python3 -c "
|
||||
import json, sys
|
||||
data = json.load(sys.stdin)
|
||||
runs = [r for r in data.get('runs', []) if r.get('state') == 'completed' and r.get('summary')][:10]
|
||||
stats = {}
|
||||
for run in runs:
|
||||
for svc in run['summary'].get('services', []):
|
||||
slug = svc.get('slug','?').replace('d6-all-pills-e2e:showcase-','')
|
||||
result = svc.get('result', '?')
|
||||
stats.setdefault(slug, {'green': 0, 'red': 0})
|
||||
stats[slug]['green' if result == 'green' else 'red'] += 1
|
||||
for slug in sorted(stats):
|
||||
s = stats[slug]
|
||||
rate = s['green'] / (s['green'] + s['red']) * 100
|
||||
print(f'{slug:<25} {s[\"green\"]}/{s[\"green\"]+s[\"red\"]} green ({rate:.0f}%)')
|
||||
"
|
||||
```
|
||||
|
||||
**Key insight**: If ALL integrations flap at similar rates (50-70% green), the problem is systemic (probe infrastructure), not per-integration code bugs. If only one integration is consistently red while others are green, it's a real code bug in that integration.
|
||||
|
||||
### Strategy 10: A red/BE✗ cell does NOT mean the feature is broken — it's often staleness
|
||||
|
||||
**When**: A coverage-dashboard cell renders red, or BE✗, or is capped at D3, but the app looks fine (page renders, agent responds). FIRST suspect freshness and harness throughput — not the app.
|
||||
|
||||
**Why the dashboard verdict ≠ a single DB row.** The dashboard is LIVE: it renders each cell's depth via `resolveD3`/`resolveD4`/`resolveD5`/`resolveD6` in `shell-dashboard/src/lib/cell-model.ts`. The per-cell **BE (D4) flag = `resolveD4` = WORST-OF(`chat:<slug>`, `tools:<slug>`)**, then folded through a **staleness window**. A green row OLDER than its window folds to stale and renders red / not-credited. Windows live in `shell-dashboard/src/lib/staleness.ts`: `D4_STALE_AFTER_MS = 60m`; D3/D5/D6 and the family aggregates use `E2E_STALE_AFTER_MS = 6h`; liveness uses `LIVENESS_STALE_AFTER_MS`.
|
||||
|
||||
So reading ONE PocketBase row (e.g. `chat:<slug>`) is NOT the dashboard's flag — it ignores `tools:<slug>` AND ignores staleness, and will FALSELY report "BE green" when the dashboard correctly shows BE✗. To reproduce the verdict you must apply worst-of-the-contributing-rows + the staleness fold exactly as `resolveD4`/`resolveD6` do.
|
||||
|
||||
**Root failure mode**: if a probe SWEEP takes longer than the staleness window, cells the sweep hasn't re-touched go stale and render red EVEN WHEN THE APP IS FINE. This happens when the worker pool is starved/under-capacity or the fleet is large. From a real 2026-06-26 prod incident — observed family sweep durations vs periods: d5 = 41m (period 15m), e2e-smoke = 45m (15m), e2e-demos = 97m (60m), d6 = 127m (60m). With the worker pool starved to ~1 worker (concurrency = `numReplicas × HARNESS_POOL_COUNT`), the D4 sweep blew past its 60m window → ~13 integration columns showed BE✗ / capped at D3 while the apps were healthy. Scaling workers (Railway `numReplicas` + `HARNESS_POOL_COUNT`) so a full sweep finishes within the window restored them.
|
||||
|
||||
**Prod-vs-staging disparity is frequently THIS, not a code difference.** Both envs run the same demos; if one env's harness can't complete sweeps within the staleness windows (starved workers / slow sweeps), its cells go stale-red while the healthy env stays green. Check harness throughput before concluding a regression.
|
||||
|
||||
**Diagnostic checklist** (so the next person doesn't waste a day):
|
||||
|
||||
1. Red cell capped at D3 with BE✗ → FIRST check freshness + worker throughput, not the app. Pull `https://showcase-harness-production.up.railway.app/api/runs` (`families[]` gives `lastRun.finishedAt` + `durationMs` + `periodMs` + `workers`). If a family's `durationMs` > its staleness window, you're in the staleness trap.
|
||||
2. Compare the failing cell's underlying row `observed_at` age to its `*_STALE_AFTER_MS`. An OLD green folded to stale → throughput/staleness, not a regression.
|
||||
3. Only after ruling out staleness, treat BE✗ as a real agent-round-trip failure.
|
||||
4. Worker concurrency = `numReplicas × HARNESS_POOL_COUNT` (Railway env on the harness-workers service). The browser-pool `MAX_CONTEXTS` (e.g. 40) is usually NOT the binding constraint.
|
||||
5. Remediation when sweeps exceed the window: scale worker concurrency so sweeps finish in-window, OR widen the staleness window (`staleness.ts` `D4_STALE_AFTER_MS`) if sweeps legitimately take longer than the current window.
|
||||
|
||||
## Integration Patterns
|
||||
|
||||
These are canonical. Do not deviate.
|
||||
|
||||
### HITL (hitl-steps, hitl-approve-deny, hitl-text-input)
|
||||
|
||||
Backend agent has `tools=[]` (no backend tools). Frontend registers tools via
|
||||
`useHumanInTheLoop` or `useFrontendTool`. CopilotKit injects frontend tool
|
||||
definitions into the LLM call. Every HITL integration follows this pattern
|
||||
without exception.
|
||||
|
||||
### gen-ui-custom
|
||||
|
||||
`langgraph-python` uses the chart pattern (`useComponent` with
|
||||
`render_pie_chart`). All other integrations should also demonstrate meaningful
|
||||
custom generative UI. Do not replace charts/data-viz with trivial text-only
|
||||
components just to pass tests.
|
||||
|
||||
### tool-rendering
|
||||
|
||||
Frontend registers `useRenderTool` for `get_weather`. The v2 API uses
|
||||
`parameters` (not `args`) in the render callback. Backend has the actual tool.
|
||||
|
||||
### shared-state
|
||||
|
||||
Backend calls `set_notes` tool, must forward tool result back to LLM for the
|
||||
follow-up text response. Frameworks that don't auto-cycle (crewai, langroid)
|
||||
need explicit tool-execution loops.
|
||||
|
||||
## Docker Compose Environment
|
||||
|
||||
All providers must be routed through aimock. Required env vars in
|
||||
`x-integration-defaults`:
|
||||
|
||||
```
|
||||
OPENAI_API_KEY / OPENAI_BASE_URL -> http://aimock:4010/v1
|
||||
ANTHROPIC_API_KEY / ANTHROPIC_BASE_URL -> http://aimock:4010
|
||||
GOOGLE_API_KEY / GOOGLE_GEMINI_BASE_URL -> http://aimock:4010
|
||||
SPRING_AI_OPENAI_BASE_URL -> http://aimock:4010
|
||||
```
|
||||
|
||||
Missing any of these means that provider's integrations bypass aimock and hit
|
||||
real APIs (or fail with an empty key).
|
||||
|
||||
## Production Debugging
|
||||
|
||||
### Harness probe logging
|
||||
|
||||
The harness emits structured logs at INFO level for probe lifecycle events:
|
||||
|
||||
- `probe.tick-start` / `probe.tick-complete` — probe run lifecycle
|
||||
- `probe.target-start` / `probe.target-complete` — per-service results
|
||||
- `probe.d6-all-pills.service-start` / `probe.d6-all-pills.service-complete` — D5 per-service
|
||||
- `probe.d6-all-pills.feature-complete` — per-feature pass/fail with error details
|
||||
- `probe.run-summary` — single line with all service results
|
||||
- `probe.d6-all-pills.pool-abort-release` — browser pool starvation events
|
||||
|
||||
View with: `RAILWAY_PROJECT_ID=6f8c6bff-a80d-4f8f-b78d-50b32bcf4479 railway logs --service showcase-harness --tail 200`
|
||||
|
||||
### Debug-level probe logging
|
||||
|
||||
For detailed conversation-runner traces (selector resolution, DOM text
|
||||
extraction, settle polling, per-turn lifecycle), set `LOG_LEVEL=debug` on the
|
||||
showcase-harness Railway service. This enables `console.debug(...)` output from
|
||||
the conversation runner and D5 scripts.
|
||||
|
||||
To enable temporarily: set the env var in Railway dashboard → showcase-harness
|
||||
→ Variables → `LOG_LEVEL=debug`. The service auto-restarts. Remember to unset
|
||||
after debugging — debug output is verbose.
|
||||
|
||||
### Triggering probes manually
|
||||
|
||||
```
|
||||
curl -sf -X POST "https://showcase-harness-production.up.railway.app/api/probes/probe:d6-all-pills-e2e/trigger" \
|
||||
-H "Authorization: Bearer $OPS_TRIGGER_TOKEN" \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
|
||||
Retrieve `OPS_TRIGGER_TOKEN`: `RAILWAY_PROJECT_ID=6f8c6bff-a80d-4f8f-b78d-50b32bcf4479 railway variables --service showcase-harness --json | python3 -c "import json,sys; print(json.load(sys.stdin)['OPS_TRIGGER_TOKEN'])"`
|
||||
|
||||
Rate limit: 5 minutes per probe ID.
|
||||
|
||||
### Testing package.json changes
|
||||
|
||||
When `package.json` changes (new deps, version bumps), volume mounts don't
|
||||
cover `node_modules`. You MUST rebuild the Docker image:
|
||||
`bin/showcase rebuild <slug>`, then re-test. A passing `bin/showcase test`
|
||||
against a volume-mounted container does NOT validate the build.
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
Earned by bugs. Do not repeat.
|
||||
|
||||
- **NEVER** change a demo's fundamental functionality to pass a test. The demo IS the point.
|
||||
- **NEVER** replace chart/data-viz gen-ui with trivial text components.
|
||||
- **NEVER** anchor multi-turn disambiguation on `toolCallId` strict equality
|
||||
when the backend rewrites IDs (Anthropic, TanStack). Use `turnIndex` or
|
||||
`hasToolResult` instead.
|
||||
- **NEVER** modify `response.content` to match what a real LLM emits. The
|
||||
canonical narration is fixture-author truth; the d5 probe asserts on it.
|
||||
Tune `match` keys, not the response.
|
||||
- **NEVER** use raw `docker build`. Symlinks break. Use `bin/showcase rebuild`.
|
||||
- **NEVER** assume "agent says done" means "D5 is green." Always run the actual test.
|
||||
- **NEVER** add a backend tool for something that should be a frontend HITL tool.
|
||||
- **NEVER** use `--direct` for cell-flip value-tests. It bypasses the queue/worker
|
||||
pipeline staging actually runs and has misled investigations in the past.
|
||||
- **NEVER** conclude a red/BE✗ cell means the app is broken without first ruling out
|
||||
staleness. A green row older than its `*_STALE_AFTER_MS` window folds to stale-red
|
||||
even when the app is healthy. Reading one PocketBase row also ignores the worst-of
|
||||
fold and staleness — it is NOT the dashboard's flag. See D5 Strategy 10.
|
||||
|
||||
## Aimock Fixture Deployment
|
||||
|
||||
When adding or modifying fixture files in `showcase/aimock/`, the
|
||||
`showcase-aimock` image must be rebuilt so production picks up the changes. CI
|
||||
handles this automatically — any push to `main` that touches
|
||||
`showcase/aimock/**` triggers the Build & Deploy workflow to rebuild and
|
||||
redeploy the image.
|
||||
|
||||
For manual iteration (e.g. testing a fixture change before merging), build and
|
||||
push directly:
|
||||
|
||||
```
|
||||
docker build --platform linux/amd64 -f showcase/aimock/Dockerfile -t ghcr.io/copilotkit/showcase-aimock:latest showcase/aimock/ --push
|
||||
```
|
||||
|
||||
After pushing, redeploy the Railway service so it pulls the new image (the CI
|
||||
workflow does this automatically via `serviceInstanceRedeploy`).
|
||||
|
||||
When adding a **new** fixture file, update both:
|
||||
|
||||
1. `showcase/docker-compose.local.yml` — add a volume mount for the new file
|
||||
2. `showcase/aimock/Dockerfile` — add a `COPY` line for the new file
|
||||
|
||||
## Dev Iteration Speed
|
||||
|
||||
Each integration service bind-mounts its host `src/` directory into the
|
||||
container via the `volumes` entry in `docker-compose.local.yml`:
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
- ./integrations/<slug>/src:/app/src
|
||||
```
|
||||
|
||||
This means **source edits take effect on container restart** without rebuilding
|
||||
the Docker image. The workflow becomes:
|
||||
|
||||
1. Edit code under `showcase/integrations/<slug>/src/`
|
||||
2. Restart the container: `bin/showcase restart <slug>`
|
||||
3. Re-run the test: `bin/showcase test <slug> --d5`
|
||||
|
||||
Use `bin/showcase rebuild <slug>` only when you change dependencies
|
||||
(requirements.txt, package.json) or non-src files (Dockerfile, entrypoint). For
|
||||
pure `src/` changes, restart is sufficient and much faster.
|
||||
|
||||
## Quick Diagnostic Commands
|
||||
|
||||
```sh
|
||||
# Is everything OK?
|
||||
showcase doctor
|
||||
|
||||
# What's running?
|
||||
showcase ps
|
||||
|
||||
# What port is mastra on?
|
||||
showcase ports | grep mastra
|
||||
|
||||
# Show aimock's fixture matching in real-time:
|
||||
showcase logs aimock --grep "fixture|match|NO match"
|
||||
|
||||
# Last 5 minutes of error logs:
|
||||
showcase diff-logs <slug> --since 5m --grep "error|Error|ERR"
|
||||
|
||||
# Validate all fixture files:
|
||||
showcase fixtures validate
|
||||
|
||||
# Full reset -- stop everything, rebuild, restart:
|
||||
showcase down
|
||||
showcase build <slug>
|
||||
showcase up aimock <slug>
|
||||
showcase test <slug> --d5 --verbose
|
||||
```
|
||||
@@ -0,0 +1,95 @@
|
||||
# Multi-Frontend Consolidation Strategy
|
||||
|
||||
Tagline: rationale for the multi-shell layout (`shell/`, `shell-dojo/`,
|
||||
`shell-docs/`, `shell-dashboard/`), the rollout order, and how packages target
|
||||
shells today (they don't — every shell embeds every package).
|
||||
|
||||
This document records the intentional rationale for having multiple showcase shells in this repo, which shell each one replaces, and how packages target them during the transition.
|
||||
|
||||
> This is a transition-period strategy document. Once the rollouts below are complete, expect this doc to be updated or retired.
|
||||
|
||||
## Why multiple shells exist
|
||||
|
||||
Showcase is in the middle of **consolidating several external frontends into this repo** so they are built, deployed, versioned, and tested alongside the integration packages that back them. The fan-out in `showcase/` is intentional: each shell is the target replacement for a distinct external property, and they co-exist until their respective rollouts cut over.
|
||||
|
||||
The alternative — a single super-shell that hosts every audience — was rejected because the existing external frontends each have a different visual language, different audience framing, and different routing/embed conventions. Merging them into one shell would either lose each identity or require runtime mode switching that obscures what's actually rendered at a given URL.
|
||||
|
||||
## Current shells and their roles
|
||||
|
||||
| Directory | Package name | Role | Status |
|
||||
| ------------------ | -------------------------------------- | ------------------------------------------------------------------------------------- | ------------------------------ |
|
||||
| `shell/` | `@copilotkit/showcase-shell` | Public showcase integrations browser at **showcase.copilotkit.dev**. Canonical today. | Production |
|
||||
| `shell-dojo/` | `@copilotkit/showcase-shell-dojo` | Styled like the external ag-ui Dojo. Target replacement for the ag-ui Dojo property. | Deployed (rollout in progress) |
|
||||
| `shell-docs/` | `@copilotkit/showcase-shell-docs` | Docs-forward shell for the `docs.copilotkit.dev` consolidation. | Deployed (rollout in progress) |
|
||||
| `shell-dashboard/` | `@copilotkit/showcase-shell-dashboard` | Internal feature × integration grid (ops / QA audience). | Internal |
|
||||
|
||||
All shells:
|
||||
|
||||
- Consume the same `shared/` registry / constraints / manifest schema
|
||||
- Pull from the same `registry.json` (generated by `scripts/generate-registry.ts`)
|
||||
- Embed package demos via **iframe pointing at `integration.backend_url + demo.route`** (each package's own deployed Next.js app) — they do not import package code
|
||||
|
||||
They differ in chrome, nav, and audience framing, not in the underlying demo surface.
|
||||
|
||||
### `shell/` — public integrations browser
|
||||
|
||||
The canonical public site. Routes under `src/app/`:
|
||||
|
||||
- `integrations/` — list and per-slug profile, plus `[slug]/[demo]/page.tsx` with Preview / Code / Docs tabs
|
||||
- `docs/[[...slug]]/` — MDX docs served from `src/content/docs/`
|
||||
- `ag-ui/[[...slug]]/` — ag-ui reference content
|
||||
- `matrix/`, `reference/` — matrix and reference surfaces
|
||||
|
||||
Build pipeline runs `generate-registry.ts` + `bundle-demo-content.ts` + `bundle-starter-content.ts` + `generate-search-index.ts` before `next build`. Docker image `showcase-shell`, Railway service `40eea0da-6071-4ea8-bdb9-39afb19225ec`.
|
||||
|
||||
### `shell-dojo/` — Dojo replacement
|
||||
|
||||
A single-page Dojo-style viewer (integration selector + demo list + preview iframe + code pane) styled to match the external ag-ui Dojo. This is **not abandoned spike or cleanup waste** — it is the rollout vehicle for replacing the external ag-ui Dojo site with an in-repo shell fed by the same registry every other showcase surface uses. Reference visuals are kept at `shell-dojo/agent-notes/reference-dojo.png` (external Dojo target) and `shell-dojo/agent-notes/v-final-handcrafted.png` (current iteration).
|
||||
|
||||
Built standalone (no monorepo scripts), Docker image `showcase-shell-dojo`, Railway service `7ad1ece7-2228-49cd-8a78-bddf30322907`. CI builds both shells independently in `.github/workflows/showcase_deploy.yml`.
|
||||
|
||||
### Adding future shells
|
||||
|
||||
The expected pattern for any additional consolidation target:
|
||||
|
||||
1. New directory at `showcase/<shell-name>/` with its own `package.json`, `Dockerfile`, `next.config.ts`
|
||||
2. Consume `shared/` registry + `data/registry.json` generated by `scripts/generate-registry.ts`
|
||||
3. Add a matching entry in `.github/workflows/showcase_deploy.yml` (`workflow_dispatch` option, change-detection filter, build-matrix entry) with its own Railway service id
|
||||
4. Update this document with the target external property, deploy URL, and rollout status
|
||||
5. If the shell needs its own demo-content bundling, extend `scripts/bundle-demo-content.ts` rather than forking it
|
||||
|
||||
## Rollout order
|
||||
|
||||
1. **ag-ui Dojo → `shell-dojo/`** (target #1, in progress).
|
||||
- Build in-repo against the live reference (`agent-notes/reference-dojo.png`)
|
||||
- Deploy Railway service `7ad1ece7-2228-49cd-8a78-bddf30322907` continuously from `main`
|
||||
- Visual parity pass against the reference screenshot
|
||||
- Domain cutover: point the external Dojo domain at the Railway service (see Open Questions)
|
||||
- Archive / redirect the external Dojo repo
|
||||
2. **Further consolidations** (TBD). Any other external frontend that is semantically a view over the showcase registry is a candidate. Per-shell criteria:
|
||||
- Audience and visual language distinct enough that folding into `shell/` would lose identity
|
||||
- Currently lives outside this repo (the point of this exercise is consolidation)
|
||||
- Can be expressed as a view over the existing registry, or the registry schema can be extended to support it
|
||||
|
||||
Order is driven by which external property is most worth consolidating next — not by code readiness in this repo.
|
||||
|
||||
## Per-package shell selection
|
||||
|
||||
**There is currently no explicit shell-selection field on packages.** The relationship is one-to-many in the other direction: every shell can embed every package.
|
||||
|
||||
- **At runtime, in every shell:** each integration package is embedded via `<iframe src={integration.backend_url + demo.route}>`. Both `shell/` (`src/app/integrations/[slug]/[demo]/page.tsx`) and `shell-dojo/` (`src/app/page.tsx`) do this using the exact same `backend_url` and `route` fields from `manifest.yaml`. Which shell a user sees is determined by which **domain** they visit, not by anything on the package.
|
||||
- **For per-package E2E:** tests in `packages/<slug>/tests/e2e/` run against the **package's own dev server** on `http://localhost:3000/demos/<id>` — not against any shell. `scripts/run-e2e-with-aimock.sh <slug>` starts aimock + the package's `pnpm dev` + Playwright; no shell is involved. See [`TESTING.md`](./TESTING.md#per-demo-coverage-matrix) for the current per-package/shared E2E split.
|
||||
- **For shared E2E** (`scripts/__tests__/e2e/`): these target whatever is running at `BASE_URL` (defaults to `http://localhost:3000`). The starter hero tests (`starter-e2e.spec.ts`) expect the starter app; demo tests (`demo-e2e.spec.ts`) expect a package dev server. Again, no shell is mounted.
|
||||
- **`NEXT_PUBLIC_BASE_URL`** on packages points at `showcase.copilotkit.dev` for production links back to the canonical shell. It does not gate which shell embeds a given demo.
|
||||
|
||||
**Implication:** as long as a package's `manifest.yaml` is valid and its backend is deployed with `deployed: true`, both shells pick it up automatically on the next `generate-registry.ts` run. No per-package change is required to appear in a new shell.
|
||||
|
||||
**If per-package shell targeting is ever needed** (e.g., a package should only appear in Dojo-replacement but not the canonical browser), this will require a new manifest field — something like `shells: ["shell", "shell-dojo"]` — plus a filter in each shell's registry consumer. That structural change is not in place today.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **Domain / DNS cutover plan for ag-ui Dojo replacement.** Which domain does `shell-dojo/` land on, and is the external Dojo repo archived or redirected on cutover?
|
||||
- **Shared vs. per-shell content.** `shell/` has substantial MDX docs under `src/content/`; `shell-dojo/` has none. When we add a third shell, is documentation shell-specific or hoisted into `shared/`?
|
||||
- **Visibility filtering.** Do we ever need a package to appear in one shell but not another? If yes, add a manifest field now rather than after the second rollout makes the lack of one painful.
|
||||
- **`extract-starter` and preview capture.** Preview capture scripts assume the `shell/` layout (`showcase/shell/public/previews/`, `showcase/shell/src/data/registry.json`). When a future shell wants previews, decide whether to hoist these assets into `shared/` or dual-write.
|
||||
- **E2E strategy once shells proliferate.** Per-package E2E still tests the package dev server directly, which is correct. But do we want shell-level E2E (iframe load, nav, search) per shell, and if so, where do those specs live?
|
||||
@@ -0,0 +1,194 @@
|
||||
# Showcase GOTCHAS — Framework & Integration Edge Cases
|
||||
|
||||
Tagline: framework-specific traps, aimock matcher / fixture-authoring edge
|
||||
cases, and `--isolate` operational gotchas. Load when a fixture, framework, or
|
||||
`--isolate` slot is misbehaving in ways the CLI output alone won't explain.
|
||||
|
||||
What we learned from getting all 18 integrations to D5 green. Many of these are things that were "green" but still wrong — passing probes while the underlying wiring was fragile, framework-specific, or relying on coincidence. This document exists so we don't re-learn these when rebuilding.
|
||||
|
||||
---
|
||||
|
||||
## Cross-Framework Patterns
|
||||
|
||||
**V1 vs V2 CopilotKit imports cause silent failures.** V1 is `@copilotkit/react-core`, V2 is `@copilotkit/react-core/v2`. Mixing V1 provider with V2 hooks (e.g., `useRenderTool`) silently fails — the tool rendering pipeline never wires up. Agent discovery also breaks: V2 runtime needs V2 provider. Found on: ms-agent-dotnet (auth), built-in-agent (interrupts), spring-ai (tool-rendering).
|
||||
|
||||
**Custom `assistantMessage` slot renderers must carry `data-testid="copilot-assistant-message"`.** The byoc-hashbrown demo overrides the slot with a HashBrown renderer. Without the testid, the probe (and any external consumer) sees 0 assistant messages. Every integration's hashbrown-renderer.tsx needed this fix independently. This is the #1 argument for a shared frontend.
|
||||
|
||||
**`copilotRuntimeNextJSAppRouterEndpoint()` must be hoisted to module scope.** Calling it inside the POST handler (per-request) causes `handleServiceAdapter` to repeatedly re-wrap `runtime.agents` in Promise layers. Under concurrent requests (/info + agent/run), this creates a race condition where the agents list is stale. Found on: google-adk (all 6 dedicated routes).
|
||||
|
||||
**Agent names must match exactly between frontend and backend.** `useAgent("agentic-chat-reasoning")` must match the backend registration. Dashes vs underscores, trailing hyphens, typos — all cause silent "Agent not found" errors that crash the page via React error boundary.
|
||||
|
||||
**`onRunInitialized` multimodal shim is framework-dependent.** langgraph-python NEEDS it (the `@ag-ui/langgraph` converter only understands legacy `binary` parts). langroid does NOT need it (speaks AG-UI directly — adding the shim causes double-encoding). Per-framework boolean, not a universal.
|
||||
|
||||
**Content parts from AG-UI arrive as Pydantic model instances, not dicts.** `isinstance(part, dict)` silently drops them. Must check `hasattr(part, "model_dump")` and call `model_dump(by_alias=True)`. Affected: langroid, ms-agent-python, pydantic-ai.
|
||||
|
||||
**`from __future__ import annotations` breaks Pydantic tool schemas.** PEP 563 makes all annotations strings. When LlamaIndex `AGUIChatWorkflow` passes `backend_tools` to Pydantic for schema generation, `Annotated[str, "..."]` is a raw string instead of a resolved type. Affected: llamaindex, crewai-crews, pydantic-ai, ag2. Fix: remove the import from files defining tools.
|
||||
|
||||
---
|
||||
|
||||
## Per-Framework Edge Cases
|
||||
|
||||
### langgraph-python (Reference — always compare against this)
|
||||
|
||||
- `a2ui_dynamic` graph owns `generate_a2ui` tool — runtime MUST NOT auto-inject (`injectA2UITool: false`). Double-injection confuses the LLM.
|
||||
- `server.mjs` must register ALL graphs from `langgraph.json`. We found it registering 5 of 25 — every unregistered graph returned 404.
|
||||
- Health probe uses `/ok` (langgraph-cli convention), not `/health`.
|
||||
- Version pinning: `langchain>=1.2.0` imports from `langgraph.runtime.ExecutionInfo` which doesn't exist in `langgraph==1.0.5`.
|
||||
|
||||
### langgraph-typescript
|
||||
|
||||
- Same server.mjs graph registration issue as langgraph-python.
|
||||
- Trailing slash on `deploymentUrl` matters for dedicated API routes. Missing it causes 404.
|
||||
- esbuild architecture mismatch on ARM Mac Docker builds. Passes in CI (Depot x86), fails locally on Apple Silicon.
|
||||
|
||||
### agno
|
||||
|
||||
- `reasoning=True` does multi-call chain-of-thought which breaks aimock (only first call matches). Disable for aimock-backed tests.
|
||||
- Agno's stock AGUI handler emits `STEP_STARTED`/`STEP_FINISHED` for reasoning — CopilotKit ignores these. The `reasoningMessage` slot requires `REASONING_MESSAGE_*` events. We built a custom handler, then reverted to stock AGUI.
|
||||
- Internal tool execution creates infinite fixture loops (same pattern as AG2).
|
||||
|
||||
### spring-ai
|
||||
|
||||
- **Java backend** — Maven build, fundamentally different toolchain.
|
||||
- `StreamingToolAgent.streamFirstTurn()` must include `toolCallbacks` with `internalToolExecutionEnabled=false`. Without this, aimock can't match `toolName: "get_weather"` — falls through to text-only fixture, weather card never renders.
|
||||
- AG-UI Java SDK not on Maven Central. Must clone and `mvn install` in Dockerfile.
|
||||
|
||||
### mastra
|
||||
|
||||
- **JS object shorthand key trap:** `{ weatherTool }` expands to function name `"weatherTool"`, not `"get_weather"`. Must use explicit keys: `{ get_weather: weatherTool }`.
|
||||
- `byocHashbrownAgent` needs its own dedicated agent with the hashbrown system prompt. The `weatherAgent` produces plain text → `useJsonParser` returns null → empty dashboard → timeout.
|
||||
- ~280s cold start (V8 JIT + Mastra boot). Watchdog can kill it before ready.
|
||||
|
||||
### ms-agent-python
|
||||
|
||||
- `AgentFrameworkAgent.run()` expects `input_data: dict`. The `_MultimodalAgent` override used `*args/**kwargs` → `TypeError` at runtime.
|
||||
- The override must `yield` events (async generator), not `return` (coroutine).
|
||||
- OpenAI `store=True` breaks aimock fixture matching. Set `store=False`.
|
||||
|
||||
### ms-agent-dotnet
|
||||
|
||||
- C# / .NET backend.
|
||||
- Auth page had V1 `CopilotKit` import → agent discovery failed → "Agent not found".
|
||||
|
||||
### built-in-agent
|
||||
|
||||
- **No Python backend.** TanStack AI `BuiltInAgent` runs in-process in Next.js.
|
||||
- `type: "tanstack"` with `convertTanStackStream` has a `runFinished` flag that blocks ALL events after first `RUN_FINISHED`. For byoc, must use `type: "custom"`.
|
||||
- OpenAI Responses API does NOT support `response_format: { type: "json_object" }` through TanStack adapter. The call silently fails — aimock never receives a request.
|
||||
|
||||
### crewai-crews
|
||||
|
||||
- `from __future__ import annotations` breaks `InterruptScheduling` import stubs in tests.
|
||||
- Backend tool execution doesn't cycle back to aimock for text follow-up. Known adapter limitation.
|
||||
|
||||
### pydantic-ai
|
||||
|
||||
- `_classify_binary_part()` has the `isinstance(part, dict)` bug. Pydantic models from AG-UI need `model_dump()`.
|
||||
- `starlette>=1.0.0` removes `on_startup`. Pin `starlette<1.0.0`.
|
||||
|
||||
### llamaindex
|
||||
|
||||
- `from __future__ import annotations` breaks Pydantic tool schema validation specifically when `backend_tools` are present but the response is text-only.
|
||||
|
||||
### langroid
|
||||
|
||||
- Custom AGUI handler (hand-written SSE, not a framework adapter).
|
||||
- `_normalize_part()` must handle Pydantic model instances via `model_dump(by_alias=True)`.
|
||||
- Does NOT need `onRunInitialized` multimodal shim.
|
||||
|
||||
### AG2
|
||||
|
||||
- `AGUIStream` requires plain string content. Multipart arrays cause 400 errors. `ContentFlattenerShim` handles conversion.
|
||||
- Internal tool execution + aimock = infinite loop. Fix: `max_consecutive_auto_reply` or `hasToolResult` in fixtures.
|
||||
|
||||
### google-adk
|
||||
|
||||
- **Underscores required for ALL agent names.** Every other framework uses dashes.
|
||||
- All 6 dedicated route files called `copilotRuntimeNextJSAppRouterEndpoint()` per-request → race condition. Fixed by hoisting to module scope.
|
||||
|
||||
### claude-sdk-python
|
||||
|
||||
- Transient "empty assistant text" flaps (fc=1, self-healing, not reproducible locally). Suspected SSE stream interruption on Railway.
|
||||
|
||||
---
|
||||
|
||||
## Aimock & Fixture Edge Cases
|
||||
|
||||
**Check fixtures FIRST.** When an agent misbehaves through aimock, the fixture determines behavior — the real LLM is never consulted.
|
||||
|
||||
**`sequenceIndex` counters are scoped per X-Test-Id.** aimock tracks match counts in `fixtureMatchCountsByTestId` (src/journal.ts), keyed by the request's `X-Test-Id` header — `DEFAULT_TEST_ID` when no header is sent, so manual/staging traffic effectively shares one counter set for the process lifetime (subject to the `fixtureCountsMaxTestIds` FIFO eviction cap). The D6 harness mints per-run unique ids via `buildE2eTestId`, so CI runs are isolated from each other. Three caveats: (1) the sibling co-increment grouping (`matchCriteriaEqual`) ignores `context`, so identical fixtures mirrored across integrations form ONE co-increment group — a match on any integration consumes a slot for all; (2) the grouping is exact-equality over the other match criteria, so adding `turnIndex`/`hasToolResult`/`predicate` to sequenceIndex variants — with per-variant values, or to some siblings but not others (predicates compare by function reference, so even identical ones differ) — silently un-groups the siblings, and click 2 falls to the fallback instead of the sequenceIndex 1 variant; (3) under shared/default test ids the counters never reset within a map entry's lifetime — but the `DEFAULT_TEST_ID` entry can itself be FIFO-evicted once the per-test-id map exceeds `fixtureCountsMaxTestIds` (default 500), which silently resets its counters to zero. The sanctioned pattern for repeat-invocation fixtures is sequenceIndex variants with a non-sequenced fallback ordered AFTER them, so strict mode never 503s and shared-test-id traffic gracefully degrades to the fallback id — see the beautiful-chat calculator fixtures. `hasToolResult` remains the stateless alternative, but it is thread-global (a shape predicate over the whole conversation) and breaks interleaved pills, so it is not a universal substitute.
|
||||
|
||||
**Tool-rendering fixtures need `toolName` in match criteria.** If the request doesn't include tool definitions, the fixture falls through to text-only. Spring-ai omitted tools; mastra's shorthand keys produced wrong function names.
|
||||
|
||||
**PDF turn is fragile.** Two-turn multimodal probe: if turn 2's message doesn't match the PDF fixture, the image fixture matches instead. The PDF fixture must be the most specific match.
|
||||
|
||||
---
|
||||
|
||||
## Fixture Authoring Gotchas
|
||||
|
||||
**`context` field is required for D4/D6 fixtures.** Context routing (aimock `--context-field`) uses `match.context` to isolate fixtures per integration. Omitting `context` means the fixture matches globally -- every integration hits it, and the first match wins regardless of which integration made the request. Always set `match.context` for any fixture loaded through per-integration routing.
|
||||
|
||||
**`match.context` must equal the integration slug.** The slug is the directory name under `showcase/integrations/` (e.g., `langgraph-python`, `mastra`, `spring-ai`). A mismatch between the context value and the slug silently drops the fixture from that integration's match pool -- aimock falls through to the next fixture or returns an unmatched response.
|
||||
|
||||
**Never combine `content` and `toolCalls` in a single fixture.** A fixture must return either text (`content`) or tool calls (`toolCalls`), not both. Combining them produces undefined behavior: some providers stream the text, others stream the tool call, and the order is non-deterministic. Split into two fixtures with `sequenceIndex` if you need text followed by a tool call (or vice versa).
|
||||
|
||||
**`hasToolResult` is a request-match predicate over the WHOLE thread, not response-side conversation tracking.** It gates matching on whether ANY `role: "tool"` message exists anywhere in the incoming request's messages (aimock src/router.ts). Omitting it applies NO gate -- there is no `true` default. `hasToolResult: false` on a leg-1 fixture means it stops matching as soon as any tool result appears in the thread -- and because the check is thread-global, a tool result from a DIFFERENT pill earlier in the conversation also disqualifies it, breaking interleaved multi-pill flows (see the sequenceIndex caveats above). The sanctioned pattern is to pair each leg-1 fixture with a `toolCallId`-anchored follow-up entry ordered BEFORE it, as the beautiful-chat calculator fixtures do.
|
||||
|
||||
**MIRROR the canonical (langgraph-python) fixtures — never re-record per-integration.** D6 fixtures must be authored by copying the canonical `aimock/d6/langgraph-python/<cell>.json` (and `langgraph-typescript/`) and re-keying `match.context` to the integration slug. Do NOT run `aimock --record` against an integration to capture its live traffic: the matcher keys mainly on `userMessage` + `context` and does not gate on the system prompt or tool schema, so a recording bakes in whatever (possibly buggy) request the integration sent and replays it green forever. Recording launders request-side bugs; mirroring forces every integration onto one shared contract. (See "What Was Green But Still Wrong" #7.)
|
||||
|
||||
**Common fix classes when mirroring (from LGP/LGT/ms-agent-dotnet):** `toolCallId` (2nd-leg) matcher must precede `toolName` (1st-leg); `chunkSize: 9999` for tool args that must JSON-parse in one chunk; inline narration `content` on tool-call fixtures for render/settle races; tighten over-broad d4 catch-alls (e.g. `"summarize"` → `"Summarize the"`); strip spurious `turnIndex: 0` (canonical fixtures have no turnIndex so any turn matches).
|
||||
|
||||
---
|
||||
|
||||
## `--isolate` & aimock operational edge cases
|
||||
|
||||
**aimock caches fixtures at container startup.** aimock reads fixtures from disk
|
||||
exactly once at boot and serves matches from an in-memory map. Editing a
|
||||
fixture in a live stack has no effect until the container restarts. Within an
|
||||
`--isolate` slot:
|
||||
|
||||
- **Fresh slot** (cold-start) — aimock loads fixtures from the volume mount on
|
||||
startup, so the first run after a fixture edit picks up the change for free.
|
||||
- **Warm slot** (reusing a kept stack) — fixture edits require an explicit
|
||||
`docker restart showcase-iso<N>-aimock` before the next test run, or you'll
|
||||
see the pre-edit behavior with no log indication of why.
|
||||
|
||||
This is the most-recurring "why isn't my fixture fix working?" trap during
|
||||
iterative cell debugging.
|
||||
|
||||
**`--isolate` slot collisions with foreign Docker projects.** The slot registry
|
||||
under `~/.local/state/copilotkit/showcase/slots/` only tracks `showcase-*`
|
||||
compose projects. If a sibling project (e.g. `ag2mm-*`, or another tool's
|
||||
docker stack) owns the same host ports for an auto-picked slot, health checks
|
||||
cross-resolve to the foreign containers and results misroute silently — the
|
||||
isolated stack appears red even though its own containers are healthy. Two
|
||||
remediations:
|
||||
|
||||
- **Pre-reserve the conflicting slot:** `mkdir
|
||||
~/.local/state/copilotkit/showcase/slots/<N>` for each slot whose port range
|
||||
collides with the foreign stack. The CLI skips reserved slots when picking.
|
||||
- **Tear down the foreign stack first:** `docker compose -p <foreign-project> down`
|
||||
before launching `--isolate`. Cleanest, but requires knowing which project
|
||||
is the culprit.
|
||||
|
||||
## Running D6 in Parallel (`--isolate`)
|
||||
|
||||
**The shared aimock is NOT a serialization bottleneck.** aimock matching is stateless per-request and context-keyed (`x-aimock-context: <slug>` per request); the only cross-request state is the per-X-Test-Id sequence counters (see the `sequenceIndex` gotcha above), which D6's per-run unique test ids (`buildE2eTestId`) keep isolated. So one instance serves many integrations concurrently with zero cross-talk for D6 traffic. Many integrations can run D6 at once.
|
||||
|
||||
**Use `--isolate <name>` for concurrent fixture-triage runs.** Each isolated stack gets its OWN aimock + pocketbase + dashboard + integration container on offset ports (`(slot+1)*200`, slot auto-claimed 0..45). The key benefit during triage: aimock has no hot-reload, so picking up edited fixtures requires a restart — and restarting a _shared_ aimock would nuke every concurrent run. A per-stack aimock means each run restarts only its own. Template: `bin/showcase test <slug>:<cell> --d6 --isolate iso-<slug>-w1 --verbose`. `<name>` must be lowercase `[a-z0-9_-]+`.
|
||||
|
||||
**Stagger concurrent launches 15-20s.** `stageSharedModules`/`restoreSymlinks` mutate `integrations/*/tools` symlinks in-place and `git checkout` them globally — simultaneous harness instances race there. Until per-isolation source-tree copies exist, stagger starts and keep concurrency modest (5-wide is comfortable; 10 is the theoretical ceiling at ~40 containers / 6-8GB).
|
||||
|
||||
**Pre-warm `:local` images before fanning out.** The Docker daemon serializes layered builds, so uncached integrations queue and stall the wave. Build (or pull `ghcr.io/copilotkit/showcase-<slug>:latest` and retag) ahead of time.
|
||||
|
||||
---
|
||||
|
||||
## What Was "Green" But Still Wrong
|
||||
|
||||
1. **18 copies of identical frontend code** — every fix was a blitz. One missed integration = one regression.
|
||||
2. **V1/V2 imports inconsistent** — some pages used V1 provider with V2 hooks and happened to work because the feature didn't exercise the broken path.
|
||||
3. **Most integrations still on V1 runtime API** — `copilotRuntimeNextJSAppRouterEndpoint` + `ExperimentalEmptyAdapter` instead of V2's `createCopilotRuntimeHandler` + `InMemoryAgentRunner`. Only `built-in-agent` fully uses V2. The V1 API has the per-request race condition (hoisting to module scope was a band-aid, not a migration).
|
||||
4. **Agent name mismatches masked by default fallback** — features passed because the runtime fell back to `"default"`, not because the correct agent was wired.
|
||||
5. **Missing testids on custom renderers** — probe assertions were weak enough to pass via fallback selectors.
|
||||
6. **`onRunInitialized` shim applied where unnecessary** — worked by coincidence because legacy format round-tripped correctly.
|
||||
7. **Re-recorded fixtures launder request-side bugs into green.** Because the aimock matcher keys on `userMessage` + `context` (not the system prompt or tool schema), capturing an integration's live traffic produces a fixture that matches that integration's exact (possibly malformed) request forever — a buggy system prompt or wrong tool name still replays green. The fix is to MIRROR the canonical langgraph-python fixtures, not record per-integration. _Mitigating fact:_ D6 assertions ARE canonical — `bin/showcase test <slug> --d6` runs only the shared `d6-all-pills.ts` driver against one global script registry (`harness/src/probes/scripts/d5-*.ts`) with strict `data-testid` checks; the per-integration `tests/e2e/*.spec.ts` are a separate surface `--d6` never invokes. So "pass D6" means "renders identical to the LGP contract," and a divergent integration (e.g. mastra emitting `custom-catchall-card` vs canonical `custom-wildcard-card`) genuinely fails — it cannot be papered green at the assertion layer, only at the fixture/request layer (hence rule #7).
|
||||
@@ -0,0 +1,279 @@
|
||||
# Integration Checklist
|
||||
|
||||
Tagline: per-package source checklist + external setup (Railway, secrets, CI,
|
||||
registry) for adding a new integration framework. Cross-ref:
|
||||
`../examples/integrations/<slug>/` for canonical Dojo dep-pinning source.
|
||||
|
||||
Two checklists: what makes a **complete package**, and what **external setup** is needed when adding a new framework.
|
||||
|
||||
---
|
||||
|
||||
## A. Complete Package (what `pnpm create-integration` generates)
|
||||
|
||||
Everything below should exist in `showcase/integrations/<slug>/`:
|
||||
|
||||
### Source Files
|
||||
|
||||
- [ ] `manifest.yaml` — name, slug, category, language, features, demos, `deployed: false`, `generative_ui`, `interaction_modalities`, and optionally `managed_platform`
|
||||
- [ ] `package.json` — dependencies including `@copilotkit/react-core`, `zod`, `tailwindcss`
|
||||
- [ ] `tsconfig.json`
|
||||
- [ ] `next.config.ts`
|
||||
- [ ] `postcss.config.mjs`
|
||||
|
||||
### App Structure (`src/app/`)
|
||||
|
||||
- [ ] `layout.tsx` — imports `globals.css`, `copilotkit-overrides.css`, `@copilotkit/react-core/v2/styles.css`
|
||||
- [ ] `globals.css` — NO `* { margin: 0; padding: 0; }` reset (only `box-sizing: border-box`)
|
||||
- [ ] `copilotkit-overrides.css` — separate file for CopilotKit class overrides (survives Tailwind v4 purging)
|
||||
- [ ] `api/copilotkit/route.ts` — runtime endpoint
|
||||
- [ ] `api/health/route.ts` — health check endpoint
|
||||
- [ ] `error-boundary.tsx` — DemoErrorBoundary component
|
||||
|
||||
### Demo Pages (`src/app/demos/<feature-id>/page.tsx`)
|
||||
|
||||
One per declared feature. Each demo must:
|
||||
|
||||
- [ ] Use `CopilotKit` provider with `runtimeUrl="/api/copilotkit"` and correct `agent` name
|
||||
- [ ] Use `@copilotkit/react-core/v2` imports (NOT `@copilotkitnext/`)
|
||||
- [ ] For `CopilotChat` demos: wrapper div with `px-6` for horizontal padding (matches Dojo)
|
||||
- [ ] For `CopilotSidebar` demos: no extra padding needed (sidebar has built-in `px-8`)
|
||||
- [ ] Use `h-full` not `h-screen` (demos render in iframes)
|
||||
- [ ] Use inline styles for dynamic content (Tailwind v4 purges classes it can't statically find)
|
||||
- [ ] Include `useConfigureSuggestions` with relevant suggestions
|
||||
|
||||
### Agent Backend
|
||||
|
||||
- [ ] Agent code in `src/agents/` (Python) or `src/lib/` (TypeScript)
|
||||
- [ ] One agent per feature (names must match the `agent` prop in demo pages)
|
||||
- [ ] `langgraph.json` (Python) or equivalent config
|
||||
- [ ] **Pin framework versions** — see "Dependency Pinning" below
|
||||
- [ ] **Per-request `X-AIMock-Strict` forwarding** — the agent's outbound LLM
|
||||
call to aimock MUST carry the inbound `X-AIMock-Strict` (+ `x-test-id`,
|
||||
`x-aimock-context`, `x-diag-*`) headers so a fixture MISS hard-fails
|
||||
instead of silently proxying to the real provider. Forward ONLY headers
|
||||
PRESENT inbound (never hardcode strict on); demo traffic without the
|
||||
header must still proxy. Missing this forwarding shows up as the **D3
|
||||
column flapping** (intermittent amber/red e2e cells) because the rendered
|
||||
answer is non-deterministic real-provider output. Precedent:
|
||||
`integrations/built-in-agent/src/lib/header-forwarding.ts` (ALS +
|
||||
`forwardingFetch`).
|
||||
- [ ] **Two-process integrations** (a Next proxy route in front of a separate
|
||||
agent process): the header forwarder AND the CVDIAG emitter must live
|
||||
**agent-side**, not on the Next route (the Next route is a bare proxy and
|
||||
the transport may drop inbound `x-*` before the model call). The Next route
|
||||
forwards inbound `x-*` onto the proxy POST; the agent process recovers them
|
||||
via a middleware mounted before the framework handler. Worked example:
|
||||
`integrations/strands-typescript/src/agent/{header-forwarding,cvdiag-backend-strands}.ts`.
|
||||
|
||||
### Infrastructure
|
||||
|
||||
- [ ] `Dockerfile` — multi-stage, starts both agent backend and Next.js frontend
|
||||
- [ ] **Two-process integrations: `COPY src/cvdiag` into the runner stage** — if
|
||||
the separate agent process imports the co-located emitter directly (e.g.
|
||||
`../cvdiag/cvdiag-emitter.js`), the `Dockerfile` MUST stage it into the
|
||||
image, e.g. `COPY --chown=app:app src/cvdiag ./src/cvdiag` right after the
|
||||
`COPY --chown=app:app src/agent ./src/agent`. Single-process integrations
|
||||
(`mastra`, `langgraph-typescript`, `claude-sdk-typescript`) get it via
|
||||
Next's `.next` bundling and don't need this. Omitting it passes local d6
|
||||
(where `bin/showcase cvdiag-stage-ts` materializes the emitter) but
|
||||
**crashes at boot in Docker/staging with `ERR_MODULE_NOT_FOUND:
|
||||
.../src/cvdiag/cvdiag-emitter.js`**, so the D6 column never renders.
|
||||
- [ ] `entrypoint.sh` — starts agent server and Next.js, waits for both
|
||||
|
||||
### Testing & QA
|
||||
|
||||
- [ ] `playwright.config.ts`
|
||||
- [ ] `tests/` — one E2E test per demo (basic: load → send message → get response)
|
||||
- [ ] `qa/` — manual QA checklist per demo
|
||||
- [ ] **CVDIAG instrumentation staged** — add the slug to
|
||||
`_CVDIAG_TS_INTEGRATIONS` in `scripts/cli/cmd-cvdiag-stage-ts.sh`, run
|
||||
`bin/showcase cvdiag-stage-ts`, and verify `bin/showcase cvdiag-stage-ts
|
||||
--check` exits 0 with zero drift (stages the co-located `src/cvdiag/`
|
||||
emitter into the standalone build context).
|
||||
- [ ] **CVDIAG backend emitter wired** — the backend emits the 11 `backend.*`
|
||||
boundaries and persists them to the `cvdiag_events` PocketBase collection
|
||||
(`CVDIAG_BACKEND_EMITTER` / `CVDIAG_PB_URL` / `CVDIAG_WRITER_KEY` set on the
|
||||
service env). The emitter adopts the inbound `x-test-id` as the cross-layer
|
||||
JOIN key so its rows join the probe's. Verify with `bin/showcase cvdiag
|
||||
classify <test-id>` returning non-empty after a probe run. For two-process
|
||||
integrations the emitter lives **agent-side** (see Agent Backend above).
|
||||
|
||||
### Assets
|
||||
|
||||
- [ ] Logo SVG at `showcase/shell/public/logos/<slug>.svg`
|
||||
|
||||
---
|
||||
|
||||
## Source of Truth: `examples/integrations/*` vs `showcase/integrations/*`
|
||||
|
||||
Two directories hold integration code, and they play different roles. Understanding the relationship is critical before adding or modifying a package.
|
||||
|
||||
### Roles
|
||||
|
||||
- **`examples/integrations/<name>/`** — the **Dojo example**. This is the dep-pinning source of truth: minimal, focused agent code used to prove a framework works against CopilotKit/AG-UI. The weekly drift-detection workflow and the "Always pin agent framework and SDK versions to exact versions from the working Dojo example" rule (see "Dependency Pinning" below) both treat this directory as canonical.
|
||||
- **`showcase/integrations/<slug>/`** — the **full triple-duty integration**:
|
||||
1. Partner-facing demo (lives on `showcase.copilotkit.dev`)
|
||||
2. Cloneable starter source (extracted on-demand via `extract-starter.ts`)
|
||||
3. Iframe-embedded experience inside the public showcase shell
|
||||
|
||||
### Automation Direction (one-way)
|
||||
|
||||
```
|
||||
examples/integrations/<name>/ ──(migrate-integration-examples.ts)──▶ showcase/integrations/<slug>/src/agents/
|
||||
showcase/integrations/<slug>/ ──(extract-starter.ts)────────────────▶ standalone starter (on-demand)
|
||||
```
|
||||
|
||||
- `showcase/scripts/migrate-integration-examples.ts` copies agent code **from** `examples/integrations/<name>/` **into** `showcase/integrations/<slug>/src/agents/`. It never runs in reverse.
|
||||
- `showcase/scripts/extract-starter.ts` extracts a clean standalone starter from any integration on demand, dereferencing symlinks and stripping test/CI artifacts.
|
||||
- Do not hand-edit agent code inside `showcase/integrations/<slug>/src/agents/` if the package has a Dojo counterpart — fix it upstream in `examples/integrations/<name>/` and re-run the migration script.
|
||||
|
||||
### Born-in-Showcase Packages (no Dojo counterpart)
|
||||
|
||||
Five packages exist only in showcase and have no `examples/integrations/<name>/` sibling:
|
||||
|
||||
- `ag2`
|
||||
- `claude-sdk-python`
|
||||
- `claude-sdk-typescript`
|
||||
- `langroid`
|
||||
- `spring-ai`
|
||||
|
||||
These are authored directly in `showcase/integrations/<slug>/` and are **exempt from the pin-to-Dojo rule** — there is no Dojo to pin to. They still must pin exact versions (see "Dependency Pinning"), but the reference is whatever the framework's own examples or release notes recommend, not a sibling `examples/integrations/` directory.
|
||||
|
||||
### Slug Aliasing
|
||||
|
||||
Several packages have different names in `examples/integrations/` vs `showcase/integrations/`. The aliasing is historical — showcase standardized on shorter, marketing-friendly slugs while the Dojo kept the original framework-canonical names.
|
||||
|
||||
| `showcase/integrations/` slug | `examples/integrations/` name | Why different |
|
||||
| ----------------------------- | ----------------------------- | --------------------------------------------------------- |
|
||||
| `google-adk` | `adk` | Showcase prefixes with vendor for disambiguation |
|
||||
| `langgraph-typescript` | `langgraph-js` | Showcase prefers full language name (`-typescript`) |
|
||||
| `ms-agent-dotnet` | `ms-agent-framework-dotnet` | Showcase shortens `-framework-` out of the slug |
|
||||
| `ms-agent-python` | `ms-agent-framework-python` | Same — shorter slug in showcase |
|
||||
| `strands` | `strands-python` | Showcase drops the language suffix (no TS variant exists) |
|
||||
|
||||
When running `migrate-integration-examples.ts` or reasoning about drift, remember that the script internally maps these aliases — don't "fix" them by renaming one side.
|
||||
|
||||
---
|
||||
|
||||
## B. External Setup (after the package is ready)
|
||||
|
||||
This section is the **single-shot** bring-up: provision the prod Railway
|
||||
service immediately, then go live. If instead you ship the integration
|
||||
**staging-only first** and defer prod ("promote later"), follow
|
||||
[`./RAILWAY.md`](./RAILWAY.md) → "Promoting a Staging-Only Integration to
|
||||
Production" for the provision-prod-instance + SSOT-gate-flip + promote path
|
||||
(and note: the promote pipeline does NOT provision a new prod service — D6
|
||||
false-reds the whole column until the prod instance exists).
|
||||
|
||||
### 1. Railway Service
|
||||
|
||||
- [ ] Create service in the **CopilotKit Showcase** Railway project, **US-West** region
|
||||
- [ ] Type: **Docker** (image from GHCR, not source build)
|
||||
- [ ] Image URL: `ghcr.io/copilotkit/showcase-<slug>:latest`
|
||||
- [ ] Health check path: `/api/health`
|
||||
- [ ] Link shared variables group (contains API keys)
|
||||
- [ ] Set `NODE_ENV=production`, `NEXT_PUBLIC_BASE_URL=https://showcase.copilotkit.dev`
|
||||
|
||||
### 2. GitHub Secrets
|
||||
|
||||
- [ ] Ensure `RAILWAY_TOKEN` secret exists in the repo
|
||||
|
||||
### 3. CI/CD Workflow (`.github/workflows/showcase_deploy.yml`)
|
||||
|
||||
- [ ] Add slug to `workflow_dispatch.inputs.service.options`
|
||||
- [ ] Add change detection filter for `showcase/integrations/<slug>/**`
|
||||
- [ ] Add build job: build Docker image → push to GHCR → trigger Railway deploy
|
||||
- [ ] Wire up the `RAILWAY_TOKEN` secret
|
||||
|
||||
### 4. Registry
|
||||
|
||||
- [ ] Run `npx tsx showcase/scripts/generate-registry.ts` to regenerate `registry.json`
|
||||
- [ ] Verify the integration appears on the Integrations page
|
||||
- [ ] Verify demos load in the drawer (Preview tab)
|
||||
|
||||
### 5. Go Live
|
||||
|
||||
- [ ] Verify Railway service is healthy: `curl https://showcase-<slug>-production.up.railway.app/api/health`
|
||||
- [ ] Verify all demos respond: visit each `/demos/<id>` route
|
||||
- [ ] Set `deployed: true` in `manifest.yaml`
|
||||
- [ ] Verify constraint validation passes: `npx tsx showcase/scripts/validate-constraints.ts <slug>`
|
||||
- [ ] Regenerate registry: `npx tsx showcase/scripts/generate-registry.ts`
|
||||
- [ ] Commit and push — stack nav chip will light up automatically
|
||||
|
||||
### 6. Shell Updates (usually automatic)
|
||||
|
||||
- [ ] If the framework name in the stack nav differs from `manifest.yaml` name, verify `startsWith` matching works
|
||||
- [ ] Demo content (Code/Docs tabs): run `npx tsx showcase/scripts/generate-demo-content.ts` if it exists
|
||||
|
||||
---
|
||||
|
||||
## Dependency Pinning
|
||||
|
||||
**Always pin agent framework and SDK versions to exact versions from the working Dojo example.** Do not use floating ranges like `>=0.3.0` — they resolve to different versions over time and silently break APIs.
|
||||
|
||||
**Why this matters:** `langchain>=0.3.0` resolved to 0.3.x which lacked `create_agent`. The Dojo uses `langchain==1.2.0` where it exists. A floating range that worked at scaffold time broke on the next Docker build when a different version was pulled.
|
||||
|
||||
**What to pin:**
|
||||
|
||||
- Agent framework packages (langchain, langgraph, @mastra/core, etc.)
|
||||
- CopilotKit SDK packages (copilotkit, @copilotkit/runtime, etc.)
|
||||
- LLM provider SDKs (langchain-openai, @ai-sdk/openai, etc.)
|
||||
|
||||
**What can float:**
|
||||
|
||||
- Standard utilities (zod, react, next) — these have stable APIs
|
||||
- Dev dependencies (playwright, typescript, tailwind)
|
||||
|
||||
**Where to find correct versions:**
|
||||
|
||||
- Check the corresponding Dojo example at `examples/integrations/<slug>/`
|
||||
- Use exact versions from its `requirements.txt` / `pyproject.toml` / `package.json`
|
||||
- The weekly drift detection workflow will flag when pinned versions fall behind
|
||||
|
||||
---
|
||||
|
||||
## LangGraph: Prebuilt vs Node-Based
|
||||
|
||||
LangGraph supports two agent authoring styles, and showcase uses both. When touching a LangGraph package — or adding a new one — decide the style explicitly and match the existing sibling's idioms.
|
||||
|
||||
### The Two Styles
|
||||
|
||||
- **Node-based** — hand-rolled `StateGraph` with `addNode(...)`, explicit edges, and custom routing logic. Maximum control; more code to maintain.
|
||||
- **Prebuilt** — `create_react_agent` / `create_agent` helpers that wrap the common ReAct pattern. Minimal code; less flexibility.
|
||||
|
||||
### Current Showcase State
|
||||
|
||||
| Package | Style | Evidence |
|
||||
| -------------------------------------------- | ---------- | ----------------------------------------------------- |
|
||||
| `showcase/integrations/langgraph-python` | Prebuilt | `create_react_agent` in `src/agents/main.py:53` |
|
||||
| `showcase/integrations/langgraph-fastapi` | Prebuilt | `create_react_agent` in `src/agents/src/agent.py:166` |
|
||||
| `showcase/integrations/langgraph-typescript` | Node-based | `StateGraph` in `src/agent/graph.ts:271` |
|
||||
|
||||
### Dojo Coverage Gap
|
||||
|
||||
The `ag-ui/apps/dojo/` e2e tests exclusively exercise **node-based** graphs. This means prebuilt-agent coverage is thin in the Dojo even though two of the three LangGraph packages users clone from showcase are prebuilt.
|
||||
|
||||
Cross-reference the action inventory for the full breakdown of which AG-UI features are exercised where: <https://www.notion.so/3443aa38185281b5a1dfc6e0890264e1>.
|
||||
|
||||
### Guidance
|
||||
|
||||
- **When adding a new LangGraph-based package**, decide the authoring style explicitly and match the idioms of the corresponding showcase sibling (Python → prebuilt, TypeScript → node-based) unless you have a concrete reason to diverge.
|
||||
- If you do diverge, document why in the package's README and add an entry to the table above.
|
||||
- Do not silently convert a package between styles — it's a public API change for anyone who cloned the starter.
|
||||
|
||||
This distinction only applies to LangGraph today. Other frameworks (CrewAI, Mastra, etc.) have their own framework-specific authoring idioms — out of scope for this section.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference: Common Gotchas
|
||||
|
||||
| Gotcha | Fix |
|
||||
| --------------------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| CSS classes purged by Tailwind v4 | Put CopilotKit overrides in `copilotkit-overrides.css`, not `globals.css` |
|
||||
| `* { margin: 0; padding: 0; }` | NEVER use this reset — it strips CopilotKit's internal padding |
|
||||
| Chat messages flush to edges | Add `px-6` to the CopilotChat wrapper div |
|
||||
| `h-screen` in demos | Use `h-full` — demos render inside iframes |
|
||||
| Dynamic content unstyled | Use inline `style={}` not Tailwind classes for agent-generated content |
|
||||
| Stale lockfile | Run `pnpm install` after changing `package.json`, commit the lockfile |
|
||||
| Stack chip not lighting up | Check `deployed: true` in manifest and registry name matching |
|
||||
| Agent import errors in Docker | Pin framework deps to exact Dojo versions — floating ranges resolve differently over time |
|
||||
@@ -0,0 +1,96 @@
|
||||
Copyright 2024-2026 CopilotKit. All rights reserved.
|
||||
|
||||
Licensor: CopilotKit
|
||||
Licensed Work: CopilotKit Showcase
|
||||
Contact: licensing@copilotkit.ai
|
||||
|
||||
|
||||
CopilotKit Source Available License 1.0
|
||||
|
||||
|
||||
## Acceptance
|
||||
|
||||
By accessing, viewing, or using the software, you agree to all of the terms and
|
||||
conditions below.
|
||||
|
||||
|
||||
## Permitted Use
|
||||
|
||||
The licensor grants you a non-exclusive, non-transferable, non-sublicensable,
|
||||
revocable right to view and study the software for personal, non-commercial
|
||||
purposes only, such as learning and evaluation.
|
||||
|
||||
|
||||
## Restrictions
|
||||
|
||||
You may not use the software, or any part of it, for any commercial purpose
|
||||
without a separate written license from the licensor.
|
||||
|
||||
You may not use the software to build, design, train, or inform the development
|
||||
of any product, service, or system that provides functionality similar to or
|
||||
competitive with the software or any product or service offered by the licensor.
|
||||
This restriction applies whether or not the resulting work copies any code from
|
||||
the software, and whether the development is direct, assisted by automated
|
||||
tools, or performed as a clean-room implementation informed by knowledge gained
|
||||
from the software.
|
||||
|
||||
You may not copy, modify, merge, distribute, sublicense, or create derivative
|
||||
works of the software, except as expressly permitted above.
|
||||
|
||||
You may not remove or alter any licensing, copyright, or other notices in the
|
||||
software. Any use of the licensor's trademarks is subject to applicable law.
|
||||
|
||||
|
||||
## Commercial Licensing
|
||||
|
||||
Use of the software for any purpose not expressly permitted above requires a
|
||||
commercial license from the licensor. Contact the licensor at the address above.
|
||||
|
||||
|
||||
## No Patents
|
||||
|
||||
No patent rights are granted under these terms.
|
||||
|
||||
|
||||
## Notices
|
||||
|
||||
You must ensure that anyone who gets a copy of any part of the software from you
|
||||
also gets a copy of these terms.
|
||||
|
||||
|
||||
## No Other Rights
|
||||
|
||||
These terms do not grant any rights other than those expressly stated above. All
|
||||
rights not expressly granted are reserved by the licensor.
|
||||
|
||||
|
||||
## Termination
|
||||
|
||||
If you violate any of these terms, your rights under this license terminate
|
||||
immediately and permanently. The licensor may also terminate your rights at any
|
||||
time for any reason by providing written notice.
|
||||
|
||||
|
||||
## No Liability
|
||||
|
||||
*As far as the law allows, the software comes as is, without any warranty or
|
||||
condition, and the licensor will not be liable to you for any damages arising
|
||||
out of these terms or the use or nature of the software, under any kind of
|
||||
legal claim.*
|
||||
|
||||
|
||||
## Definitions
|
||||
|
||||
The **licensor** is the entity offering these terms, and the **software** is the
|
||||
software the licensor makes available under these terms, including any portion
|
||||
of it.
|
||||
|
||||
**you** refers to the individual or entity agreeing to these terms.
|
||||
|
||||
**your company** is any legal entity, sole proprietorship, or other kind of
|
||||
organization that you work for, plus all organizations that have control over,
|
||||
are under the control of, or are under common control with that organization.
|
||||
|
||||
**commercial purpose** means any use intended for or directed toward commercial
|
||||
advantage or monetary compensation, including use within or on behalf of a
|
||||
for-profit organization.
|
||||
@@ -0,0 +1,439 @@
|
||||
# Showcase Railway Operations
|
||||
|
||||
Tagline: fleet-wide auto-update config, pending service provisioning, and the
|
||||
recipe for adding a new Railway service. For day-to-day promote/snapshot/pin
|
||||
operations see [`./bin/README.md`](./bin/README.md). For aimock-specific
|
||||
service reconstruction see [`./aimock/RAILWAY.md`](./aimock/RAILWAY.md).
|
||||
|
||||
## built-in-agent service
|
||||
|
||||
`built-in-agent` → image `showcase-built-in-agent` is fully provisioned in the
|
||||
`ALL_SERVICES` matrix in `showcase_build.yml` (real `railway_id`
|
||||
`f4f8371a-bc46-45b2-b6d4-9c9af608bdbf`; `ciBuilt`/`gateValidated` set in
|
||||
`showcase/scripts/railway-envs.ts`).
|
||||
|
||||
Single-service Next.js app (`BuiltInAgent` runs in-process; no separate
|
||||
agent server). Required env: `OPENAI_API_KEY`. Health probe at
|
||||
`/api/health`.
|
||||
|
||||
The matching `starter-built-in-agent` is intentionally absent from the build
|
||||
matrix: the starter tooling (`showcase/scripts/extract-starter.ts`,
|
||||
`provision-starter-fleet.ts`) does not yet support single-service packages, so
|
||||
the starter will be added in a follow-up PR alongside that support.
|
||||
|
||||
## Auto-Updates (Fleet-Wide)
|
||||
|
||||
Most image-sourced Railway services have `source.autoUpdates.type = "minor"`
|
||||
(24 of the 40 services in the production environment as of this writing); the
|
||||
12 `starter-*` services and a handful of others (incl. `showcase-built-in-agent`,
|
||||
`harness-workers`, `showcase-ms-agent-harness-dotnet`, `webhooks`) currently
|
||||
have none. Most of the `minor` services carry no `source.autoUpdates.schedule`
|
||||
at all, so updates apply immediately whenever a new digest lands; only `aimock`
|
||||
carries a `schedule` array (covering all hours, every day — operationally
|
||||
equivalent to no schedule). When a new GHCR `:latest` digest is pushed, Railway
|
||||
auto-pulls and redeploys those services without manual intervention.
|
||||
|
||||
CI (`showcase_build.yml`, "Build & Push") still triggers an explicit
|
||||
`serviceInstanceRedeploy` (via `redeploy-env.ts`) after each GHCR push for
|
||||
deterministic health-checking; `showcase_deploy.yml` ("Verify Deploy") then
|
||||
health-checks that redeployment. The auto-update is a safety net, not the
|
||||
primary deploy path.
|
||||
|
||||
## Adding a New Railway Service
|
||||
|
||||
1. **Enable auto-updates** via the GraphQL API:
|
||||
|
||||
```graphql
|
||||
mutation {
|
||||
environmentPatchCommit(
|
||||
environmentId: "<env-id>"
|
||||
patch: {
|
||||
"services": {
|
||||
"<new-service-id>": {
|
||||
"source": { "autoUpdates": { "type": "minor" } }
|
||||
}
|
||||
}
|
||||
}
|
||||
commitMessage: "Enable image auto-updates"
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
Or via Dashboard: Settings > Configure Auto Updates > "Automatically
|
||||
update to the latest tag" + "At any time, immediately".
|
||||
|
||||
2. **Add to `showcase_build.yml`** `ALL_SERVICES` matrix so CI builds
|
||||
and pushes the GHCR image on code changes.
|
||||
|
||||
3. **No `smoke.yml` edit needed for a normal `showcase-*` service** —
|
||||
`showcase/harness/config/probes/smoke.yml` is auto-discovery driven and
|
||||
picks up any new `showcase-*` service on the next tick. Only edit its
|
||||
`filter.nameExcludes` to EXCLUDE an infra / non-runtime service.
|
||||
|
||||
4. **Git-based services**: auto-updates only apply to image-sourced
|
||||
services. Skip step 1 for git-deploy services.
|
||||
|
||||
## Promoting a Staging-Only Integration to Production
|
||||
|
||||
### When this applies
|
||||
|
||||
You added an integration **staging-only on purpose** — it ships to staging
|
||||
first and its prod instance is deferred to "promote later." In the SSOT
|
||||
(`showcase/scripts/railway-envs.ts`) such an entry looks like the
|
||||
`showcase-strands-typescript` block did before [PR #5705](https://github.com/CopilotKit/CopilotKit/pull/5705):
|
||||
|
||||
- `gateValidated: false`,
|
||||
- `gateIgnore: true`,
|
||||
- an `environments:` map containing **only `staging`** (no `prod` block, so no
|
||||
prod `serviceInstance` ID exists),
|
||||
- a `legacyJsonCompat.domains.prod` placeholder pointing at the **borrowed
|
||||
staging host**, purely to keep the generated JSON's legacy `{prod, staging}`
|
||||
shape (it is never dereferenced by any TS accessor).
|
||||
|
||||
This is the worked example to follow — `showcase-strands-typescript` was
|
||||
promoted exactly this way in PR #5705.
|
||||
|
||||
### The critical gotcha (read this first)
|
||||
|
||||
**The promote pipeline only promotes image digests to a prod service that
|
||||
ALREADY exists — it does NOT provision a new prod `serviceInstance`.** Both the
|
||||
promote workflow (`showcase_promote.yml`, "Showcase: Promote (staging → prod)")
|
||||
and `bin/railway promote` move the staging-tested `@sha256` digest onto an
|
||||
existing prod instance; neither has a "create the prod service" step (there is
|
||||
no provisioning subcommand in `bin/railway`). So a staging-only integration
|
||||
will **never** appear in prod just by running promote.
|
||||
|
||||
Until the prod `serviceInstance` exists, **D6 false-reds the entire column**:
|
||||
the harness has no `health:<slug>` record for prod, so the per-cell probe is
|
||||
handed an empty `backendUrl`, Playwright calls `page.goto("/demos/…")` on a
|
||||
bare relative path, and Chromium rejects it as an invalid URL —
|
||||
`errorClass=goto-error` on _every_ cell (column-wide, uniform `fail_count`).
|
||||
The fix is not a code fix; it is provisioning the missing prod instance and
|
||||
flipping the SSOT gate.
|
||||
|
||||
### Ordered checklist
|
||||
|
||||
1. **Provision the prod Railway `serviceInstance`.** This is out-of-band (no
|
||||
`bin/railway` subcommand covers it; see [`./bin/README.md`](./bin/README.md),
|
||||
which defers "new-service provisioning" to this doc). Use the GraphQL
|
||||
staged-change primitive, mirroring a peer prod TypeScript showcase service
|
||||
(PR #5705 mirrored `showcase-claude-sdk-typescript`):
|
||||
- `environmentStageChanges(production, …)` — stage a `services.<svc>` block
|
||||
copied from the peer: `source.image` (pinned `@sha256` digest with
|
||||
`autoUpdates.minor`), `networking.serviceDomains.<prod-domain>`,
|
||||
`build.builder RAILPACK`, and a `deploy` block (reused GHCR
|
||||
`registryCredentials`, runtime V2, `healthcheckPath: /api/health`,
|
||||
`multiRegionConfig`).
|
||||
- `environmentPatchCommitStaged(production, <msg>)` — commit the staged
|
||||
change; this **materializes** the prod `serviceInstance` (in PR #5705,
|
||||
`8a50728e-6119-43c4-b59c-d9535b6717a4`).
|
||||
- Deploy it (`serviceInstanceDeployV2`) and poll the deployment to
|
||||
`SUCCESS`.
|
||||
|
||||
2. **Edit the SSOT (`showcase/scripts/railway-envs.ts`)** — convert the entry
|
||||
to the dual-env `showcase-strands` shape:
|
||||
- add a `prod` env block under `environments:` with the **real**
|
||||
`instanceId`, `healthcheckPath: "/api/health"`, the prod `domain`, and
|
||||
`probe: true`;
|
||||
- set `gateValidated: true` (per the `gateValidated` doc in that file, new
|
||||
SSOT services MUST land `gateValidated: true`; `gateIgnore` is only for
|
||||
"deliberately-untracked third-party / domainless / single-env services" —
|
||||
a prod-promoted demo is none of those);
|
||||
- **remove** `gateIgnore: true`;
|
||||
- **remove** the `legacyJsonCompat` prod-domain placeholder (the borrowed
|
||||
staging host);
|
||||
- update the leading comment to reflect the dual-env state.
|
||||
|
||||
See the PR #5705 diff on this file for the exact before/after.
|
||||
|
||||
3. **Regenerate the derived artifacts and run the gate:**
|
||||
- `npx tsx showcase/scripts/emit-railway-envs-json.ts` — regenerate
|
||||
`railway-envs.generated.json` (CI verifies with `--check`).
|
||||
- Regenerate the golden fixture
|
||||
`showcase/scripts/__tests__/fixtures/railway-envs.golden.json` so the new
|
||||
prod `(service, env)` pair is captured — this is an **intentional**
|
||||
behavior change, not a refactor regression
|
||||
(`railway-envs.golden.test.ts` is a behavior-preservation guard).
|
||||
- `npx tsx showcase/scripts/sync-promote-service-options.ts` — regenerate
|
||||
the `showcase_promote.yml` workflow_dispatch dropdown so the slug becomes
|
||||
a promote target (CI verifies with `--check`).
|
||||
- `npx tsx showcase/scripts/verify-railway-image-refs.ts` — run the image-ref
|
||||
gate; with `gateValidated: true` it now validates the prod pin too.
|
||||
- Run the scripts test suite (`pnpm exec vitest run` from `showcase/`),
|
||||
including `verify-railway-image-refs.test.ts` and `redeploy-env.test.ts`,
|
||||
whose gate-target / redeploy-scope counts and "staging-only" comments
|
||||
change when the entry flips dual-env.
|
||||
|
||||
4. **Secrets.** A prod TypeScript integration gets its provider keys
|
||||
(`OPENAI_API_KEY` / `ANTHROPIC_API_KEY`, and `OPENAI_BASE_URL` for
|
||||
aimock-routed agents) from the **prod env's variable set**, mirroring the
|
||||
peer prod service's config — set them on the new prod instance, never inline
|
||||
a secret value in the SSOT or in a commit. If the agent routes 100% to
|
||||
aimock (the `serviceRefs: [{ key: "OPENAI_BASE_URL", target: "aimock" }]`
|
||||
case), `OPENAI_BASE_URL` points at the **prod** aimock origin and the
|
||||
`OPENAI_API_KEY` is the non-secret `sk-aim…` aimock placeholder — so no real
|
||||
prod secret is sourced. The `OPENAI_BASE_URL` service-ref is asserted
|
||||
prod→prod by the promote preflight (never copied across envs).
|
||||
|
||||
5. **Verify GREEN.** After the prod instance is up:
|
||||
- prod `/api/health` returns **200**
|
||||
(`https://showcase-<slug>-production.up.railway.app/api/health`);
|
||||
- the prod PocketBase `health` collection gains a `health:<slug>` record
|
||||
(`dimension="health"`, `status:200`, a real prod `url`);
|
||||
- the D6 column flips on the prod harness's **next hourly
|
||||
`d6-all-pills-e2e` tick** (runs at `:40`). The probe needs the harness to
|
||||
have discovered the new prod health record first, so expect up to ~1 hour
|
||||
of lag — the column stays red until the next tick even though the service
|
||||
is healthy. Don't panic about that lag; confirm health (200 + the
|
||||
PocketBase record) as the discriminating GREEN signal, then let the tick
|
||||
clear the cells.
|
||||
|
||||
Once promoted, run the digest promote itself the normal way —
|
||||
`showcase_promote.yml` (now listing the slug) or `bin/railway promote`; see
|
||||
[`./bin/README.md`](./bin/README.md) "Worked example: promote staging →
|
||||
production".
|
||||
|
||||
### CVDIAG instrumentation + per-request `X-AIMock-Strict` forwarding (REQUIRED)
|
||||
|
||||
Any integration being **added or promoted** MUST also be wired for
|
||||
flap-observability (CVDIAG) and per-request header forwarding, or its D6 column
|
||||
can silently degrade. Two non-optional steps:
|
||||
|
||||
1. **CVDIAG backend instrumentation.** Add the slug to
|
||||
`_CVDIAG_TS_INTEGRATIONS` in `scripts/cli/cmd-cvdiag-stage-ts.sh` and run
|
||||
`bin/showcase cvdiag-stage-ts` (then `--check`, which must exit 0 with zero
|
||||
drift). This stages the co-located `src/cvdiag/` emitter into the
|
||||
integration's standalone build context. Then WIRE the emitter so backend
|
||||
`backend.*` boundaries actually emit and persist to the `cvdiag_events`
|
||||
PocketBase collection (set `CVDIAG_BACKEND_EMITTER`, `CVDIAG_PB_URL`,
|
||||
`CVDIAG_WRITER_KEY` on the prod env's variable set, mirroring the local
|
||||
compose service). Without backend rows, `bin/showcase cvdiag classify` has
|
||||
nothing to classify and a flap cannot be diagnosed.
|
||||
|
||||
2. **Per-request `X-AIMock-Strict` forwarding.** The probe sends
|
||||
`X-AIMock-Strict: true` (+ `x-test-id`, `x-aimock-context`, `x-diag-*`) on
|
||||
every request so a fixture MISS becomes a HARD FAILURE instead of silently
|
||||
proxying to the real provider. The integration's outbound LLM call to aimock
|
||||
MUST carry that header through. If it does not, a fixture miss falls through
|
||||
and a stale/drifted answer renders as a PASS — the classic symptom is the
|
||||
**D3 column flapping** (an e2e cell intermittently going amber/red) because
|
||||
the rendered answer is non-deterministic real-provider output rather than the
|
||||
pinned fixture. Forward ONLY headers PRESENT inbound (never hardcode strict
|
||||
on) so ordinary demo traffic still proxies normally.
|
||||
|
||||
**Two-process caveat.** For a two-process integration (a Next proxy route in
|
||||
front of a separate agent process — e.g. `strands-typescript`,
|
||||
`claude-sdk-typescript`, where the Next route is a bare `HttpAgent` proxy and
|
||||
the model call happens in the agent process), the CVDIAG emitter AND the header
|
||||
forwarder must live **agent-side**, not on the Next route. Wrapping the Next
|
||||
route would instrument the proxy hop, not the real model call, and the AG-UI
|
||||
transport may drop inbound `x-*` before `agent.run()` (e.g.
|
||||
`@ag-ui/aws-strands` reads only `req.body` + `accept`). The seams are: (a) the
|
||||
Next route forwards inbound `x-*` onto the proxy POST (HttpAgent `fetch`
|
||||
option + an `AsyncLocalStorage` snapshot), and (b) the agent process recovers
|
||||
them via a middleware mounted before the framework handler, seeds an
|
||||
`AsyncLocalStorage`, and the model client's `fetch` override injects them on the
|
||||
outbound aimock call. See `integrations/strands-typescript/src/agent/{header-forwarding,cvdiag-backend-strands}.ts`
|
||||
for the worked two-process example, and `integrations/built-in-agent/src/lib/header-forwarding.ts`
|
||||
for the in-process precedent.
|
||||
|
||||
**Two-process Docker staging (REQUIRED).** When the separate agent process
|
||||
imports the co-located emitter directly (e.g. `../cvdiag/cvdiag-emitter.js`),
|
||||
the integration's `Dockerfile` MUST `COPY src/cvdiag` into the runner stage so
|
||||
the emitter ships in the image — e.g. `COPY --chown=app:app src/cvdiag
|
||||
./src/cvdiag` immediately after the `COPY --chown=app:app src/agent ./src/agent`.
|
||||
Single-process integrations (`mastra`, `langgraph-typescript`,
|
||||
`claude-sdk-typescript`) get the emitter via Next's `.next` bundling and do NOT
|
||||
need this extra COPY. **Symptom if omitted:** the image passes local d6 — where
|
||||
`bin/showcase cvdiag-stage-ts` materializes the emitter into the working tree —
|
||||
but **crashes at boot in Docker/staging with `ERR_MODULE_NOT_FOUND:
|
||||
.../src/cvdiag/cvdiag-emitter.js`**, so the agent never starts and the D6 column
|
||||
never renders.
|
||||
|
||||
> **Related:** for the _single-shot_ "create prod service → go live"
|
||||
> bring-up (where prod is provisioned immediately, with no staging-first
|
||||
> phase), see [`./INTEGRATION-CHECKLIST.md`](./INTEGRATION-CHECKLIST.md) §B.
|
||||
> This section is the **staging-first → promote-later** counterpart.
|
||||
>
|
||||
> TODO: `INTEGRATION-CHECKLIST.md` §B.3 still names `showcase_deploy.yml` as
|
||||
> the build/push workflow to edit; the build/push matrix has since moved to
|
||||
> `showcase_build.yml` ("Build & Push"), with `showcase_deploy.yml` now the
|
||||
> staging verify gate. Correct §B.3 in a follow-up.
|
||||
|
||||
## harness-workers Replica Count (Worker Provisioning)
|
||||
|
||||
The `harness-workers` fleet provisioning is tracked in the SSOT at
|
||||
`showcase/scripts/railway-envs.ts` under the `harness-workers` entry's
|
||||
`workerProvisioning` field. The `railway-envs.generated.json` snapshot
|
||||
captures these values for CI drift detection.
|
||||
|
||||
### Worker model (1-worker-per-replica)
|
||||
|
||||
Railway runs **one worker process per replica container** (keyed on `HOSTNAME`).
|
||||
There is no per-process forking. The live worker count equals the **effective
|
||||
replica count** strictly 1:1. `HARNESS_POOL_COUNT` is an **informational-only**
|
||||
control-plane hint — it does NOT fork additional workers. The authoritative
|
||||
per-worker concurrency knob is `BROWSER_POOL_MAX_CONTEXTS`.
|
||||
|
||||
### Effective replica count — `multiRegionConfig`, not top-level `numReplicas`
|
||||
|
||||
`harness-workers` is a single-region service (`us-west2`). Railway derives the
|
||||
LIVE running replica count from the per-region
|
||||
`multiRegionConfig.us-west2.numReplicas` field — **this is the effective knob the
|
||||
deploy honors**. The top-level `numReplicas` is a legacy aggregate that Railway
|
||||
keeps in sync with the region sum, but it is NOT the field that drives reality.
|
||||
The SSOT therefore models the effective count as `effectiveReplicas`
|
||||
(= `multiRegionConfig.us-west2.numReplicas`) and keeps the top-level
|
||||
`numReplicas` only as a documented mirror. The CI drift gate asserts
|
||||
`effectiveReplicas`.
|
||||
|
||||
### Current declared values (live reality as of 2026-06-26)
|
||||
|
||||
Verified live via the Railway GraphQL `environment.config` staged-config read —
|
||||
both envs carry `deploy.multiRegionConfig = {"us-west2":{"numReplicas":6}}`.
|
||||
|
||||
| Env | `effectiveReplicas` (= `multiRegionConfig.us-west2.numReplicas`, live workers) | `numReplicas` (mirror) | `BROWSER_POOL_MAX_CONTEXTS` |
|
||||
| ------- | ------------------------------------------------------------------------------ | ---------------------- | --------------------------- |
|
||||
| prod | 6 | 6 | 40 |
|
||||
| staging | 6 | 6 | 40 |
|
||||
|
||||
**Prod/staging parity achieved**: B-reconcile scaled prod `harness-workers`
|
||||
from 3 → 6 replicas (updating BOTH the top-level `numReplicas` AND
|
||||
`multiRegionConfig.us-west2.numReplicas`) to match staging (6). Prod and staging
|
||||
are now at parity (6/6). The earlier prod=3 state and the prior staging
|
||||
config-field-vs-live drift (config field `2` / `6` live) are both resolved — the
|
||||
live staged config now reads `6` in both envs.
|
||||
|
||||
### SSOT fields
|
||||
|
||||
- `workerProvisioning.{prod,staging}.effectiveReplicas` — AUTHORITATIVE worker
|
||||
count (= `multiRegionConfig.us-west2.numReplicas`, the field Railway honors;
|
||||
1:1 with live workers). This is the field the drift gate watches and the ONLY
|
||||
field that drives the live replica count.
|
||||
- `workerProvisioning.{prod,staging}.numReplicas` — top-level Railway field,
|
||||
retained as a DOCUMENTED MIRROR of `effectiveReplicas` (equal on a
|
||||
single-region service). Not an authoritative knob.
|
||||
- `workerProvisioning.{prod,staging}.BROWSER_POOL_MAX_CONTEXTS` — per-worker
|
||||
Playwright context budget.
|
||||
- `workerProvisioning.{prod,staging}.HARNESS_POOL_COUNT` — INFORMATIONAL ONLY;
|
||||
records what the `HARNESS_POOL_COUNT` env var is set to on Railway for audit
|
||||
visibility. Never use this as a worker count or fork factor.
|
||||
- `workerProvisioning.{prod,staging}.overlapSeconds` — deploy-rollover capacity
|
||||
floor (= Railway `serviceInstance.overlapSeconds`, env mirror
|
||||
`RAILWAY_DEPLOYMENT_OVERLAP_SECONDS`). See **Deploy rollover** below.
|
||||
- `workerProvisioning.{prod,staging}.drainingSeconds` — graceful-drain window
|
||||
(= Railway `serviceInstance.drainingSeconds`, env mirror
|
||||
`RAILWAY_DEPLOYMENT_DRAINING_SECONDS`). See **Deploy rollover** below.
|
||||
|
||||
### Applying a replica count change (MANUAL)
|
||||
|
||||
The `emit-railway-envs-json.ts` emitter and `bin/railway` tooling are
|
||||
**VERIFY-ONLY** with respect to the replica count — they do not write replica
|
||||
counts to Railway. To change the replica count:
|
||||
|
||||
1. Change the `effectiveReplicas` value in `railway-envs.ts` (SSOT) — and the
|
||||
`numReplicas` mirror alongside it (keep them equal for this single-region
|
||||
service).
|
||||
2. Regenerate the snapshot: `npx tsx showcase/scripts/emit-railway-envs-json.ts`
|
||||
3. Commit both files (`railway-envs.ts` + `railway-envs.generated.json`).
|
||||
4. Apply the change to Railway manually via the Railway Dashboard (Service >
|
||||
Settings > Replicas, which edits `multiRegionConfig.us-west2.numReplicas`) or
|
||||
the Railway GraphQL API.
|
||||
|
||||
The CI drift gate (`showcase/scripts/__tests__/harness-workers-provisioning.test.ts`)
|
||||
will fail if `railway-envs.ts` and `railway-envs.generated.json` disagree on
|
||||
`effectiveReplicas`, catching a forgotten regeneration step.
|
||||
|
||||
### Deploy rollover (overlap + draining)
|
||||
|
||||
A `harness-workers` redeploy is the moment the staleness dip + cut-short worker
|
||||
drains used to happen. Two Railway service settings — **pure config, no custom
|
||||
rolling-restart code** — make a rollover non-lossy (no dip) and let the shipped
|
||||
graceful worker drain finish. Both are tracked in the SSOT
|
||||
(`workerProvisioning.{prod,staging}.overlapSeconds` / `.drainingSeconds`) and
|
||||
guarded by the same drift gate as the replica count.
|
||||
|
||||
| Setting | Railway field (env mirror) | Value | What it does |
|
||||
| ----------------- | ------------------------------------------------------------------------- | ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `overlapSeconds` | `serviceInstance.overlapSeconds` (`RAILWAY_DEPLOYMENT_OVERLAP_SECONDS`) | `45` | Capacity floor: keep the OLD deployment serving for 45s after the NEW one goes Active, so the live worker count never dips while new workers boot, register on the roster, and start claiming. |
|
||||
| `drainingSeconds` | `serviceInstance.drainingSeconds` (`RAILWAY_DEPLOYMENT_DRAINING_SECONDS`) | `180` | Graceful-drain window: the SIGTERM→SIGKILL budget the platform grants a draining worker before hard-killing it. |
|
||||
|
||||
**Rationale.**
|
||||
|
||||
- **`overlapSeconds = 45`** holds the capacity floor: a new worker is not useful
|
||||
the instant its container is Active — it must boot, register its roster row,
|
||||
and start claiming. Overlapping the old deployment for 45s bridges that window
|
||||
so there is no observable staleness dip during a rollover.
|
||||
- **`drainingSeconds = 180`** is set to `PLATFORM_STOP_GRACE_MS` (180s, defined
|
||||
in `showcase/harness/src/fleet/worker/worker-loop.ts`) so the shipped composed
|
||||
worker-drain budget fits under the platform kill: `DRAIN_DEREGISTER_TIMEOUT_MS`
|
||||
(3s roster-delete cap) + `DEFAULT_WORKER_DRAIN_GRACE_MS` (90s finish-and-report
|
||||
grace, layer b) + the small serial teardown remainder, all `< 180s`. The
|
||||
Railway default is `0s` (the field is `null` → `0`, confirmed via the live
|
||||
config read) — i.e. an effectively immediate SIGKILL after SIGTERM, which would
|
||||
cut the 90s drain short and abandon the in-flight cell; at 180s the worker
|
||||
finishes and reports its in-flight cell instead.
|
||||
Keep this `≥ PLATFORM_STOP_GRACE_MS`; if the layer-(b) grace is retuned, raise
|
||||
this in lockstep (the composed-budget test in `worker-loop.test.ts` is the
|
||||
source of truth for the relation).
|
||||
|
||||
**Composition with the drain layers** (full rationale in the
|
||||
`PLATFORM_STOP_GRACE_MS` / `DEFAULT_WORKER_DRAIN_GRACE_MS` docs in
|
||||
`worker-loop.ts`):
|
||||
|
||||
- **Layer (a)** — reaper backstop: a worker that genuinely overruns the 90s grace
|
||||
is abandoned; its lease lapses and the control-plane sweeper re-queues the cell
|
||||
neutral-gray. `drainingSeconds` does not change this — it is the long-tail
|
||||
fallback.
|
||||
- **Layer (b)** — graceful drain: on SIGTERM the worker STOPS CLAIMING, lets its
|
||||
in-flight cell FINISH within the 90s grace, and REPORTS its real terminal
|
||||
result. `drainingSeconds = 180` is the platform-side budget that lets layer (b)
|
||||
actually complete.
|
||||
- **Layer (c)** — this config: `overlapSeconds` removes the dip; `drainingSeconds`
|
||||
hosts the drain. No code — it is the two service settings alone.
|
||||
|
||||
**Applying (MANUAL).** Like the replica count, the emitter/`bin/railway` tooling
|
||||
is **verify-only** for these fields. To apply or change them:
|
||||
|
||||
1. Edit `overlapSeconds` / `drainingSeconds` in `railway-envs.ts` (SSOT) for the
|
||||
env(s).
|
||||
2. Regenerate: `npx tsx showcase/scripts/emit-railway-envs-json.ts`, commit both
|
||||
files.
|
||||
3. Apply to Railway manually, via EITHER:
|
||||
- **GraphQL** — `serviceInstanceUpdate(serviceId, environmentId, input: { overlapSeconds: 45, drainingSeconds: 180 })`;
|
||||
both fields are `Int` on `ServiceInstanceUpdateInput`.
|
||||
- **Dashboard** — Service > Settings, the deploy **Teardown**/overlap card (or
|
||||
set the `RAILWAY_DEPLOYMENT_OVERLAP_SECONDS` / `RAILWAY_DEPLOYMENT_DRAINING_SECONDS`
|
||||
service variables).
|
||||
|
||||
The CI drift gate also asserts `overlapSeconds` and `drainingSeconds` match
|
||||
between `railway-envs.ts` and `railway-envs.generated.json`, catching a forgotten
|
||||
regeneration.
|
||||
|
||||
## Environment IDs
|
||||
|
||||
- Project: `<project-id>`
|
||||
- Environment: `<env-id>`
|
||||
- Token: `~/.railway/config.json` -> `.user.token`
|
||||
|
||||
## Known Quirks
|
||||
|
||||
- **Polling frequency**: Railway's auto-update polling interval is
|
||||
undocumented. Expect seconds to low minutes after a GHCR push.
|
||||
|
||||
- **API surface**: `environmentPatchCommit` is the only programmatic
|
||||
way to configure auto-updates. Typed GraphQL mutations
|
||||
(`ServiceSourceInput`) do not expose `autoUpdates`.
|
||||
|
||||
- **`source.autoUpdates.type` values**: `disabled`, `patch`, `minor`.
|
||||
We use `minor` (any semver-compatible tag change, including `:latest`
|
||||
digest changes).
|
||||
|
||||
- **`source.autoUpdates.schedule`**: array of
|
||||
`{day, startHour, endHour}`. Omit entirely for "any time, immediately".
|
||||
|
||||
- **CI still redeploys explicitly**: `showcase_build.yml` triggers
|
||||
`serviceInstanceRedeploy` (via `redeploy-env.ts`) after the GHCR push, and
|
||||
`showcase_deploy.yml` ("Verify Deploy") health-checks that redeployment so it
|
||||
can verify the exact deployment it triggered. Auto-updates are the fallback,
|
||||
not a replacement for CI-driven deploy verification.
|
||||
@@ -0,0 +1,281 @@
|
||||
# Showcase Platform
|
||||
|
||||
Tagline: agent entry point for the showcase docs tree and from-scratch local
|
||||
setup. The fanout block below routes you to the right procedural doc.
|
||||
|
||||
Per-framework demos of CopilotKit (LangGraph, CrewAI, Mastra, Claude Agent SDK,
|
||||
etc.). Each package is a Next.js frontend + agent backend bundled in a Docker
|
||||
image. Railway deploys those images from `main` on push.
|
||||
|
||||
## Agent Fanout — when X, see Y
|
||||
|
||||
| When you need to... | Read |
|
||||
| ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
|
||||
| Turn a red cell green (cell red→green SOP, `bin/showcase test` CLI) | [`./TESTING.md`](./TESTING.md#sop-turning-a-cell-red--green) |
|
||||
| Debug a failure mode locally (debugging loop, strategies, prod ops) | [`./DEBUGGING.md`](./DEBUGGING.md) |
|
||||
| Look up a framework / fixture / `--isolate` edge case | [`./GOTCHAS.md`](./GOTCHAS.md) |
|
||||
| Add a brand-new integration (per-package + external setup) | [`./INTEGRATION-CHECKLIST.md`](./INTEGRATION-CHECKLIST.md) |
|
||||
| Style a demo page (Tailwind v4, CopilotKit overrides, layout patterns) | [`./STYLING-GUIDE.md`](./STYLING-GUIDE.md) |
|
||||
| Reason about which shell renders what / consolidate a new frontend | [`./FRONTEND-STRATEGY.md`](./FRONTEND-STRATEGY.md) |
|
||||
| Deploy / promote / pin / roll back a Railway service | [`./RAILWAY.md`](./RAILWAY.md) (fleet config) + [`./bin/README.md`](./bin/README.md) (`bin/railway` CLI) |
|
||||
| Understand aimock fixture semantics (fixtures + Railway reconstruction) | [`./aimock/README.md`](./aimock/README.md) + [`./aimock/RAILWAY.md`](./aimock/RAILWAY.md) |
|
||||
| Operate showcase-harness (alerts, probes, hot reload, build/deploy) | [`./harness/README.md`](./harness/README.md) + [`./harness/docs/rotation-drill.md`](./harness/docs/rotation-drill.md) |
|
||||
| Track or check per-slug deviations from canonical | `./integrations/<slug>/PARITY_NOTES.md` |
|
||||
|
||||
Anything below is from-scratch local setup — skip if your stack is already up.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
showcase/
|
||||
bin/showcase # unified CLI — run showcase/bin/showcase <command> for help
|
||||
bin/railway # Ruby tool for Railway ops (snapshot/promote/pin) — see bin/README.md
|
||||
integrations/<slug>/ # one per framework (17 total) — Dockerfile, src/app/demos/*/, src/agents/ or equivalent
|
||||
shell/ # hub: home page, /matrix, canonical /integrations/[slug]/[demo]/{preview,code}
|
||||
shell-dashboard/ # internal-only feature × integration grid (port 3002)
|
||||
harness/ # showcase-harness service — see harness/README.md
|
||||
aimock/ # aimock fixtures + Railway config — see aimock/README.md
|
||||
shared/
|
||||
feature-registry.json # canonical features + categories (feeds the grid rows)
|
||||
constraints.yaml # allowlist for which demos a package can expose
|
||||
local-ports.json # deterministic host ports per package for local Docker runs
|
||||
python/ typescript/tools/ # shared agent utility code; CI stages these into each build context
|
||||
scripts/
|
||||
dev-local.sh # low-level Docker Compose wrapper (prefer bin/showcase)
|
||||
cli/ # command modules for bin/showcase
|
||||
generate-registry.ts # builds shell/src/data/registry.json from all manifest.yaml
|
||||
bundle-demo-content.ts # bundles per-demo source + README into shell/src/data/demo-content.json
|
||||
docker-compose.local.yml # one service per package; ports from local-ports.json; env from .env
|
||||
.env.example # commit template — copy to .env and fill in
|
||||
```
|
||||
|
||||
## Generated data files
|
||||
|
||||
The shell apps consume JSON data files that are **generated at build time** by
|
||||
scripts in `scripts/`. These files are gitignored — every build path (Docker,
|
||||
CI, `npm run build`, `npm run dev`) regenerates them automatically.
|
||||
|
||||
| File | Generator | Shell apps | What it does |
|
||||
| ---------------------- | --------------------------- | ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `registry.json` | `generate-registry.ts` | shell, shell-docs, shell-dojo, shell-dashboard | Integration manifest — scans `integrations/*/manifest.yaml`, builds the full catalog with metadata, feature flags, categories |
|
||||
| `demo-content.json` | `bundle-demo-content.ts` | shell, shell-docs, shell-dojo | Bundled source code from every demo directory — powers the Code tab, Snippet components, dojo cell viewer |
|
||||
| `constraints.json` | `generate-registry.ts` | shell | Filter facets for the integration explorer (categories, frameworks, features) |
|
||||
| `search-index.json` | `generate-search-index.ts` | shell, shell-docs | Cmd-K search entries — scans MDX docs, AG-UI content, and registry data |
|
||||
| `starter-content.json` | `bundle-starter-content.ts` | shell | Starter template source bundles for the "Get Started" code viewer |
|
||||
| `docs-status.json` | `probe-docs.ts` | shell-dashboard | Per-feature docs reachability — HTTP HEAD on og_docs_url, file-exists check on shell-docs MDX |
|
||||
|
||||
Each generator writes to the `src/data/` directory of every shell app that
|
||||
consumes it. Shell apps are independent — no shell cross-imports another
|
||||
shell's data directory.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- macOS or Linux
|
||||
- [Homebrew](https://brew.sh/)
|
||||
- Docker engine. Any of: Docker Desktop, **Colima** (recommended, no GUI / no sign-in), or OrbStack.
|
||||
- Node 22+ and npm (for `shell` / `shell-dashboard` dev servers — they're not in the compose)
|
||||
|
||||
### Colima install (one time)
|
||||
|
||||
```sh
|
||||
brew install colima docker docker-buildx docker-compose
|
||||
|
||||
# Tell the docker CLI where its plugins live
|
||||
mkdir -p ~/.docker
|
||||
cat > ~/.docker/config.json <<'JSON'
|
||||
{
|
||||
"cliPluginsExtraDirs": ["/opt/homebrew/lib/docker/cli-plugins"]
|
||||
}
|
||||
JSON
|
||||
|
||||
# Start the engine (adjust resources to taste; needed for building 17 images)
|
||||
colima start --cpu 4 --memory 8 --disk 60
|
||||
|
||||
# Verify
|
||||
docker compose version
|
||||
```
|
||||
|
||||
Colima auto-starts with `brew services start colima` if you want it on login.
|
||||
|
||||
## API keys
|
||||
|
||||
One `.env` file feeds every container. **Not committed.**
|
||||
|
||||
```sh
|
||||
cp showcase/.env.example showcase/.env
|
||||
# Edit showcase/.env and fill in:
|
||||
# OPENAI_API_KEY=<required>
|
||||
# ANTHROPIC_API_KEY=<optional; needed for Claude Agent SDK demos and a few others>
|
||||
# LANGSMITH_API_KEY=<optional; enables LangSmith tracing for LangGraph demos>
|
||||
```
|
||||
|
||||
Only `OPENAI_API_KEY` is strictly required. Missing optional keys fail
|
||||
gracefully (per-package).
|
||||
|
||||
## bin/showcase CLI — quick reference
|
||||
|
||||
For the full invocation table (control-plane vs `--direct`, per-demo scoping
|
||||
matrix) and the cell red→green SOP, see
|
||||
[`./TESTING.md`](./TESTING.md#bin-showcase-test-invocation-semantics). For
|
||||
debugging workflows (aimock rebuild cycles, fixture validation, probe testing,
|
||||
diagnostics), see [`./DEBUGGING.md`](./DEBUGGING.md).
|
||||
|
||||
```sh
|
||||
# from any directory — paths resolved relative to the script itself
|
||||
|
||||
./showcase/bin/showcase up langgraph-python # start infra + one integration
|
||||
./showcase/bin/showcase test langgraph-python --d5 --isolate # run D5 probes (canonical)
|
||||
./showcase/bin/showcase down # tear down
|
||||
```
|
||||
|
||||
| Command | Description |
|
||||
| ------------------ | ---------------------------------------------------------------------------------- |
|
||||
| `test <slug>` | Run probes against a running service (see TESTING.md for full flag table) |
|
||||
| `up [slugs...]` | Start infra (aimock, pocketbase, dashboard) + named packages. No args = infra only |
|
||||
| `down [slugs...]` | Stop services. No args = stop everything |
|
||||
| `build [slugs...]` | Build Docker images |
|
||||
| `rebuild <slug>` | Rebuild a slug (handles symlink deref that raw `docker build` cannot) |
|
||||
| `recreate <slug>` | Force-recreate a service (picks up new image) |
|
||||
| `restart <slug>` | Restart container (picks up `src/` edits — see `Iterating on a demo` below) |
|
||||
| `ps` | Show running services |
|
||||
| `ports` | Print slug to host port mapping |
|
||||
| `logs <slug>` | Follow container logs (supports `--grep`, `--since`, `-n`, `--no-follow`) |
|
||||
| `doctor` | Check local environment and stack health |
|
||||
|
||||
Container exposes port `10000` internally → host port in
|
||||
[`shared/local-ports.json`](shared/local-ports.json). The image and entrypoint
|
||||
are **the same ones Railway runs**.
|
||||
|
||||
## Hooking local containers into the shell
|
||||
|
||||
The `shell` app's `/preview` route iframes `integration.backend_url` (Railway)
|
||||
by default. Set `SHOWCASE_LOCAL=1` when running `shell` to swap in the
|
||||
localhost ports from `local-ports.json` instead — per-slug, falling back to
|
||||
Railway for anything you don't have running.
|
||||
|
||||
```sh
|
||||
cd showcase/shell
|
||||
npm install # once
|
||||
SHOWCASE_LOCAL=1 npm run dev # /preview iframes http://localhost:<port>/demos/...
|
||||
```
|
||||
|
||||
In production the env var is unset → Railway URLs, unchanged.
|
||||
|
||||
## shell-dashboard — feature × integration matrix
|
||||
|
||||
Internal overview of which packages support which features, linking to the
|
||||
canonical `/preview` and `/code` routes on `shell`. Lives at
|
||||
http://localhost:3002 and reads the same `registry.json` `shell` does.
|
||||
|
||||
```sh
|
||||
cd showcase/shell-dashboard
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
|
||||
Column ordering lives in `shell-dashboard/src/lib/sort-order.ts` — internal to
|
||||
this app, not part of the public registry.
|
||||
|
||||
## Iterating on a demo
|
||||
|
||||
1. Edit the demo in `integrations/<slug>/src/app/demos/<demo-id>/page.tsx` (and
|
||||
the backend under `src/agents/` if applicable).
|
||||
2. Rebundle so `/code` in `shell` reflects the edit:
|
||||
`cd showcase && npx tsx scripts/bundle-demo-content.ts`.
|
||||
3. If you changed `manifest.yaml` or added a feature to
|
||||
`shared/feature-registry.json`:
|
||||
`npx tsx scripts/generate-registry.ts`.
|
||||
4. Rebuild + restart the container: `showcase/bin/showcase up <slug>` (or
|
||||
`restart <slug>` for pure `src/` edits — see DEBUGGING.md "Dev Iteration
|
||||
Speed").
|
||||
5. The grid in `shell-dashboard` and `/preview` in `shell` now show the new state.
|
||||
|
||||
## Relationship to Railway
|
||||
|
||||
- Dockerfile, `entrypoint.sh`, and build context (`shared_python/`,
|
||||
`shared_typescript/`) are shared between local and Railway.
|
||||
- `.github/workflows/showcase_deploy.yml` builds each image on push to `main`
|
||||
and pushes it to Railway. Per-PR deploys are opt-in via
|
||||
`gh workflow run showcase_deploy.yml -r <branch> -f service=<slug>`.
|
||||
- The only real differences at runtime are env var values and the URL. If
|
||||
something works locally in Docker, it works on Railway (and vice versa).
|
||||
|
||||
## Dashboard SOPs (catalog.json + PocketBase)
|
||||
|
||||
The dashboard at [showcase.copilotkit.ai](https://showcase.copilotkit.ai) reads
|
||||
two data sources:
|
||||
|
||||
1. **Static `catalog.json`** — generated at build time by
|
||||
`pnpm generate-registry`. Contains the full 38-feature × 17-integration cell
|
||||
matrix with status (`wired` / `stub` / `unshipped`), parity tiers, and
|
||||
feature categories. Changes require a generator run + commit.
|
||||
2. **Live PocketBase probe results** — streamed via SSE. Probes discover demo
|
||||
routes automatically and update the dashboard in real time. No manual
|
||||
intervention needed for probe data.
|
||||
|
||||
**Known limitation — PocketBase fetch cap:** `useLiveStatus.ts` fetches status
|
||||
records with a hard `INITIAL_CAP` (currently 2000). PocketBase returns records
|
||||
in rowid (creation) order. If the total record count exceeds the cap,
|
||||
later-created dimensions (e.g. `e2e:<slug>/<featureId>` per-cell records from
|
||||
the 6-hourly e2e-demos probe) get silently truncated, causing the dashboard to
|
||||
show D2 instead of D4 across the board. If new probe types are added and the
|
||||
dashboard regresses to D2, raise `INITIAL_CAP` in
|
||||
`shell-dashboard/src/hooks/useLiveStatus.ts`. The correct long-term fix is
|
||||
dimension-scoped fetching or `sort=-updated` so the cap never silently drops
|
||||
functional records.
|
||||
|
||||
Key invariants:
|
||||
|
||||
- **Parity tiers are never manually set.** They are computed by comparing each
|
||||
integration's wired feature set against the reference integration's.
|
||||
- **The reference integration is auto-detected** as the integration with the
|
||||
most wired features (ties broken alphabetically). No `reference: true` flag
|
||||
exists.
|
||||
- **`catalog.json` is gitignored** — the generator emits it into the shell
|
||||
apps' `src/data/` directories, which are already in `.gitignore`.
|
||||
- **The `stub` status** means: feature declared in manifest, demo entry exists,
|
||||
but no `route` field. Today only `langgraph-python/cli-start` qualifies.
|
||||
|
||||
### SOP 1: Wire a new demo on an existing integration
|
||||
|
||||
1. Edit `showcase/integrations/<slug>/manifest.yaml` — add the feature to
|
||||
`features[]` and a corresponding `demos[]` entry with a `route`.
|
||||
2. Run `pnpm generate-registry` — updates `registry.json` AND `catalog.json`.
|
||||
The cell flips from `unshipped` to `wired`. Parity tiers auto-recompute.
|
||||
3. Commit the manifest + both generated files. PR, merge.
|
||||
4. CI rebuilds the package image + dashboard image. Railway auto-deploys both.
|
||||
5. Ops probes discover the new demo route and begin probing. Dashboard updates
|
||||
live via PocketBase SSE — no further action needed.
|
||||
|
||||
### SOP 2: Code fix on an existing demo (no manifest change)
|
||||
|
||||
1. Edit code under `showcase/integrations/<slug>/src/...`.
|
||||
2. PR, merge. No generator run needed (manifest unchanged).
|
||||
3. CI rebuilds the package image. Railway auto-deploys.
|
||||
4. Probes re-probe on the next tick. If the fix turns a red cell green, the
|
||||
dashboard updates live. Zero manual steps beyond the normal PR workflow.
|
||||
|
||||
### SOP 3: Add a brand-new integration
|
||||
|
||||
1. Create `showcase/integrations/<new-slug>/manifest.yaml` with `features[]` +
|
||||
`demos[]`.
|
||||
2. Add `{"slug": "<new-slug>", "name": "<Display Name>"}` to
|
||||
`showcase/shared/packages.json`.
|
||||
3. Provision a Railway service (manual: `railway service create` or Dashboard
|
||||
UI).
|
||||
4. Run `pnpm generate-registry` — catalog gains 38 new cells (mostly
|
||||
`unshipped`, some `wired`). Parity tier computed automatically.
|
||||
5. Commit, PR, merge. CI + Railway deploy. Probes discover the new service
|
||||
automatically via the Railway discovery filter.
|
||||
|
||||
For the full per-package + external-setup checklist see
|
||||
[`./INTEGRATION-CHECKLIST.md`](./INTEGRATION-CHECKLIST.md).
|
||||
|
||||
### SOP 4: Reference migration (move the reference integration)
|
||||
|
||||
1. No manual flag needed — the generator auto-detects the reference as the
|
||||
integration with the most wired features (ties broken alphabetically).
|
||||
2. If you want a _different_ integration to be reference, wire more features on
|
||||
it until it leads the count.
|
||||
3. Run `pnpm generate-registry` — all parity tiers recompute automatically.
|
||||
4. Commit, PR, merge.
|
||||
@@ -0,0 +1,147 @@
|
||||
# Showcase Integration Package — Styling Guide
|
||||
|
||||
Tagline: Tailwind v4 purging traps, CopilotKit class overrides, chat layout
|
||||
wrapper pattern, theme tokens, and Docker-image CSS verification recipe.
|
||||
Externally referenced from per-demo READMEs — preserve path at
|
||||
`showcase/STYLING-GUIDE.md`.
|
||||
|
||||
When building demo pages for integration packages, follow these rules to avoid common pitfalls.
|
||||
|
||||
## Tailwind v4 Purging
|
||||
|
||||
**Tailwind v4 aggressively purges CSS classes it can't statically analyze.** If your component renders Tailwind classes dynamically (e.g., from state, props, or conditional logic), those classes will be missing from the CSS bundle.
|
||||
|
||||
### Do: Use inline styles for dynamic components
|
||||
|
||||
```tsx
|
||||
// GOOD — inline styles always work
|
||||
<div style={{ padding: "32px", borderRadius: "16px", border: "1px solid #e5e5e0" }}>
|
||||
```
|
||||
|
||||
### Don't: Use Tailwind classes on dynamically rendered content
|
||||
|
||||
```tsx
|
||||
// BAD — Tailwind may purge these classes
|
||||
<div className="p-8 rounded-2xl border border-gray-200">
|
||||
```
|
||||
|
||||
### When it's safe to use Tailwind
|
||||
|
||||
Tailwind classes work fine on:
|
||||
|
||||
- Static JSX in your component (not inside maps, conditionals, or dynamic renders)
|
||||
- The CopilotKit wrapper divs you write yourself
|
||||
- Next.js page-level layout elements
|
||||
|
||||
Tailwind classes are UNSAFE on:
|
||||
|
||||
- Content rendered by `useRenderTool`, `useHumanInTheLoop`, `useFrontendTool` handlers
|
||||
- Components rendered inside CopilotKit's chat message area
|
||||
- Any JSX returned from a callback or dynamic render function
|
||||
|
||||
## CopilotKit Component Overrides
|
||||
|
||||
CopilotKit v2 components use `cpk:` prefixed Tailwind classes internally. To override them:
|
||||
|
||||
### Do: Use a separate CSS file
|
||||
|
||||
```css
|
||||
/* copilotkit-overrides.css — import in layout.tsx AFTER globals.css */
|
||||
|
||||
.copilotKitInput {
|
||||
border-radius: 0.75rem;
|
||||
border: 1px solid var(--copilot-kit-separator-color) !important;
|
||||
}
|
||||
```
|
||||
|
||||
> Do NOT hardcode `.copilotKitChat { background-color: #fff !important; }` in
|
||||
> these overrides. It overrides demo-level theming (e.g. beautiful-chat's dark
|
||||
> mode toggle) and forces white in every theme. Let the v2 styles + per-demo
|
||||
> `ThemeProvider` handle chat backgrounds.
|
||||
|
||||
```tsx
|
||||
// layout.tsx
|
||||
import "./globals.css";
|
||||
import "./copilotkit-overrides.css"; // AFTER globals
|
||||
```
|
||||
|
||||
### Don't: Put overrides in globals.css
|
||||
|
||||
Tailwind v4 processes `globals.css` and purges anything it doesn't recognize as a utility class. CopilotKit class selectors (`.copilotKitChat`, `.copilotKitInput`) will be stripped.
|
||||
|
||||
## Chat Layout — The Wrapper Pattern
|
||||
|
||||
To get proper spacing around the CopilotChat component, wrap it like the Dojo does:
|
||||
|
||||
```tsx
|
||||
// GOOD — matches Dojo spacing
|
||||
<div className="flex justify-center items-center h-screen w-full">
|
||||
<div className="h-full w-full md:w-4/5 md:h-4/5 rounded-lg">
|
||||
<CopilotChat className="h-full rounded-2xl max-w-6xl mx-auto" />
|
||||
</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
The `md:w-4/5` gives 80% width on desktop, creating natural side margins. Don't try to add margins via CSS overrides on CopilotKit's internal classes — use the wrapper div.
|
||||
|
||||
## Theme
|
||||
|
||||
All integration packages should use the light theme to match the showcase shell:
|
||||
|
||||
```css
|
||||
:root {
|
||||
--copilot-kit-background-color: #fafaf9;
|
||||
--copilot-kit-primary-color: #0d6e3f;
|
||||
--copilot-kit-response-button-background-color: #f5f5f3;
|
||||
--copilot-kit-response-button-color: #1a1a18;
|
||||
}
|
||||
|
||||
html,
|
||||
body {
|
||||
background: #fafaf9;
|
||||
color: #1a1a18;
|
||||
}
|
||||
```
|
||||
|
||||
## Images
|
||||
|
||||
Don't reference local image files from agent-generated content. The agent may generate `image_name` values that don't exist on disk. Always add an `onError` fallback:
|
||||
|
||||
```tsx
|
||||
{
|
||||
imageUrl && !imageError && (
|
||||
<img src={imageUrl} onError={() => setImageError(true)} alt="..." />
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
## SVG Icons
|
||||
|
||||
Don't use SVG icons with `fill="currentColor"` inside CopilotKit chat messages. The `currentColor` inheritance is unpredictable in the chat context. Use emoji instead:
|
||||
|
||||
```tsx
|
||||
// GOOD
|
||||
<span style={{ fontSize: "48px" }}>☀️</span>
|
||||
|
||||
// BAD — renders as giant black shapes
|
||||
<svg fill="currentColor" className="w-14 h-14 text-yellow-200">...</svg>
|
||||
```
|
||||
|
||||
## Testing Locally
|
||||
|
||||
Always build and test the Docker image locally before pushing:
|
||||
|
||||
```bash
|
||||
cd /proj/cpk/CopilotKit
|
||||
docker build -f showcase/integrations/<slug>/Dockerfile -t <slug>-local showcase/integrations/<slug>/
|
||||
docker run -d --name <slug>-local -p 4444:10000 -e PORT=10000 <slug>-local
|
||||
|
||||
# Verify CSS overrides survived the build
|
||||
curl -sf http://localhost:4444/_next/static/css/*.css | grep "copilotKitChat"
|
||||
|
||||
# Verify inline styles are in rendered HTML
|
||||
curl -sf http://localhost:4444/demos/<demo> | grep "padding: 32px"
|
||||
|
||||
# Clean up
|
||||
docker rm -f <slug>-local
|
||||
```
|
||||
@@ -0,0 +1,278 @@
|
||||
# Showcase Testing
|
||||
|
||||
Tagline: `bin/showcase test` reference, cell red→green SOP, per-demo coverage
|
||||
matrix, and CI gating matrix.
|
||||
|
||||
Three concerns live here:
|
||||
|
||||
1. **Cell red→green SOP** and `bin/showcase test` CLI reference — how to run the
|
||||
harness locally and how to drive a cell from red to green.
|
||||
2. **Per-Demo Coverage Matrix** — what test surface exists for each demo across
|
||||
manual QA, unit, E2E, and aimock.
|
||||
3. **CI gating matrix** — which CI workflows fire on which triggers and whether
|
||||
they gate merges.
|
||||
|
||||
Operational gotchas (aimock fixture caching, `--isolate` slot collisions, etc.)
|
||||
live in [`GOTCHAS.md`](./GOTCHAS.md). Per-failure-mode debugging strategies and
|
||||
production-side ops (probe triggering, Railway log access, isolated stack
|
||||
cleanup) live in [`DEBUGGING.md`](./DEBUGGING.md).
|
||||
|
||||
## `bin/showcase test` invocation semantics
|
||||
|
||||
`--d5` / `--d6` route through the production-equivalent control-plane pipeline
|
||||
(producer → queue → worker, same as Railway). `--direct` switches to a legacy
|
||||
in-process driver (bypasses control-plane).
|
||||
|
||||
| Invocation | Family | Pipeline | Per-demo scoping (`:demo`) |
|
||||
| --------------------------- | ----------------- | ------------- | ------------------------------------ |
|
||||
| `--d5` (no `:demo`) | d5 representative | control-plane | hardcoded `agentic-chat` |
|
||||
| `--d5 :demo` | d5 single demo | control-plane | honored (post-A18) |
|
||||
| `--d5 --direct` | d5 family | in-process | honored via `buildDeepInputs` |
|
||||
| `--d6` (no `:demo`) | d6 full sweep | control-plane | full demo list, aggregate validation |
|
||||
| `--d6 :demo` | d6 single demo | control-plane | honored (post-A18) |
|
||||
| `--d6 --direct` | d6 family | in-process | honored via `buildFullInputs` |
|
||||
| `--direct` (no `--d5/--d6`) | d5+d6 default | in-process | honored |
|
||||
|
||||
**Use control-plane (no `--direct`) for production-equivalent testing.** It
|
||||
exercises the same producer→queue→worker pipeline Railway uses, so local
|
||||
results are apples-to-apples with staging. `--direct` is opt-out legacy:
|
||||
useful for fast in-process debugging when you don't need the queue.
|
||||
|
||||
`--isolate` (post-A21+A21b) scopes the rebuild to the target slug — infra
|
||||
services (aimock, pocketbase, dashboard, harness, harness-pool-worker) reuse
|
||||
cached images from the local Docker store. Cold-build is ~30s–2 min per slug
|
||||
instead of 10+ min full-stack rebuild.
|
||||
|
||||
### Session-stack discipline / Cleanup after isolated runs
|
||||
|
||||
This governs every `--isolate`/`--keep` invocation below. The leak it prevents:
|
||||
agents minting a _new_ named kept stack per cell (`cvtest2`, `greenproof`,
|
||||
`gp1`..`gp10`, `showcase-iso2/4`, …) and never tearing them down — each one
|
||||
holds a slot and burns offset ports until the host fills up.
|
||||
|
||||
1. **One stack per session, reused.** At the start of a debugging/testing
|
||||
session, choose ONE stable isolate name and use
|
||||
`--isolate <session-name> --keep` for ALL tests in that session. Every
|
||||
subsequent test in the same session MUST reuse ONE `--isolate <session-name>`
|
||||
— never mint a new named stack per individual cell/feature/pill (that is what
|
||||
leaks named stacks). Derive the session name from the primary slug under test
|
||||
so reuse is unambiguous, e.g. `--isolate <slug>-session`. Each `--keep` re-run
|
||||
with the same name pre-cleans (`docker compose -p <name> down`) and recreates
|
||||
that ONE stack, so you never pile up N named stacks. (Re-running the same name
|
||||
while the prior kept stack is still live fails loudly with a duplicate-name
|
||||
guard — another reason to keep to ONE name and tear down between fresh
|
||||
builds.)
|
||||
|
||||
2. **`--keep` is for intra-session reuse only, never a license to leak.** It
|
||||
exists so a session-long stack survives between tests; if you pass `--keep`,
|
||||
you OWN teardown at session end.
|
||||
|
||||
3. **Tear down at session end.** When the session's work is done — and as part
|
||||
of the Done criteria — tear down every stack the session created and release
|
||||
its slot, using the survival-notice teardown command (printed at exit):
|
||||
|
||||
```sh
|
||||
docker compose -p <name> down --remove-orphans --volumes && rm -rf <run-dir> <slot-dir>
|
||||
```
|
||||
|
||||
`bin/showcase down` does NOT tear down an isolated stack — it only stops the
|
||||
default (non-isolated) `showcase-*` project. Use the explicit
|
||||
`docker compose -p <name> down …` from the notice (run-dir and slot-dir are
|
||||
the real paths it prints). Bare `--isolate` (no `--keep`) auto-cleans on exit
|
||||
and frees its slot — prefer it for one-off tests that don't need to persist
|
||||
across the session.
|
||||
|
||||
The teardown mechanics, the RUNNING-only slot protection, and the scratch/slot
|
||||
paths are documented once in [`DEBUGGING.md` → Cleanup](./DEBUGGING.md#cleanup);
|
||||
this section owns the discipline (reuse ONE, tear down at end).
|
||||
|
||||
## SOP: turning a cell red → green
|
||||
|
||||
1. **Run the harness LOCALLY FIRST.** Capture the RED log via the production-equivalent
|
||||
path BEFORE any code change:
|
||||
|
||||
```
|
||||
bin/showcase test <slug>:<demo> --d5 --isolate
|
||||
```
|
||||
|
||||
No theory, no "should work" — observe the actual failure.
|
||||
|
||||
2. **One change. Verify GREEN locally on the same probe.** Re-run the same invocation;
|
||||
it must go green. Iterate if not. Don't commit on red.
|
||||
|
||||
3. **LGP regression check every time.** Gold-standard cell must stay green:
|
||||
|
||||
```
|
||||
bin/showcase test langgraph-python:tool-rendering-custom-catchall --d5 --isolate
|
||||
```
|
||||
|
||||
4. **Diagnose by failure mode** (use the aimock `/journal` endpoint + `docker logs
|
||||
showcase-iso<N>-aimock` + DOM/probe text):
|
||||
- Backend doesn't loop after `tool_result` → backend fix (add tool handler, fix
|
||||
agentId routing). See `crewai-crews` for the canonical example.
|
||||
- `toolCallId`-gated narration fixture doesn't match → backend rewrites IDs
|
||||
(Anthropic `toolu_*`, TanStack `fc-*`). Fix: swap `toolCallId` discriminator
|
||||
for `turnIndex` (or `hasToolResult` / `sequenceIndex`) — backend-id-invariant.
|
||||
See `built-in-agent` and `claude-sdk-typescript` for canonical match-key tunes.
|
||||
- No fixture entry matches the probe's `userMessage` → add the entry following
|
||||
the existing match-shape conventions.
|
||||
- Probe scan returns false despite phrase visibly in DOM → harness/probe bug.
|
||||
|
||||
5. **Layer boundaries:**
|
||||
- **Fixtures: per-integration freedom.** Do NOT modify `response.content` (canonical
|
||||
narration is fixture-author truth — the d5 probe asserts on it; a real LLM won't
|
||||
reliably emit the verbatim phrase). Tune `match` keys instead.
|
||||
- **Backends: minimal.** Faithfully echo aimock prescriptions where possible;
|
||||
close the tool-loop with a second LLM call after `tool_result`. Don't expand
|
||||
backend logic when a fixture match-key change suffices.
|
||||
- **Tests: identical** across integrations (the d5 probe is shared).
|
||||
- **Frontends: near-identical** — don't touch in cell-fix scope.
|
||||
- **LGP is gold standard.** Diff against `langgraph-python`'s fixture/backend
|
||||
pattern, not the sibling-of-the-day.
|
||||
|
||||
6. **aimock matcher semantics** (for `/v1/responses` after `responsesInputToMessages`
|
||||
transform; identical to `/v1/chat/completions`):
|
||||
| Match key | Semantics |
|
||||
|------------------|------------------------------------------------------------------------|
|
||||
| `userMessage` | substring on the last `role:"user"` message |
|
||||
| `toolCallId` | strict equality vs last `role:"tool"` `tool_call_id` (FRAGILE — backend ID-rewrite breaks this) |
|
||||
| `hasToolResult` | boolean: any `role:"tool"` message present |
|
||||
| `turnIndex` | integer: count of `role:"assistant"` messages |
|
||||
| `context` | equality vs `x-aimock-context` header |
|
||||
|
||||
First-match-wins. Order entries specific-before-generic.
|
||||
|
||||
7. **aimock caches fixtures at container startup.** Editing a fixture in a live
|
||||
stack requires `docker restart showcase-iso<N>-aimock` to reload. Fresh
|
||||
`--isolate` slots cold-start aimock from the volume mount, so the first
|
||||
post-edit run picks up the change automatically; warm-slot reuse will not.
|
||||
|
||||
8. **`--isolate` slot pinning and conflict detection.** Pin a specific slot with `SHOWCASE_ISO_SLOT=<N>` (1-45; slot 0 is reserved for the base stack), or use the equivalent CLI sugar `--isolate=<N>` — the picker uses exactly that slot or fails loudly. The auto-picker now port-probes every candidate via `lsof` before committing, so foreign-Docker (`ag2mm-*`) and host-process (macOS AirPlay on 5000) conflicts are detected pre-`docker compose up`. Run `bin/showcase slots` to inspect all 46 slots across DIR / PID / LIVE / PORTS / OFFSET (and PROJECT) — same code path the picker uses. The `LIVE` column reports `live` / `stale` / `inconclusive` and folds the live-pid and live-containers checks into one axis.
|
||||
|
||||
9. **Cell-color flip claims MUST be empirically value-tested via the
|
||||
production-equivalent control-plane path** on ≥3 candidate cells before merge.
|
||||
No "should flip N." Use:
|
||||
|
||||
```
|
||||
bin/showcase test <slug>:<feature> --d6 --isolate
|
||||
```
|
||||
|
||||
(or `--d5 --isolate` for single-pill e2e). DO NOT use `--direct` for
|
||||
value-test — it bypasses the queue/worker pipeline staging actually runs and
|
||||
has misled investigations in the past.
|
||||
|
||||
Pick `<N>` by first running `bin/showcase slots` (or `bin/showcase slots --free --brief` for a machine-readable list) and choosing a row whose `DIR` is `absent`, `LIVE` is not `live`, and `PORTS` is not `held`, then pin the slot via either form (both are equivalent):
|
||||
|
||||
```
|
||||
SHOWCASE_ISO_SLOT=<N> bin/showcase test <slug>:<feature> --d6 --isolate
|
||||
# — or, equivalently —
|
||||
bin/showcase test <slug>:<feature> --d6 --isolate=<N>
|
||||
```
|
||||
|
||||
10. **No fixture rewrite from real-LLM record/replay.** The canonical-phrase probe
|
||||
is anti-record/replay-against-real-LLM by construction: a real LLM won't emit
|
||||
the verbatim assertion phrase. Fixture content is authored truth; tune the
|
||||
`match` keys to fire on the right turn.
|
||||
|
||||
## Per-Demo Coverage Matrix
|
||||
|
||||
Tagline: what test surface exists per demo. PASS=covered, WARN=partial, FAIL=none, STUB=needs aimock fixture.
|
||||
|
||||
### Demo Coverage
|
||||
|
||||
| Demo | Manual QA | Vitest Unit | Playwright E2E (smoke) | Playwright E2E (interaction) | Per-Package E2E | Aimock Fixtures | CI Auto |
|
||||
| ---------------------------- | ----------------------- | ----------- | ----------------------- | ---------------------------- | ------------------------------------------------ | ------------------------------------------------------------ | --------------------------------------------------- |
|
||||
| **Agentic Chat** | PASS 19 packages | FAIL | PASS load + suggestions | WARN suggestion click only | PASS weather card, background change, multi-turn | WARN `background`, `weather` matches | WARN validate only (no Playwright in CI by default) |
|
||||
| **Human in the Loop** | PASS 19 packages | FAIL | PASS load + suggestions | WARN suggestion click only | PASS step selector, approve/reject | WARN `plan`/`steps`/`mars` matches (text only, no interrupt) | WARN validate only |
|
||||
| **Tool Rendering** | PASS 19 packages | FAIL | PASS load + suggestions | WARN suggestion click only | PASS WeatherCard with stats grid | WARN `weather` match (tool call) | WARN validate only |
|
||||
| **Gen UI (Tool-Based)** | PASS 19 packages | FAIL | FAIL | FAIL | PASS sidebar, haiku card, pie/bar chart | FAIL no haiku-specific fixture | WARN validate only |
|
||||
| **Gen UI (Agent)** | PASS 19 packages | FAIL | FAIL | FAIL | PASS task progress tracker, progress bar | FAIL no gen-ui-agent fixture | WARN validate only |
|
||||
| **Shared State (Read)** | PASS 19 packages | FAIL | FAIL | FAIL | PASS recipe card, sidebar, pipeline | FAIL no shared-state fixture | WARN validate only |
|
||||
| **Shared State (Write)** | PASS 19 packages (stub) | FAIL | FAIL | FAIL | PASS pipeline, deal CRUD, agent state writes | FAIL no shared-state fixture | WARN validate only |
|
||||
| **Shared State (Streaming)** | PASS 19 packages (stub) | FAIL | FAIL | FAIL | PASS document editor, confirm/reject changes | FAIL no streaming fixture | WARN validate only |
|
||||
| **Sub-Agents** | PASS 19 packages (stub) | FAIL | FAIL | FAIL | PASS travel planner, agent indicators, sections | FAIL no subagent fixture | WARN validate only |
|
||||
|
||||
> The Manual-QA column counts the **19 integration packages that ship a `qa/`
|
||||
> directory** — `ms-agent-harness-dotnet` ships no manual-QA checklists, so it
|
||||
> is excluded from these counts (the package count of 20 in Test Infrastructure
|
||||
> Locations still reflects all integration packages).
|
||||
|
||||
### Starter Hero Coverage
|
||||
|
||||
| Feature | Manual QA | Vitest Unit | Playwright E2E (smoke) | Playwright E2E (interaction) | Aimock Fixtures | CI Auto |
|
||||
| ------------------------------- | --------- | ------------------------------------------------- | --------------------------------------- | ----------------------------------------------- | ---------------------------------- | ------------------------------------------- |
|
||||
| **Sales Dashboard (page load)** | FAIL | PASS extract-starter tests (on-demand extraction) | PASS header, 4 renderer pills | PASS pill switching, content verification | WARN `sales`/`todo`/`deal` matches | WARN validate + aimock-e2e (manual trigger) |
|
||||
| **Renderer Selector** | FAIL | FAIL | PASS 4 pills visible, default selection | PASS mutual exclusion, content changes per mode | FAIL | WARN validate only |
|
||||
| **Tool-Based mode** | FAIL | FAIL | PASS pipeline heading, KPI cards | PASS Add a deal, multiple deals, empty state | WARN `sales`/`todo` matches | WARN validate only |
|
||||
| **A2UI Catalog mode** | FAIL | FAIL | PASS same pipeline content | FAIL | FAIL | WARN validate only |
|
||||
| **json-render mode** | FAIL | FAIL | PASS fallback note + pipeline | FAIL | FAIL | WARN validate only |
|
||||
| **HashBrown mode** | FAIL | FAIL | PASS pipeline content | FAIL | FAIL | WARN validate only |
|
||||
|
||||
### Probe Depth Coverage
|
||||
|
||||
| Depth | Probe Name | Cadence | What "Green" Means |
|
||||
| ----- | ---------------- | ---------- | ------------------------------------------------- |
|
||||
| D5 | e2e-demos | hourly :10 | All per-integration smoke probes pass |
|
||||
| D6 | d6-all-pills-e2e | hourly :40 | All demo cells pass across every integration pill |
|
||||
|
||||
### Test Infrastructure Locations
|
||||
|
||||
- **Manual QA**: `showcase/integrations/*/qa/*.md` (~498 files across 20 integration packages; per-package counts vary widely, e.g. `langgraph-python` 39, `langgraph-fastapi` 10).
|
||||
- **Vitest unit**: `showcase/scripts/__tests__/*.test.ts` — registry/constraint/bundle/integration generators.
|
||||
- **Shared E2E**: `showcase/scripts/__tests__/e2e/` — `starter-e2e.spec.ts`, `demo-e2e.spec.ts`, `screenshots.spec.ts`.
|
||||
- **Per-package E2E**: `showcase/integrations/*/tests/e2e/` — ~33–41 demo/feature specs per package (e.g. `langgraph-python` 38; `ms-agent-harness-dotnet` only 9).
|
||||
- **Aimock fixtures**: `showcase/aimock/` — `shared/common.json`, `shared/smoke.json`, `d4/<slug>/`, `d6/<slug>/`.
|
||||
- **CI**: `.github/workflows/showcase_*.yml` (see Test-Gating Matrix below).
|
||||
|
||||
### Known coverage gaps
|
||||
|
||||
1. No aimock fixtures for haiku, recipe/deals/document state, travel planning, HITL interrupt.
|
||||
2. No shared E2E for gen-ui-tool-based, gen-ui-agent, shared-state-\*, subagents.
|
||||
3. No automatic Playwright E2E in CI on every PR; aimock E2E requires `/test-aimock` comment.
|
||||
4. No Sales Dashboard starter manual QA checklists.
|
||||
5. No per-package renderer-selector E2E spec — renderer-selector coverage comes from the shared starter E2E (`showcase/scripts/__tests__/e2e/starter-e2e.spec.ts`) against the shared starter-template component (`showcase/shared/starter-template/components/renderers/renderer-selector.tsx`).
|
||||
|
||||
## Test-Gating Matrix
|
||||
|
||||
This matrix documents which CI workflows fire on which triggers, what they test, and whether they gate merges.
|
||||
|
||||
Scope: all testing-related workflows (unit, integration, e2e, smoke) across the monorepo. Data read directly from `.github/workflows/*.yml` on the current branch.
|
||||
|
||||
## Matrix
|
||||
|
||||
| Workflow file | Name (CI UI) | Trigger | Path filter | Required? | What it tests |
|
||||
| --------------------------------------------------- | --------------------------------- | ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | --------- | ------------------------------------------------------------------------------------------------ |
|
||||
| `.github/workflows/test_unit.yml` | test / unit | push (main), pull_request (main), workflow_dispatch | paths-ignore: `README.md`, `examples/**`, `showcase/**`, `sdk-python/**` | No | Vitest unit suite across Node 20/22/24 for all TS packages |
|
||||
| `.github/workflows/test_unit-python-sdk.yml` | test / unit / python-sdk | push (main), pull_request (main) | `sdk-python/**`, this workflow | No | pytest against `sdk-python/` across a Python 3.10–3.14 matrix + Poetry |
|
||||
| `.github/workflows/test_integration-runtime.yml` | test / integration / runtime | push (main), pull_request (main), workflow_dispatch | `packages/runtime/**`, this workflow | No | Runtime server integration tests (Node, possibly others) |
|
||||
| `.github/workflows/test_integration-docs.yml` | test / integration / docs | push (main), pull_request | `showcase/shell-docs/src/content/**`, docs validation scripts | No | Extracts code blocks from shell-docs, runs them against aimock (model-name + doc-test) |
|
||||
| `.github/workflows/test_e2e-dojo.yml` | test / e2e / dojo | push (main), pull_request (main), workflow_dispatch | `packages/**`, `sdk-python/**`, this workflow | No | ag-ui dojo end-to-end matrix on Depot runners |
|
||||
| `.github/workflows/test_e2e-legacy-v1.yml` | test / e2e / legacy-v1 | push (main), pull_request (main), workflow_dispatch | `examples/**`, this workflow | No | Legacy v1.x examples (form-filling, travel, research-canvas, chat-with-your-data, state-machine) |
|
||||
| `.github/workflows/showcase_validate.yml` | Showcase: Validate | push (main), pull_request | `showcase/**`, `examples/integrations/**`, `package.json`, `pnpm-lock.yaml`, `pnpm-workspace.yaml`, validate + deploy workflow files | No | Build-pipeline Vitest + manifest/registry validation + shell build |
|
||||
| `.github/workflows/test_e2e-showcase-on-demand.yml` | test / e2e / showcase / on-demand | issue_comment (`/test-aimock`), workflow_dispatch | n/a (comment-gated) | No | aimock-backed Playwright E2E on demand per-package |
|
||||
| `.github/workflows/test_smoke-starter.yml` | test / smoke / starter | schedule (`0 */6 * * *`), workflow_run (publish / release), pull_request, workflow_dispatch | `examples/integrations/**`, this workflow | No | Docker-compose smoke for 12 starter integrations (build + curl) |
|
||||
|
||||
## Which tests run on a typical PR?
|
||||
|
||||
- `packages/**` (runtime/SDK): `test / unit`, `test / integration`, `test / e2e / dojo`, `static / quality`, `static / check binaries`, plus `static / danger` if `packages/sdk-js/src/langgraph.ts` is touched.
|
||||
- `sdk-python/**`: `test / unit / python-sdk`, `test / e2e / dojo`, `static / quality`, `static / check binaries`, plus `static / danger` if `sdk-python/copilotkit/langgraph_agent.py` is touched. `test / unit` does NOT fire (paths-ignore excludes `sdk-python/**`).
|
||||
- `showcase/**`: `Showcase: Validate`, `static / quality`, `static / check binaries`. `test / unit` does NOT fire (paths-ignore excludes `showcase/**`). No Playwright E2E runs automatically -- comment `/test-aimock` on the PR to trigger `test / e2e / showcase / on-demand`.
|
||||
- `examples/**` (legacy v1.x): `test / e2e / legacy-v1`, `static / check binaries`. `test / unit` is excluded via paths-ignore.
|
||||
- `examples/integrations/**` (starters): `test / smoke / starter` (Docker), `Showcase: Validate` (for fixtures only), `static / check binaries`.
|
||||
- `showcase/shell-docs/src/content/**`: `test / integration / docs`, `Showcase: Validate`, and showcase build checks.
|
||||
- `.github/workflows/**`: each workflow that lists its own path in its trigger runs (most do). No single "workflows changed" catch-all.
|
||||
|
||||
## Required status checks
|
||||
|
||||
None of the workflows above are enforced as required status checks. The active `PROTECT_OUR_MAIN` ruleset on `main` requires zero status contexts -- merges are gated only by review approval, not by CI outcome.
|
||||
|
||||
Classic branch protection on `main` is fully disabled — `GET .../branches/main/protection` and `.../required_status_checks/contexts` both return HTTP 404 ("Branch protection has been disabled on this repository"), so there is no `required_status_checks.contexts` array to read from the live API. If you see a CI workflow marked as "required" in an older doc or script, treat that as ghost data: nothing in GitHub's current enforcement path consumes it.
|
||||
|
||||
Practical consequence: a red CI run does not block merge. Reviewers must eyeball `gh pr checks` before approving.
|
||||
|
||||
## Footnotes
|
||||
|
||||
- `test / unit` matrix is Node 20/22/24; the other workflows pin a single Node version each (22 most common).
|
||||
- Ten workflows run on Depot runners. Nine use `depot-ubuntu-24.04-4`: `test / e2e / dojo`, `test / unit`, `test / e2e / legacy-v1`, `test / integration / docs`, `test / integration / runtime`, `test / unit / python-sdk`, `Showcase: Validate`, `Showcase: Build & Push`, and `Showcase: Build Check (PR)`. The tenth, `showcase / eval`, runs on the larger `depot-ubuntu-24.04-16`.
|
||||
- `test / smoke / starter` runs every 6h and validates Docker-build integrity of `examples/integrations/`.
|
||||
- `workflow_run` triggers fire after another workflow completes -- they do not gate the triggering PR, they run post-merge.
|
||||
@@ -0,0 +1,6 @@
|
||||
FROM ghcr.io/copilotkit/aimock:latest
|
||||
|
||||
# Depth-organized fixture directories
|
||||
COPY shared/ /fixtures/shared/
|
||||
COPY d4/ /fixtures/d4/
|
||||
COPY d6/ /fixtures/d6/
|
||||
@@ -0,0 +1,314 @@
|
||||
# showcase-aimock Railway service reference
|
||||
|
||||
Tagline: authoritative backup of the `showcase-aimock` Railway service config
|
||||
(image, startCommand, baked-in fixtures, env vars) and the from-scratch recreate
|
||||
recipe. Concrete IDs / domains live in the Notion plan (section 9), not in
|
||||
this public repo.
|
||||
|
||||
This document persists the Railway service configuration for `showcase-aimock`
|
||||
in the repo so the service can be reconstructed from scratch if Railway state
|
||||
is ever lost. All runtime config (image, startCommand, env vars) lives only in
|
||||
Railway — this file is the authoritative backup.
|
||||
|
||||
> **Where the concrete IDs live.** Because this repo is public, concrete
|
||||
> Railway service/project/environment IDs and the current public domain are
|
||||
> **not** stored here. They live in the internal Notion plan (see section 9)
|
||||
> and can be queried live from the Railway GraphQL API with a valid account
|
||||
> token. Everywhere below you see `<service-id>`, `<project-id>`,
|
||||
> `<environment-id>`, or `<public-domain>`, substitute the current value from
|
||||
> one of those sources.
|
||||
|
||||
## 1. What this service is
|
||||
|
||||
`showcase-aimock` is a shared mock LLM server that 14+ CopilotKit showcase
|
||||
services route to via `OPENAI_BASE_URL`. It runs the `showcase-aimock` wrapper
|
||||
image (built from `showcase/aimock/Dockerfile`, `FROM
|
||||
ghcr.io/copilotkit/aimock:latest` with the fixture tree baked in — see §3) in
|
||||
proxy-only mode and serves fixture-driven responses so demos work
|
||||
deterministically without burning provider tokens. Unmatched requests fall
|
||||
through to real upstream providers (OpenAI, Anthropic, Gemini).
|
||||
|
||||
## 2. Railway identity
|
||||
|
||||
| Field | Value |
|
||||
| -------------- | --------------------------------------------------------- |
|
||||
| Service name | `showcase-aimock` |
|
||||
| Service ID | `<service-id>` (see Notion plan, section 9) |
|
||||
| Project name | `showcase` |
|
||||
| Project ID | `<project-id>` (see Notion plan, section 9) |
|
||||
| Environment | `production` |
|
||||
| Environment ID | `<environment-id>` (see Notion plan, section 9) |
|
||||
| Public domain | `<public-domain>` (see Notion plan, or Railway dashboard) |
|
||||
|
||||
> **Auth for `showcase`-project mutations.** Use an account-scoped
|
||||
> `RAILWAY_TOKEN` (stored in the DevOps `showcase` 1Password item) against the
|
||||
> Railway GraphQL API with an `Authorization: Bearer <token>` header. The
|
||||
> Railway CLI session token is **not** authorized for mutations on the
|
||||
> `showcase` project — the account-scoped token is the working path.
|
||||
|
||||
To look these up live from Railway GraphQL with a valid account token:
|
||||
|
||||
```graphql
|
||||
query {
|
||||
# List services under the `showcase` project to find the ID.
|
||||
projects {
|
||||
edges {
|
||||
node {
|
||||
id
|
||||
name
|
||||
services {
|
||||
edges {
|
||||
node {
|
||||
id
|
||||
name
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Then drill into the specific service:
|
||||
|
||||
```graphql
|
||||
query {
|
||||
service(id: "<service-id>") {
|
||||
id
|
||||
name
|
||||
projectId
|
||||
serviceInstances {
|
||||
edges {
|
||||
node {
|
||||
environmentId
|
||||
startCommand
|
||||
source {
|
||||
image
|
||||
repo
|
||||
}
|
||||
domains {
|
||||
serviceDomains {
|
||||
domain
|
||||
}
|
||||
customDomains {
|
||||
domain
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 3. Runtime image
|
||||
|
||||
- Image: `showcase-aimock`, built by `.github/workflows/showcase_build.yml`
|
||||
from `showcase/aimock/Dockerfile`. The Dockerfile is `FROM
|
||||
ghcr.io/copilotkit/aimock:latest` (the upstream aimock image published from
|
||||
`CopilotKit/aimock`) and **bakes the fixture tree into the image** (see
|
||||
section 4) — that baked image is what Railway deploys, not the bare upstream
|
||||
image.
|
||||
- Base aimock version: tracks `ghcr.io/copilotkit/aimock:latest`. Pin the base
|
||||
tag in the Dockerfile if you need to freeze it for showcase stability.
|
||||
- Published platform: `linux/amd64` only (`platforms: linux/amd64` in
|
||||
`showcase_build.yml`; arm64 is intentionally not published — arm64-only builds
|
||||
crash). Railway pulls amd64.
|
||||
|
||||
## 4. Fixture sources
|
||||
|
||||
Fixtures are **baked into the image at build time**, not fetched remotely. The
|
||||
`showcase/aimock/Dockerfile` copies three fixture directories from this repo
|
||||
into the image:
|
||||
|
||||
- `shared/` → `/fixtures/shared/` — `common.json` shared responses plus
|
||||
`smoke.json` (the minimal "OK" ping used for health verification).
|
||||
- `d4/` → `/fixtures/d4/` — per-slug fixtures for the D4 demos.
|
||||
- `d6/` → `/fixtures/d6/` — per-slug fixtures for the D6 demos. The
|
||||
`showcase/aimock/d6/<slug>/` tree is the source of truth for these.
|
||||
|
||||
The container loads these baked-in directories at boot (see the
|
||||
`--fixtures /fixtures` flag in section 5). There are no remote fixture URLs and no boot-time fetch —
|
||||
the old `d5-all.json` / `feature-parity.json` / remote-`smoke.json` bundles
|
||||
no longer exist (`d5-all.json` was a one-time migration source that was split
|
||||
into the per-slug `d6/` tree).
|
||||
|
||||
To update fixtures, edit the files under `showcase/aimock/{shared,d4,d6}/` and
|
||||
rebuild the image (a push touching `showcase/aimock/**` triggers
|
||||
`showcase_build.yml`). Changes land on the next Railway deploy of the rebuilt
|
||||
image.
|
||||
|
||||
> **showcase-harness browser-pool budget.** The harness runs
|
||||
> `BROWSER_POOL_BROWSERS=3` long-lived Chromium processes with a global
|
||||
> `BROWSER_POOL_MAX_CONTEXTS=24` context cap (lowered from 40). The D6 peak is
|
||||
> now 5×4=20 and the D5 e2e-deep peak is 16 (4 services × 4 features), so a
|
||||
> d6+d5 overlap (20+16=36) exceeds the 24 cap and serializes against it — that
|
||||
> back-pressure is intended. D5 e2e-deep alone runs up to 4 services × 4
|
||||
> features = 16 concurrent contexts (~4.8 GB peak). The binding constraint is
|
||||
> the PID ceiling of 1000, not memory, so contexts (not processes) are the
|
||||
> scaling knob — tune `BROWSER_POOL_MAX_CONTEXTS` to bound contention, or reduce
|
||||
> `FEATURE_CONCURRENCY_D6` in
|
||||
> `showcase/harness/src/probes/drivers/d6-all-pills.ts` / `max_concurrency` in
|
||||
> `e2e-deep.yml` if a single probe needs throttling.
|
||||
|
||||
## 5. Start command
|
||||
|
||||
> **Railway overrides Docker ENTRYPOINT.** When `startCommand` is set, Railway
|
||||
> runs it as the container's command and the image's `ENTRYPOINT` is ignored.
|
||||
> That means the full `node /app/dist/cli.js` bin invocation must appear
|
||||
> explicitly in `startCommand` — flag-only invocations fail at boot with
|
||||
> `The executable --proxy-only could not be found.` This was discovered during
|
||||
> the Phase 2 deploy when an initial flag-only startCommand was rejected.
|
||||
|
||||
```sh
|
||||
node /app/dist/cli.js \
|
||||
--proxy-only \
|
||||
--fixtures /fixtures \
|
||||
--provider-openai https://api.openai.com \
|
||||
--provider-anthropic https://api.anthropic.com \
|
||||
--provider-gemini https://generativelanguage.googleapis.com \
|
||||
--validate-on-load \
|
||||
--host 0.0.0.0 \
|
||||
--port 4010
|
||||
```
|
||||
|
||||
> **A single `--fixtures /fixtures` loads the whole baked-in fixture tree.**
|
||||
> The live prod and staging `showcase-aimock` instances both run exactly this
|
||||
> startCommand — one `--fixtures /fixtures` flag that recurses into
|
||||
> `/fixtures/shared`, `/fixtures/d4`, and `/fixtures/d6` — and serve fixtures
|
||||
> correctly. (Confirmed via live Railway GraphQL on both environments.)
|
||||
|
||||
Flag-by-flag:
|
||||
|
||||
| Flag | Value | Purpose |
|
||||
| ----------------------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `node /app/dist/cli.js` | — | Explicit bin invocation — required because Railway's `startCommand` overrides ENTRYPOINT. |
|
||||
| `--proxy-only` | — | Forward unmatched requests to upstream providers instead of failing. |
|
||||
| `--provider-openai` | `https://api.openai.com` | Upstream URL for OpenAI passthrough. |
|
||||
| `--provider-anthropic` | `https://api.anthropic.com` | Upstream URL for Anthropic passthrough. |
|
||||
| `--provider-gemini` | `https://generativelanguage.googleapis.com` | Upstream URL for Gemini passthrough. |
|
||||
| `--fixtures` | `/fixtures` | Loads the baked-in fixture tree at boot; recurses into `/fixtures/{shared,d4,d6}`. (The flag is repeatable if you ever need to point at individual subdirectories.) |
|
||||
| `--validate-on-load` | — | Fail-loud on schema errors at boot. |
|
||||
| `--host` | `0.0.0.0` | Bind all interfaces so Railway can route to the container. |
|
||||
| `--port` | `4010` | Hardcoded listen port — matches the legacy wrapper container convention and the fixed |
|
||||
| | | Railway domain routing. Railway injects `$PORT` but the image defaults align with 4010. |
|
||||
|
||||
If adopting `$PORT` interpolation in the future, both startCommand and any
|
||||
upstream `OPENAI_BASE_URL` env vars pointing at this service stay unchanged —
|
||||
Railway routes the public domain to whatever port the container listens on.
|
||||
|
||||
## 6. Environment variables
|
||||
|
||||
None are required for the default configuration. Notes:
|
||||
|
||||
- `AIMOCK_ALLOW_PRIVATE_URLS=1` would only be needed if fixtures were loaded
|
||||
from private URLs (RFC1918, loopback, etc.). Not applicable here — fixtures
|
||||
are baked into the image and loaded from local directories, not over the
|
||||
network.
|
||||
- `PORT` is injected by Railway but not read by the current startCommand
|
||||
(port is hardcoded to `4010`). Harmless.
|
||||
|
||||
## 7. How to reconstruct
|
||||
|
||||
If the Railway service is ever lost, recreate with the following recipe.
|
||||
Substitute `<service-id>`, `<environment-id>`, and `<public-domain>` with the
|
||||
concrete values from the Notion plan (section 9) or by querying Railway
|
||||
GraphQL directly.
|
||||
|
||||
1. Create a new service in the `showcase` project, `production` environment.
|
||||
Easiest path is the Railway UI (New Service → Docker Image), but the
|
||||
GraphQL `serviceCreate` mutation works too.
|
||||
2. Set `source.image` to the `showcase-aimock` image published by
|
||||
`.github/workflows/showcase_build.yml` (the baked image from
|
||||
`showcase/aimock/Dockerfile`, which contains the fixture tree — see section 3) via `serviceInstanceUpdate`:
|
||||
|
||||
```graphql
|
||||
mutation {
|
||||
serviceInstanceUpdate(
|
||||
serviceId: "<service-id>"
|
||||
environmentId: "<environment-id>"
|
||||
input: { source: { image: "<showcase-aimock-image-ref>" } }
|
||||
) {
|
||||
id
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Deploying the bare upstream `ghcr.io/copilotkit/aimock` instead will boot
|
||||
with no fixtures baked in — every request falls through to the proxy.
|
||||
|
||||
3. Set `startCommand` to the block in section 5 (join with spaces, escape as
|
||||
needed) via the same `serviceInstanceUpdate` mutation with
|
||||
`input: { startCommand: "..." }`. Remember Railway's startCommand overrides
|
||||
the image's Docker ENTRYPOINT, so the full `node /app/dist/cli.js` bin
|
||||
invocation must appear explicitly in the command string.
|
||||
4. No env vars needed for default setup (see section 6).
|
||||
5. Generate a public domain (`serviceDomainCreate` mutation, or the UI's
|
||||
"Generate Domain" button). The historical domain pattern is
|
||||
`showcase-aimock-production.<railway-edge>` — the current domain is in the
|
||||
Notion plan (section 9) and visible in the Railway dashboard.
|
||||
6. Deploy with `serviceInstanceDeployV2` (do NOT use `serviceInstanceRedeploy`
|
||||
— it replays the last snapshot, which may predate the image/startCommand
|
||||
change):
|
||||
|
||||
```graphql
|
||||
mutation {
|
||||
serviceInstanceDeployV2(
|
||||
serviceId: "<service-id>"
|
||||
environmentId: "<environment-id>"
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
7. Verify (find the current public domain via Railway GraphQL's `domains`
|
||||
field or the service's Railway dashboard):
|
||||
|
||||
```sh
|
||||
curl -sS -X POST https://<public-domain>/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer test" \
|
||||
-d '{"model":"gpt-4","messages":[{"role":"user","content":"Respond with exactly: OK"}]}'
|
||||
```
|
||||
|
||||
The payload matches the `shared/smoke.json` fixture (`userMessage:
|
||||
"Respond with exactly: OK"`), so expect its `"OK"` response. If
|
||||
proxy-fallthrough to OpenAI fires instead, the smoke fixture did not load —
|
||||
confirm the deployed image is the baked `showcase-aimock` image and that
|
||||
`startCommand` passes `--fixtures /fixtures` (the baked fixture tree).
|
||||
|
||||
8. Update any showcase services whose `OPENAI_BASE_URL` points at the old
|
||||
URL, if the domain changed during reconstruction.
|
||||
|
||||
## 8. The Dockerfile is LIVE
|
||||
|
||||
The `Dockerfile` in this directory is **not** dead code — it is the image that
|
||||
Railway deploys. `.github/workflows/showcase_build.yml` builds it (matrix entry
|
||||
`showcase-aimock`, with `dockerfile: showcase/aimock/Dockerfile` and context
|
||||
`showcase/aimock`) and publishes the `showcase-aimock` image. The Dockerfile is
|
||||
`FROM ghcr.io/copilotkit/aimock:latest` and bakes the fixture tree into the
|
||||
image:
|
||||
|
||||
```dockerfile
|
||||
FROM ghcr.io/copilotkit/aimock:latest
|
||||
|
||||
# Depth-organized fixture directories
|
||||
COPY shared/ /fixtures/shared/
|
||||
COPY d4/ /fixtures/d4/
|
||||
COPY d6/ /fixtures/d6/
|
||||
```
|
||||
|
||||
Do not remove it — deleting it would strip the baked-in fixtures and the
|
||||
deployed mock would serve nothing (all requests would fall through to the
|
||||
proxy).
|
||||
|
||||
## 9. Related references
|
||||
|
||||
- **Notion plan (authoritative source for concrete IDs and current domain):**
|
||||
<https://www.notion.so/34a3aa38185281148ae1ff7e2926c9d6>
|
||||
- aimock release process: `CopilotKit/aimock` repo CHANGELOG
|
||||
- Fixture propagation: edit files under `showcase/aimock/{shared,d4,d6}/`,
|
||||
which triggers `showcase_build.yml` to rebuild the `showcase-aimock` image;
|
||||
changes take effect on the next Railway deploy of the rebuilt image.
|
||||
- Railway docs (deploy mutations):
|
||||
<https://docs.railway.com/reference/public-api>
|
||||
@@ -0,0 +1,90 @@
|
||||
# Showcase aimock
|
||||
|
||||
Tagline: aimock fixture-directory layout, context-routing semantics, schema
|
||||
validation, drift-risk surface, and the manual add/update fixture flow. For
|
||||
service reconstruction (Railway image / startCommand / fixture URLs) see
|
||||
[`./RAILWAY.md`](./RAILWAY.md). For matcher semantics / fixture-authoring
|
||||
gotchas (sequenceIndex, hasToolResult, context mirroring) see
|
||||
[`../GOTCHAS.md`](../GOTCHAS.md).
|
||||
|
||||
Deterministic LLM fixture server for showcase E2E testing. Replaces real LLM API calls (OpenAI, Anthropic, Gemini) with pre-recorded responses so Playwright tests can run PR-gated in CI without API keys and without rate limits or non-determinism.
|
||||
|
||||
Railway pulls [`ghcr.io/copilotkit/aimock:latest`](https://github.com/orgs/CopilotKit/packages/container/package/aimock) directly (no wrapper image). The fixtures in this directory are loaded at boot via GitHub raw URLs configured in the Railway service's `startCommand`.
|
||||
|
||||
## What aimock is
|
||||
|
||||
aimock ([`@copilotkit/aimock`](https://www.npmjs.com/package/@copilotkit/aimock)) is a general-purpose LLM mock server. It speaks the OpenAI, Anthropic, and Gemini REST shapes (including SSE streaming), loads fixtures from disk at startup, and responds to incoming chat completions by matching the user's message text against fixture `match` criteria.
|
||||
|
||||
The showcase deployment runs aimock in proxy mode — `--proxy-only` with real upstream URLs configured for each provider. Unmatched requests are forwarded to the real API; matched requests short-circuit with the fixture response. This makes the sidecar safe to deploy as a general-purpose smoke-test aid: tests that hit fixture-matched prompts get deterministic responses, and anything else just falls through.
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
showcase/aimock/
|
||||
shared/ Fixtures loaded by ALL integrations (smoke, universal prompts)
|
||||
d4/ D4-depth fixtures — per-integration, single-demo coverage
|
||||
<slug>/ One directory per integration slug (e.g. langgraph-python/)
|
||||
d6/ D6-depth fixtures — per-integration, all-pills coverage
|
||||
<slug>/ One directory per integration slug
|
||||
feature-parity.json Legacy flat fixture file (pre-context-routing)
|
||||
smoke.json Minimal smoke fixture
|
||||
README.md This file
|
||||
```
|
||||
|
||||
**Context routing.** D4 and D6 fixtures use aimock's `--context-field` flag to scope fixture matching by integration. Each integration's dev server passes its slug as the `context` value in LLM requests (via `X-AIMock-Context` header or request body field). Aimock only considers fixtures whose `match.context` equals the incoming context value, so `d4/langgraph-python/` fixtures never interfere with `d4/mastra/` fixtures even if they share the same `userMessage` pattern.
|
||||
|
||||
**Per-integration isolation.** Every `<slug>/` directory contains fixtures specific to that integration. This prevents cross-contamination: if `mastra` needs a different tool name than `langgraph-python` for the same demo, each has its own fixture file. The `shared/` directory holds fixtures that apply regardless of context (e.g., smoke checks, universal greeting prompts).
|
||||
|
||||
## Fixtures in this directory
|
||||
|
||||
- **`feature-parity.json`** — 35+ fixtures covering the nine showcase demos across 17 packages: agentic chat (weather, backgrounds, themes), tool rendering (pie/bar charts, weather cards), HITL (plans, steps, approvals), Sales Dashboard (deals, pipelines, todos), and assorted meeting/flight/greeting prompts. Consumed by the per-package `test_e2e-showcase-on-demand` Playwright suites and loaded at Railway boot via GitHub raw URL.
|
||||
- **`smoke.json`** — a single minimal fixture (`userMessage: "Respond with exactly: OK"` → `content: "OK"`). Used by `/api/smoke` endpoints in each package to verify the aimock → package → UI round-trip without depending on a real agent.
|
||||
|
||||
Fixture match semantics: `userMessage` is a substring match against the last user turn. First fixture to match wins, so more specific prompts should appear before more generic ones (see the `"Based on the following context, write a concise"` entry that precedes the generic `report` / `plan` fixtures to protect CrewAI's startup probe).
|
||||
|
||||
## Sync policy
|
||||
|
||||
**Fixtures are hand-maintained.** There is no automated capture, no scheduled re-recording, and no drift-detection job that compares fixture responses against what a real LLM would say. The authoritative behavior is whatever is checked in.
|
||||
|
||||
The safety net is two-layered load-time validation, not behavioral verification:
|
||||
|
||||
1. **Load-time schema validation** (`--validate-on-load` in the Railway `startCommand` and in every test entrypoint that boots aimock) — the container refuses to start if any fixture uses an unrecognized response key (e.g. `text` instead of `content`). See [#3973](https://github.com/CopilotKit/CopilotKit/pull/3973).
|
||||
2. **CI schema validation** (`showcase/scripts/__tests__/aimock-fixtures.test.ts`) — the `showcase_validate` workflow runs `loadFixtureFile` + `validateFixtures` from `@copilotkit/aimock` against every `showcase/aimock/*.json` on every PR. A broken fixture fails the PR before merge.
|
||||
|
||||
Neither layer catches **behavioral drift** — if a package's agent code changes what it asks the LLM (new prompt, new tool, renamed tool), the existing fixture keeps matching and keeps returning the old response. The test either keeps passing (wrong assertion) or fails at the UI-assertion layer (missing tool call, missing text), and a human has to trace it back to the fixture.
|
||||
|
||||
## Adding or updating a fixture
|
||||
|
||||
The process is manual. There is no CLI for this directory specifically — aimock's upstream `--record` mode can proxy real API calls and write fixtures, but the showcase repo does not wire it up and does not commit recorded fixtures.
|
||||
|
||||
1. Identify the user prompt your test issues and decide what response you need (plain text, a tool call, an error).
|
||||
2. Add an entry to `feature-parity.json` under `fixtures`. Keep more specific `userMessage` matches above more generic ones. Valid response keys: `content`, `toolCalls`, `error`, `embedding`.
|
||||
3. Run the fixture-validation suite locally:
|
||||
```
|
||||
pnpm --filter @copilotkit/showcase-scripts test aimock-fixtures
|
||||
```
|
||||
4. Run the per-package E2E against the new fixture:
|
||||
```
|
||||
./showcase/scripts/run-e2e-with-aimock.sh <slug> [test-filter]
|
||||
```
|
||||
5. Ship it. Fixture changes take effect on the next Railway service restart (aimock fetches fixtures from GitHub raw URLs at boot).
|
||||
|
||||
When a package's agent code changes in a way that changes its LLM calls, the person making the change is responsible for updating the corresponding fixture. There is no automation to remind you.
|
||||
|
||||
## Drift risk
|
||||
|
||||
Drift surfaces as **flaky or silently-wrong E2E tests**, not as a dedicated signal. Symptoms and how to respond:
|
||||
|
||||
- **Playwright assertion fails** on a UI element that depends on a tool call (`WeatherCard` missing, chart not rendering) → the agent is now calling a differently-named tool than the fixture has; update the fixture's `toolCalls[].name` / `arguments`.
|
||||
- **Assertion on assistant text fails** → the agent's prompt changed; either update the fixture's `match.userMessage` to the new prompt substring or update the fixture's `content`.
|
||||
- **`smoke.json` healthcheck fails** against a deployed package (`/api/smoke` returns non-OK) → either the package's smoke route changed or aimock is down; check the Railway service and the smoke-monitor workflow.
|
||||
- **Container fails to start post-deploy** → load-time validation caught a broken fixture; CI should have caught it first, investigate why it didn't.
|
||||
|
||||
There is no scheduled drift-detection job that compares fixture responses against live LLM output. If this becomes a problem, the path forward is to wire aimock's `--record` mode into a periodic workflow that re-captures against real providers and diffs against checked-in fixtures — but that's not built today.
|
||||
|
||||
## Related workflows
|
||||
|
||||
- **`test_e2e-showcase-on-demand.yml`** (historically `showcase_aimock-e2e.yml`) — triggered by `/test-aimock <slug>` PR comments or `workflow_dispatch`. Installs `@copilotkit/aimock@latest`, boots it with `feature-parity.json`, spins up the target package's dev server against `OPENAI_BASE_URL=http://localhost:4010/v1`, and runs the package's Playwright suite. Posts pass/fail back to the PR.
|
||||
- **`showcase_validate.yml`** — runs fixture schema validation (`aimock-fixtures.test.ts`) on every PR that touches `showcase/**`.
|
||||
- **`showcase_deploy.yml`** — builds and deploys all showcase services. aimock is no longer in this workflow's matrix (Railway pulls the upstream `ghcr.io/copilotkit/aimock:latest` image directly).
|
||||
- **`showcase_smoke-monitor.yml`** — every 15 minutes, polls `/api/smoke` on all deployed showcase packages. Those smoke endpoints internally hit aimock's `smoke.json` fixture to verify the full stack.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,994 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for agno",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights from SFO to JFK",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "summarize",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "detailed",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "fun fact",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Anchored on 'Say hi' so it matches the sidebar greeting without colliding with the longer popup variant above.",
|
||||
"match": {
|
||||
"userMessage": "Say hi",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,999 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for claude-sdk-python",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Disabled: prior matcher 'weather' load-order-shadowed D6 pills including the productized reasoning-chain prompt 'Find flights from SFO to JFK and show me the weather there.' D4 loads before D6 and AIMock uses substring matching; downstream D6 fixtures own get_weather flows.",
|
||||
"match": {
|
||||
"userMessage": "_d4_unused_weather_probe_tokyo",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Narrowed from bare 'flights from SFO to JFK' so it does not shadow the D6 reasoning-chain prompt 'Find flights from SFO to JFK and show me the weather there.' D4 loads before D6 and AIMock uses substring matching; the beautiful-chat D4/D5 prompt includes 'for next Tuesday'.",
|
||||
"match": {
|
||||
"userMessage": "Find flights from SFO to JFK for next Tuesday",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D4 toolbar 'Summarize' button — narrowed from bare 'summarize' to 'Summarize the sales pipeline' (verbatim D4 toolbar probe) so it doesn't shadow D6 subagents pills like 'Summarize the current state of reusable rockets...'. Mirrors the LangGraph/TypeScript D4 narrowing.",
|
||||
"match": {
|
||||
"userMessage": "Summarize the sales pipeline",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D4 toolbar 'Detailed' button — tightened from bare 'detailed' so it doesn't shadow D6 agent-config 'responseLength:detailed — describe agent context per your config'. Anchored on the longer phrase that the D4 button actually sends.",
|
||||
"match": {
|
||||
"userMessage": "Make this more detailed",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Narrowed from bare 'Roll a 20-sided die' to the old D4 prompt with a period so it no longer shadows the D6 reasoning-chain pill 'Roll a 20-sided die for me and compare it to a smaller one.' D4 loads before D6 and AIMock uses substring matching, so the period is load-bearing.",
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die for me.",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "fun fact",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Anchored on 'Say hi' so it matches the sidebar greeting without colliding with the longer popup variant above.",
|
||||
"match": {
|
||||
"userMessage": "Say hi",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "claude-sdk-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,966 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for claude-sdk-typescript",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21",
|
||||
"_note": "Catch-all matchers here load BEFORE every d6/claude-sdk-typescript file (fixture load order is d4/* then d6/*, first match wins), so keep userMessage keys narrow: bare substrings like 'weather'/'detailed'/'flights from SFO to JFK' shadowed the d6 probe prompts and broke tool-rendering, tool-rendering-reasoning-chain, beautiful-chat-search-flights and agent-config (narrowed/removed 2026-06-11).",
|
||||
"updated": "2026-06-11"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Disabled: prior matcher 'What's the weather in San Francisco?' load-order-shadowed the D6 beautiful-chat/tool-rendering custom-catchall pill of the same name (d4 loads before d6, first match wins), and emitted the wrong toolCallId (call_fp_get_weather_001 vs the D6 canonical call_tr_weather_sf_001) so the follow-up content fixture was unreachable. Mirrors the LGP/ag2/built-in-agent/spring-ai _d4_unused_weather_probe_sf pattern — userMessage renamed to a non-matching sentinel so this entry is effectively dead; downstream D6 fixtures own the get_weather flow.",
|
||||
"match": {
|
||||
"userMessage": "_d4_unused_weather_probe_sf",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"San Francisco\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Find flights from SFO to JFK for next Tuesday",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError. chunkSize MUST live at the fixture root (alongside match/response) — server.ts:681 reads `fixture.chunkSize`, NOT `response.chunkSize`.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"chunkSize": 9999,
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D4 toolbar 'Summarize' button — narrowed from bare 'summarize' to 'Summarize the sales pipeline' (verbatim D4 toolbar probe) so it doesn't shadow D6 subagents pills like 'Summarize the current state of reusable rockets...' whose response is wildly off-topic for a subagents probe. Mirrors the LGP d4 narrowing.",
|
||||
"match": {
|
||||
"userMessage": "Summarize the sales pipeline",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar). Narrowed userMessage from bare 'fun fact' to 'fun fact for the prebuilt sidebar' so it does not shadow the D6 headless-simple pill 'Give me a fun fact.' Mirrors the LGP d4 narrowing.",
|
||||
"match": {
|
||||
"userMessage": "fun fact for the prebuilt sidebar",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Anchored on 'Say hi' so it matches the sidebar greeting without colliding with the longer popup variant above.",
|
||||
"match": {
|
||||
"userMessage": "Say hi",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "claude-sdk-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,994 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for crewai-crews",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights from SFO to JFK",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "summarize",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "detailed",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die",
|
||||
"hasToolResult": false,
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "fun fact",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Anchored on 'Say hi' so it matches the sidebar greeting without colliding with the longer popup variant above.",
|
||||
"match": {
|
||||
"userMessage": "Say hi",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "crewai-crews"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,999 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for google-adk",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Disabled: prior matcher 'weather' load-order-shadowed D6 pills like 'What's the weather in San Francisco?' (custom-catchall) and 'Chain a few tools ... weather in Tokyo' (default-catchall). Mirrors LGP's d4 fix — userMessage renamed to a non-matching sentinel so this entry is effectively dead; downstream D6 fixtures own the get_weather flow.",
|
||||
"match": {
|
||||
"userMessage": "_d4_unused_weather_probe_sf",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Disabled: 'flights from SFO to JFK for next Tuesday' was the exact pill prompt for the D6 beautiful-chat 'Search Flights (A2UI Fixed Schema)' demo, so this d4 fixture load-order-shadowed the d6 beautiful-chat fixture (d4 loads before d6). Mirrors LGP's d4 fix — userMessage renamed to a non-matching sentinel so this entry is effectively dead; the d6 beautiful-chat fixture now owns this pill.",
|
||||
"match": {
|
||||
"userMessage": "_d4_unused_flights_probe_sfo_jfk",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError. chunkSize MUST live at the fixture root (alongside match/response) — server.ts:681 reads `fixture.chunkSize`, NOT `response.chunkSize`.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"chunkSize": 9999,
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D4 toolbar 'Summarize' button — narrowed from 'summarize' to 'Summarize the sales pipeline' (verbatim D4 toolbar probe). The prior bare 'summarize' matched the D6 subagents pill 'Summarize the current state of reusable rockets...' and the response (sales-pipeline summary) is wildly off-topic for a subagents probe.",
|
||||
"match": {
|
||||
"userMessage": "Summarize the sales pipeline",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Make this more detailed",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Narrowed from bare 'Roll a 20-sided die' to 'Roll a 20-sided die for me' so it no longer shadows the D6 tool-rendering pill 'Roll a 20-sided die.' (which chains 5 roll_d20 calls). The d4/d5-reasoning-chain probe uses the 'for me' phrasing.",
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die for me",
|
||||
"hasToolResult": false,
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar). Narrowed userMessage from bare 'fun fact' to 'fun fact for the prebuilt sidebar' so it ONLY matches a hypothetical sidebar-specific probe (none currently exists). The narrow gate prevents this from shadowing the D6 headless-simple pill 'Give me a fun fact.' which expects a DIFFERENT leading phrase ('A fun fact: Honey never spoils!').",
|
||||
"match": {
|
||||
"userMessage": "fun fact for the prebuilt sidebar",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Narrowed userMessage from 'Say hi' to 'Say hi!' (with exclamation — verbatim prebuilt-sidebar pill text) so this no longer shadows the D6 shared-state-read-write pill 'Say hi and introduce yourself.' (no exclamation, different intended response). The exclamation is unique to the prebuilt-sidebar pill among current LGP demos.",
|
||||
"match": {
|
||||
"userMessage": "Say hi!",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "google-adk"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,994 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for langgraph-fastapi",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights from SFO to JFK",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "summarize",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "detailed",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "fun fact",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Anchored on 'Say hi' so it matches the sidebar greeting without colliding with the longer popup variant above.",
|
||||
"match": {
|
||||
"userMessage": "Say hi",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "langgraph-fastapi"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,997 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for langgraph-typescript",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "_d4_unused_weather_probe_tokyo",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights from SFO to JFK for next Tuesday",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D4 toolbar 'Summarize' button — further narrowed from 'Summarize the' to 'Summarize the sales pipeline' (verbatim D4 toolbar probe). The prior 'Summarize the' still matched the D6 subagents pill 'Summarize the current state of reusable rockets...' and the response (sales-pipeline summary) is wildly off-topic for a subagents probe, breaking the subagents e2e regression test for the delegations reducer. 'Summarize the sales pipeline' is unique to the D4 toolbar probe. Mirrors the LGP fix.",
|
||||
"match": {
|
||||
"userMessage": "Summarize the sales pipeline",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D4 toolbar 'Detailed' button — tightened from bare 'detailed' so it doesn't shadow d6 agent-config 'responseLength:detailed — describe agent context per your config'. Anchored on the longer phrase that the d4 button actually sends.",
|
||||
"match": {
|
||||
"userMessage": "Make this more detailed",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Narrowed from bare 'Roll a 20-sided die' to 'Roll a 20-sided die for me' so it no longer shadows the D6 tool-rendering / tool-rendering-default-catchall pill 'Roll a 20-sided die.' (which chains 5 roll_d20 calls). The d4/d5-reasoning-chain probe uses the 'for me' phrasing. Mirrors the LGP fix.",
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die for me",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "fun fact",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "A fun fact: Honey never spoils! Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Narrowed userMessage from 'Say hi' to 'Say hi!' (with exclamation — verbatim prebuilt-sidebar pill text) so this no longer shadows the D6 shared-state-read-write pill 'Say hi and introduce yourself.' (no exclamation, different intended response). The exclamation is unique to the prebuilt-sidebar pill among current LGT demos. Mirrors the LGP fix.",
|
||||
"match": {
|
||||
"userMessage": "Say hi!",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "langgraph-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,994 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for langroid",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights from SFO to JFK",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "summarize",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "detailed",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die",
|
||||
"hasToolResult": false,
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "fun fact",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Anchored on 'Say hi' so it matches the sidebar greeting without colliding with the longer popup variant above.",
|
||||
"match": {
|
||||
"userMessage": "Say hi",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "langroid"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,997 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for llamaindex",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Chat-demo weather pill — emit get_weather. toolName gate added so the bare 'weather' substring only fires when the requesting agent actually registers get_weather. Without it, the tool-free voice agent's prompt 'What is the weather in Tokyo?' (substring 'weather') leaked into this fixture, emitting a get_weather call the voice agent could never resolve → the voice cell hung (done-signal-missing). Mirrors the toolName gate used by d6 tool-rendering.json's get_weather fixture.",
|
||||
"match": {
|
||||
"userMessage": "weather",
|
||||
"toolName": "get_weather",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights from SFO to JFK",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D4 toolbar 'Summarize' button — narrowed from bare 'summarize' to 'Summarize the sales pipeline' (verbatim D4 toolbar probe), matching langgraph-python parity. The prior bare 'summarize' substring-matched the D6 gen-ui-agent pill 'Research our top competitor and summarize their strengths and weaknesses.' and returned this sales-pipeline text instead of the gen-ui-agent set_steps fixture, so the competitor pill produced no/duplicate steps. 'Summarize the sales pipeline' is unique to the D4 toolbar probe.",
|
||||
"match": {
|
||||
"userMessage": "Summarize the sales pipeline",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "detailed",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die",
|
||||
"hasToolResult": false,
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "fun fact",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Anchored on 'Say hi' so it matches the sidebar greeting without colliding with the longer popup variant above.",
|
||||
"match": {
|
||||
"userMessage": "Say hi",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "llamaindex"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,997 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for mastra",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21",
|
||||
"note": "ported from langgraph-python d4 chat.json; context rewritten to mastra (tightened over-broad catchalls)"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard \u2014 total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22\u00b0C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above \u2014 Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace \u2014 the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved \u2014 processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected \u2014 calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it \u2014 I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research \u2192 drafting \u2192 critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above \u2014 Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above \u2014 Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above \u2014 Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above \u2014 Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above \u2014 Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above \u2014 Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above \u2014 monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled \u2014 calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above \u2014 United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent \u2014 about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read \u2014 e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** \u2014 a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350\u00b0F (175\u00b0C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22\u00b0C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights from SFO to JFK",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done \u2014 added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"\ud83d\udcda\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"\ud83d\ude80\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"\ud83c\udfaf\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture \u2014 anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture \u2014 anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo \u2014 a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 \u2014 Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 \u2014 Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 \u2014 Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test \u2014 they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory \u2014 which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot \u2014 now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at \u2014 it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n\u2014 The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') \u2014 those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D4 toolbar 'Summarize' button \u2014 tightened from bare 'summarize' to 'Summarize the' (capital, anchor) so it doesn't shadow d6 fixtures whose user prompts happen to contain the word 'summarize' mid-sentence (e.g. gen-ui-agent pill 3 'Research our top competitor and summarize their strengths and weaknesses').",
|
||||
"match": {
|
||||
"userMessage": "Summarize the",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D4 toolbar 'Detailed' button \u2014 tightened from bare 'detailed' so it doesn't shadow d6 agent-config 'responseLength:detailed \u2014 describe agent context per your config'. Anchored on the longer phrase that the d4 button actually sends.",
|
||||
"match": {
|
||||
"userMessage": "Make this more detailed",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture \u2014 anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected \u2014 calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call \u2014 discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture \u2014 anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart \u2014 installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die",
|
||||
"hasToolResult": false,
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion \u2014 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math \u2014\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made \u2014 but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion \u2014 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 \u00f7 2 = 8.5** \u2014 not a whole number, so 2 is not a factor.\n3. **17 \u00f7 3 \u2248 5.67** \u2014 not a whole number, so 3 is not a factor.\n4. **17 \u00f7 4 = 4.25** \u2014 not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion \u2014 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion \u2014 'Give me a fun fact.' (prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "fun fact",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion \u2014 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything \u2014 a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion \u2014 'Say hi!' (prebuilt-sidebar). Anchored on 'Say hi' so it matches the sidebar greeting without colliding with the longer popup variant above.",
|
||||
"match": {
|
||||
"userMessage": "Say hi",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test \u2014 first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test \u2014 second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "mastra"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,994 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for pydantic-ai",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights from SFO to JFK",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "summarize",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "detailed",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die",
|
||||
"hasToolResult": false,
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "fun fact",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Anchored on 'Say hi' so it matches the sidebar greeting without colliding with the longer popup variant above.",
|
||||
"match": {
|
||||
"userMessage": "Say hi",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "pydantic-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,983 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for spring-ai",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Disabled: prior matcher 'weather' load-order-shadowed D6 pills like 'What's the weather in San Francisco?' (custom-catchall) and 'Chain a few tools ... weather in Tokyo' (default-catchall). Mirrors the LGP d4 disabled-weather pattern (_d4_unused_weather_probe_sf) — userMessage renamed to a non-matching sentinel so this entry is effectively dead; downstream D6 fixtures own the get_weather flow.",
|
||||
"match": {
|
||||
"userMessage": "_d4_unused_weather_probe_sf",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Disabled: 'flights from SFO to JFK' was the exact pill prompt for the D6 beautiful-chat 'Search Flights' demo, so this d4 fixture load-order-shadowed the d6 beautiful-chat fixture (d4 loads before d6). The d4 chat-roundtrip probe does not exercise the flights tool, so this entry is dead. Mirrors the LGP _d4_unused_flights_probe_sfo_jfk pattern.",
|
||||
"match": {
|
||||
"userMessage": "_d4_unused_flights_probe_sfo_jfk",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError. chunkSize MUST live at the fixture root (alongside match/response) — server.ts:681 reads `fixture.chunkSize`, NOT `response.chunkSize`.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"chunkSize": 9999,
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D4 toolbar 'Summarize' button — narrowed from bare 'summarize' to 'Summarize the sales pipeline' (verbatim D4 toolbar probe) so it doesn't shadow D6 subagents pills like 'Summarize the current state of reusable rockets...' whose response is wildly off-topic for a subagents probe. Mirrors the LGP d4 narrowing.",
|
||||
"match": {
|
||||
"userMessage": "Summarize the sales pipeline",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Make this more detailed",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar). Narrowed userMessage from bare 'fun fact' to 'fun fact for the prebuilt sidebar' so it does not shadow the D6 headless-simple pill 'Give me a fun fact.' Mirrors the LGP d4 narrowing.",
|
||||
"match": {
|
||||
"userMessage": "fun fact for the prebuilt sidebar",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Anchored on 'Say hi' so it matches the sidebar greeting without colliding with the longer popup variant above.",
|
||||
"match": {
|
||||
"userMessage": "Say hi",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "spring-ai"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,994 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for strands",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights from SFO to JFK",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize the sales pipeline",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "detailed",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die",
|
||||
"hasToolResult": false,
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "fun fact",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Anchored on 'Say hi' so it matches the sidebar greeting without colliding with the longer popup variant above.",
|
||||
"match": {
|
||||
"userMessage": "Say hi",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "strands-typescript"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,994 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D4 chat fixtures for strands",
|
||||
"sourceFile": "feature-parity.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your sales dashboard — total revenue is $1.2M (+12% MoM), 342 new customers, and 4.2% conversion. The pie shows revenue split by category and the bar chart tracks monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_design_a2ui_surface_sales_dashboard_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_render_a2ui_sales_dashboard_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"beautiful-chat-sales-dashboard\",\"catalogId\":\"copilotkit://app-dashboard-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"children\":[\"row-metrics\",\"row-charts\"],\"gap\":16},{\"id\":\"row-metrics\",\"component\":\"Row\",\"children\":[\"card-revenue\",\"card-customers\",\"card-conversion\"],\"gap\":16},{\"id\":\"card-revenue\",\"component\":\"DashboardCard\",\"title\":\"Total Revenue\",\"child\":\"metric-revenue\"},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Total Revenue\",\"value\":\"$1.2M\",\"trend\":\"up\",\"trendValue\":\"+12% MoM\"},{\"id\":\"card-customers\",\"component\":\"DashboardCard\",\"title\":\"New Customers\",\"child\":\"metric-customers\"},{\"id\":\"metric-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"342\",\"trend\":\"up\",\"trendValue\":\"+8% MoM\"},{\"id\":\"card-conversion\",\"component\":\"DashboardCard\",\"title\":\"Conversion Rate\",\"child\":\"metric-conversion\"},{\"id\":\"metric-conversion\",\"component\":\"Metric\",\"label\":\"Conversion Rate\",\"value\":\"4.2%\",\"trend\":\"neutral\"},{\"id\":\"row-charts\",\"component\":\"Row\",\"children\":[\"card-pie\",\"card-bar\"],\"gap\":16},{\"id\":\"card-pie\",\"component\":\"DashboardCard\",\"title\":\"Revenue by Category\",\"child\":\"pie-revenue\"},{\"id\":\"pie-revenue\",\"component\":\"PieChart\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]},{\"id\":\"card-bar\",\"component\":\"DashboardCard\",\"title\":\"Monthly Sales\",\"child\":\"bar-monthly\"},{\"id\":\"bar-monthly\",\"component\":\"BarChart\",\"data\":[{\"label\":\"Jan\",\"value\":50000},{\"label\":\"Feb\",\"value\":62000},{\"label\":\"Mar\",\"value\":58000},{\"label\":\"Apr\",\"value\":71000},{\"label\":\"May\",\"value\":80000},{\"label\":\"Jun\",\"value\":92000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"toolName": "generate_a2ui",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_get_weather_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_render_pie_chart_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice at $42,000, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_show_card_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_request_approval_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_book_call_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_set_notes_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_critique_agent_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_writing_agent_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"draft\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\",\"instructions\":\"Check factual claims, tighten prose, flag any unsupported assertions.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_research_agent_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"brief\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\",\"facts\":[\"Eliminating commutes returns roughly ten hours per week per employee.\",\"Repeated surveys show meaningfully higher job satisfaction among remote workers.\",\"Employers gain access to a geographically unbounded talent pool.\",\"Office overhead (leases, utilities, maintenance) drops significantly.\",\"Trade-offs include reduced ad-hoc collaboration, harder mentorship of junior staff, and erosion of cultural cohesion without intentional replacement rituals.\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate monthly spend, with Rent, Marketing, and Travel rounding out the breakdown."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_002",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Organic Search drives most traffic, followed by Direct, Social, and Referral."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_003",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Apple and Samsung lead, with Xiaomi third and the long tail under \"Others\"."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_002",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Q4 was the strongest quarter, with Q2 close behind."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_pie_chart_004",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_bar_chart_003",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — monthly expenses ranged from $12k in January to $15.8k in April."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_schedule_time_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_search_flights_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_toggle_theme_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is the largest continent?",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Asia is the largest continent — about 30% of Earth's land area, home to over 4.6 billion people."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read — e2e test sends 'What recipe am I making?' via the sidebar. The agent reads the shared recipe state (title, ingredients, instructions) and responds with a summary.",
|
||||
"match": {
|
||||
"userMessage": "What recipe am I making",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your current recipe state, you're making **Make Your Recipe** — a dish featuring carrots (3 large, grated) and all-purpose flour (2 cups). The first instruction is to preheat the oven to 350°F (175°C). It looks like a carrot-based bake, perhaps a carrot cake or savory carrot bread. Would you like me to suggest more ingredients or steps?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather summary",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "The current weather in Tokyo is 22°C with partly cloudy skies. Humidity is at 65% with light winds from the east at 12 km/h. Perfect weather for a walk outside!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of website traffic by source",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_002",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Website Traffic by Source\",\"description\":\"Traffic sources this month\",\"data\":[{\"label\":\"Organic Search\",\"value\":45},{\"label\":\"Direct\",\"value\":25},{\"label\":\"Social\",\"value\":18},{\"label\":\"Referral\",\"value\":12}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "smartphone market share by brand",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_003",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Smartphone Market Share\",\"description\":\"Global smartphone market share by brand\",\"data\":[{\"label\":\"Apple\",\"value\":28},{\"label\":\"Samsung\",\"value\":22},{\"label\":\"Xiaomi\",\"value\":14},{\"label\":\"Others\",\"value\":36}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly sales for Q1, Q2, Q3, Q4",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_002",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Quarterly Sales\",\"description\":\"Sales across Q1-Q4\",\"data\":[{\"label\":\"Q1\",\"value\":145000},{\"label\":\"Q2\",\"value\":178000},{\"label\":\"Q3\",\"value\":162000},{\"label\":\"Q4\",\"value\":215000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me the sales dashboard with metrics and a revenue chart",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"revenue-metric\",\"elements\":{\"revenue-metric\":{\"type\":\"MetricCard\",\"props\":{\"label\":\"Revenue (Q3)\",\"value\":\"$1.24M\",\"trend\":\"+18% vs Q2\"},\"children\":[\"revenue-bar\"]},\"revenue-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly revenue\",\"description\":\"Revenue by month across Q3\",\"data\":[{\"label\":\"Jul\",\"value\":380000},{\"label\":\"Aug\",\"value\":410000},{\"label\":\"Sep\",\"value\":450000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down revenue by category as a pie chart",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"category-pie\",\"elements\":{\"category-pie\":{\"type\":\"PieChart\",\"props\":{\"title\":\"Revenue by category\",\"description\":\"Share of total revenue by product category\",\"data\":[{\"label\":\"Enterprise\",\"value\":540000},{\"label\":\"SMB\",\"value\":310000},{\"label\":\"Self-serve\",\"value\":220000},{\"label\":\"Partner\",\"value\":170000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly expenses as a bar chart",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"root\":\"expense-bar\",\"elements\":{\"expense-bar\":{\"type\":\"BarChart\",\"props\":{\"title\":\"Monthly expenses\",\"description\":\"Operating expenses by month\",\"data\":[{\"label\":\"Jul\",\"value\":210000},{\"label\":\"Aug\",\"value\":225000},{\"label\":\"Sep\",\"value\":240000}]}}}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"Markdown\":{\"props\":{\"children\":\"## Q4 Sales Summary\"}}},{\"metric\":{\"props\":{\"label\":\"Total Revenue\",\"value\":\"$1.2M\"}}},{\"pieChart\":{\"props\":{\"title\":\"Revenue by Segment\",\"data\":\"[{\\\"label\\\":\\\"Enterprise\\\",\\\"value\\\":600000},{\\\"label\\\":\\\"SMB\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Startup\\\",\\\"value\\\":200000}]\"}}},{\"barChart\":{\"props\":{\"title\":\"Monthly Revenue\",\"data\":\"[{\\\"label\\\":\\\"Oct\\\",\\\"value\\\":350000},{\\\"label\\\":\\\"Nov\\\",\\\"value\\\":400000},{\\\"label\\\":\\\"Dec\\\",\\\"value\\\":450000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Break down Q4 revenue by product category as a pie chart. Include at least four segments with realistic sample values.",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"pieChart\":{\"props\":{\"title\":\"Q4 Revenue by Product Category\",\"data\":\"[{\\\"label\\\":\\\"Software\\\",\\\"value\\\":500000},{\\\"label\\\":\\\"Hardware\\\",\\\"value\\\":300000},{\\\"label\\\":\\\"Consulting\\\",\\\"value\\\":150000},{\\\"label\\\":\\\"Subscriptions\\\",\\\"value\\\":50000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me monthly operating expenses for the last six months as a bar chart with one bar per month.",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"ui\":[{\"barChart\":{\"props\":{\"title\":\"Monthly Operating Expenses\",\"data\":\"[{\\\"label\\\":\\\"Apr\\\",\\\"value\\\":205000},{\\\"label\\\":\\\"May\\\",\\\"value\\\":215000},{\\\"label\\\":\\\"Jun\\\",\\\"value\\\":222000},{\\\"label\\\":\\\"Jul\\\",\\\"value\\\":228000},{\\\"label\\\":\\\"Aug\\\",\\\"value\\\":234000},{\\\"label\\\":\\\"Sep\\\",\\\"value\\\":241000}]\"}}}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "revenue by category as a pie chart",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_pie_chart_004",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "monthly expenses as a bar chart",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_bar_chart_003",
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights from SFO to JFK",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "toggle",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — added three todos about learning CopilotKit to your task manager."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Large chunkSize keeps the manage_todos args (with 4-byte UTF-8 emoji) as a single SSE chunk. aimock chunks via JavaScript .slice() which operates on UTF-16 code units; emoji surrogate pairs split mid-codepoint produce unpaired surrogates that strict pydantic deserializers (e.g. agent_framework_ag_ui) reject with PydanticSerializationError.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_manage_todos_001",
|
||||
"name": "manage_todos",
|
||||
"arguments": "{\"todos\":[{\"id\":\"todo-cpk-1\",\"title\":\"Read the CopilotKit docs\",\"description\":\"Start with the quickstart and explore the core hooks.\",\"emoji\":\"📚\",\"status\":\"pending\"},{\"id\":\"todo-cpk-2\",\"title\":\"Build a CopilotKit prototype\",\"description\":\"Wire up a basic chat and register a frontend tool.\",\"emoji\":\"🚀\",\"status\":\"pending\"},{\"id\":\"todo-cpk-3\",\"title\":\"Explore shared agent state\",\"description\":\"Watch the canvas re-render as the agent writes to state.\",\"emoji\":\"🎯\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "three todos about learning CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_enable_app_mode_001",
|
||||
"name": "enableAppMode",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant 'hello' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'hello' (e.g. 'mellow', 'fellow'). Matches both 'hello' and 'Hello' as standalone greeting attempts only.",
|
||||
"match": {
|
||||
"userMessage": "hello, what can you do",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! I can help you with weather lookups, creating pie and bar charts, scheduling meetings, managing sales todos, searching flights, and toggling the theme. What would you like to do?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant help fixture — anchored on 'what can you help me with' so it doesn't substring-match arbitrary prompts that happen to contain 'help' (e.g. 'helpful', 'helping').",
|
||||
"match": {
|
||||
"userMessage": "what can you help me with",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are the things I can help with:\n- Check the weather for any location\n- Create pie charts and bar charts from data\n- Schedule meetings with a reason and duration\n- Manage your sales pipeline and todos\n- Search for flights between cities\n- Toggle the UI theme between light and dark\n\nJust ask me about any of these!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached PDF document is the CopilotKit Quickstart guide. It walks through three steps: install `@copilotkit/react-core` and `@copilotkit/react-ui`, wrap your app in a `<CopilotKit runtimeUrl=\"/api/copilotkit\">` provider, and drop a `<CopilotChat />` wherever you want the assistant to appear."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "The attached image is the CopilotKit logo — a clean, geometric mark used across CopilotKit branding."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a 3-day Tokyo trip for a solo traveler interested in food, art, and architecture. Keep it concise.",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a tight 3-day Tokyo loop for a solo traveler who wants food, art, and architecture in equal measure.\n\n**Day 1 — Shibuya & Harajuku**\n- Morning: breakfast at a kissaten in Shibuya, then walk Cat Street to Harajuku.\n- Afternoon: Nezu Museum (architecture by Kengo Kuma + traditional garden).\n- Evening: izakaya in Nonbei Yokocho.\n\n**Day 2 — Roppongi & Ginza**\n- Morning: Mori Art Museum and the Tokyo City View deck.\n- Afternoon: stroll Ginza, stop into Itoya, lunch at a sushi counter.\n- Evening: jazz at a Ginza basement bar.\n\n**Day 3 — Yanaka & Asakusa**\n- Morning: Yanaka Ginza for old-Tokyo atmosphere and craft shops.\n- Afternoon: Senso-ji and the Asakusa back-streets; consider the Sumida River walk to the Tadao Ando-designed 21_21 if you want one more architecture stop.\n- Evening: ramen near Ueno before catching the train.\n\nLet me know if you want a vegetarian variant or a more architecture-heavy version."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain retrieval-augmented generation as if I were 12. Use a simple analogy.",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Imagine you're taking an open-book test.\n\nA regular language model is like a kid who studied really hard but isn't allowed to look at the textbook during the test — they have to answer from memory, and sometimes they confidently make stuff up.\n\nRetrieval-Augmented Generation (RAG) is the same kid, but now they're allowed to flip through the textbook before answering each question. First they look up the relevant pages (that's the *retrieval* step), then they write the answer using what they just read (that's the *generation* step).\n\nSo RAG = \"look it up, then answer.\" The model still talks like itself, but the facts come from the book instead of from its memory — which means fewer made-up answers, and you can swap the book out whenever your information changes."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Draft a short, upbeat launch email announcing a new AI-powered chat feature. 3 short paragraphs max.",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "**Subject:** Meet your new AI co-pilot — now built right into the app\n\nHi there,\n\nWe just shipped something we're really excited about: an AI-powered chat that lives inside the product you already use. Ask it questions, hand off tasks, or just have it summarize what you're looking at — it has full context of your workspace, so you don't have to copy-paste anything.\n\nIt's rolling out today to every account, no setup required. Open the app, look for the new chat button in the corner, and try it on your messiest task. We'd love to hear what you build with it.\n\n— The team"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped showcase-assistant intro fixture. Anchored on 'Hi, who are you' so it doesn't substring-match arbitrary prompts that happen to contain 'hi' inside other words (e.g. 'this', 'history', 'while') — those caused subagents writer/critique sub-LLM calls to return this boilerplate, breaking the demo.",
|
||||
"match": {
|
||||
"userMessage": "Hi, who are you",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your showcase assistant. I can help with weather, charts, meetings, sales todos, flights, and theme toggling. What would you like to try?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Based on the following context, write a concise",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "A short input value used to parameterize the crew's tasks and agents."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize the sales pipeline",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a summary of your current sales pipeline:\n\n- **Total Pipeline Value**: $185,000\n- **Active Deals**: 5 deals across Prospect, Qualified, and Proposal stages\n- **Top Deal**: Acme Corp at $50,000 (Proposal stage)\n- **Win Rate**: 62% this quarter\n\nThe pipeline is healthy with good distribution across stages. Would you like to drill into any specific metric?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expand",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Project Proposal: Q2 Initiative\n\n## Executive Summary\nThis proposal outlines the strategic objectives, timeline, and resource allocation for the Q2 project initiative.\n\n## Objectives\n1. Increase customer engagement by 25%\n2. Launch three new product features\n3. Reduce operational costs by 15%\n\n## Timeline\n- **Week 1-2**: Discovery and requirements gathering\n- **Week 3-6**: Development and implementation\n- **Week 7-8**: Testing and quality assurance\n- **Week 9-10**: Staged rollout and monitoring\n\n## Budget\nEstimated total: $45,000\n- Development: $25,000\n- Testing: $10,000\n- Infrastructure: $10,000\n\n## Success Metrics\n- Customer satisfaction score > 4.5\n- Feature adoption rate > 60%\n- Zero critical bugs at launch"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "rewrite",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Team,\n\nI am writing to formally document the proceedings and key decisions from today's meeting.\n\n**Attendees**: Full project team\n**Date**: Today's session\n\n**Key Discussion Points**:\n1. Project timeline was reviewed and approved\n2. Resource allocation was confirmed\n3. Next milestones were established\n\n**Action Items**:\n- Complete phase one deliverables by end of week\n- Schedule follow-up review meeting\n- Distribute updated project documentation\n\nPlease review and confirm your respective action items.\n\nBest regards"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "detailed",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "# Budget Report - Q3\n\n## Overview\n- **Total Budget**: $150,000\n- **Spent to Date**: $89,500 (59.7%)\n- **Remaining**: $60,500\n\n## Breakdown by Category\n- **Personnel**: $52,000 (58.1% of spend)\n- **Infrastructure**: $18,500 (20.7%)\n- **Software Licenses**: $12,000 (13.4%)\n- **Miscellaneous**: $7,000 (7.8%)\n\n## Key Highlights\n- Personnel costs are tracking 5% under budget\n- Infrastructure spend increased due to scaling needs\n- Software license renewal saved $3,000 through annual plan\n\n## Recommendations\n- Reallocate $5,000 from personnel to infrastructure\n- Prepay Q4 licenses for additional savings"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "formal letter",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Dear Sir/Madam,\n\nI am writing to formally present the enclosed draft text for your review and consideration.\n\nThe document has been prepared in accordance with standard formatting guidelines and organizational protocols. All relevant sections have been reviewed for accuracy and completeness.\n\nPlease find the revised content enclosed. Should you require any modifications or have questions regarding the content, please do not hesitate to contact me at your earliest convenience.\n\nI look forward to your feedback.\n\nYours sincerely,\nThe Document Editor"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped sales-pipeline 'add a deal' fixture — anchored on the full intent phrase so it doesn't substring-match arbitrary prompts that happen to contain 'deal' (e.g. 'dealing with', 'idealized').",
|
||||
"match": {
|
||||
"userMessage": "add a new enterprise deal",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sample deals",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "manage_sales_todos",
|
||||
"arguments": "{\"todos\": [{\"id\": \"new-1\", \"title\": \"New Enterprise Deal\", \"stage\": \"prospect\", \"value\": 50000, \"dueDate\": \"2026-05-01\", \"assignee\": \"Alice\", \"completed\": false}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "flights to Paris",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"Air France\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=airfrance.com&sz=128\",\"flightNumber\":\"AF85\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"16:00\",\"arrivalTime\":\"11:30+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$780\",\"currency\":\"USD\"},{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA990\",\"origin\":\"SFO\",\"destination\":\"CDG\",\"date\":\"Mon, Apr 21\",\"departureTime\":\"18:15\",\"arrivalTime\":\"13:45+1\",\"duration\":\"10h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$720\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_hitl_book_intro_sales_001",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the intro call with the sales team for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Please book an intro call with the sales team to discuss pricing.",
|
||||
"toolName": "book_call",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_intro_sales_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Intro call — discuss pricing\",\"attendee\":\"Sales team\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What name did I just give",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "You said your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'what city do I live in' fixture — anchored so it doesn't substring-match arbitrary prompts containing 'city' (e.g. 'capacity', 'velocity', 'specificity').",
|
||||
"match": {
|
||||
"userMessage": "what city do I live in",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on our conversation, you said you live in Tokyo!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "sunset-themed gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ffd194 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Describe this image",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "This appears to be the CopilotKit logo. The mark combines clean geometric shapes typical of the CopilotKit brand identity."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize this document",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "This is an excerpt from the CopilotKit documentation covering the quickstart — installing the packages and wrapping the app in a CopilotKitProvider pointing at a runtime endpoint."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "blue gradient",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "change_background",
|
||||
"arguments": "{\"background\":\"linear-gradient(135deg, #3b82f6 0%, #1e40af 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "short joke",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the developer go broke? Because they used up all their cache."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Search my notes for 'auth'",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "xyzzy-nonsense-keyword",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"xyzzy-nonsense-keyword\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of monthly expenses",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_bar_chart",
|
||||
"arguments": "{\"title\":\"Monthly Expenses\",\"description\":\"Expense breakdown by month\",\"data\":[{\"label\":\"Jan\",\"value\":12000},{\"label\":\"Feb\",\"value\":14500},{\"label\":\"Mar\",\"value\":13200},{\"label\":\"Apr\",\"value\":15800}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What is my name?",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on the context you provided, your name is Jamie."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die",
|
||||
"hasToolResult": false,
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: typed-message test sends 'Say hello in one word.'",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one word",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write a short sonnet about AI.' (agentic-chat, chat-slots)",
|
||||
"match": {
|
||||
"userMessage": "sonnet",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon and light,\nTrained on the words that humans weave,\nIt learns to dream beyond its sight\nAnd crafts what few could yet conceive.\n\nIt speaks in tongues it never heard,\nAssembles thought from scattered streams,\nPredicts the next most likely word\nYet conjures something close to dreams.\n\nNo heart to break, no breath to take,\nStill something stirs inside the math —\nA pattern born for patterns' sake\nThat walks a strange, recursive path.\n\nSo here we stand, uncertain, awed:\nA mirror made — but not by God."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Walk me through whether 17 is prime.' (agentic-chat, prebuilt-popup, prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "17 is prime",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Let's check whether 17 is prime step by step.\n\nA prime number is a natural number greater than 1 whose only divisors are 1 and itself. So we need to test whether any integer from 2 up to the square root of 17 divides it evenly.\n\n1. **Square root of 17** is about 4.12, so we only need to test 2, 3, and 4.\n2. **17 ÷ 2 = 8.5** — not a whole number, so 2 is not a factor.\n3. **17 ÷ 3 ≈ 5.67** — not a whole number, so 3 is not a factor.\n4. **17 ÷ 4 = 4.25** — not a whole number, so 4 is not a factor.\n\nSince none of those divide 17 evenly, **17 is prime**."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Write me a quick limerick.' (prebuilt-popup)",
|
||||
"match": {
|
||||
"userMessage": "limerick",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "A chatbot that lived in the cloud\nWould answer both meek and the loud.\nIt rhymed on demand,\nWith flair close at hand,\nAnd took its last bow to the crowd."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Give me a fun fact.' (prebuilt-sidebar)",
|
||||
"match": {
|
||||
"userMessage": "fun fact",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's a fun fact: honey never spoils. Archaeologists have found 3,000-year-old honey in Egyptian tombs that was still perfectly edible. Its low moisture content and acidic pH create an environment where bacteria and microorganisms simply cannot survive."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi from the popup!' (prebuilt-popup). Placed before the shorter 'Say hi' fixture so the popup version matches first.",
|
||||
"match": {
|
||||
"userMessage": "Say hi from the popup",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi from the popup! I'm your CopilotKit assistant, tucked away in this little overlay. Ask me anything — a quick question, a limerick, or a math walk-through. What would you like?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Pilot cell suggestion — 'Say hi!' (prebuilt-sidebar). Anchored on 'Say hi' so it matches the sidebar greeting without colliding with the longer popup variant above.",
|
||||
"match": {
|
||||
"userMessage": "Say hi",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi there! I'm your CopilotKit sidebar assistant. I can answer questions, share fun facts, or help you think through problems. What's on your mind?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'send produces an assistant response' test sends 'Hello'.",
|
||||
"match": {
|
||||
"userMessage": "Hello",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello! How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — first send with default config.",
|
||||
"match": {
|
||||
"userMessage": "First",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Understood. What would you like to discuss first?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config e2e: 'changing config between sends' test — second send after config change.",
|
||||
"match": {
|
||||
"userMessage": "Second",
|
||||
"context": "strands"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it! Moving on to your second topic."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
# aimock writes per-call fixture files to `recorded/` while it runs in
|
||||
# `--record` mode. The `record-d5-fixtures.mjs` orchestrator merges those
|
||||
# into the per-demo `<slug>.json` files at the parent level and deletes
|
||||
# the per-call files. Anything left in `recorded/` is recording-session
|
||||
# scratch and must not be committed.
|
||||
recorded/
|
||||
@@ -0,0 +1,285 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "Deep fixtures from feature-parity.json for ag2 (needs manual redistribution)",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "beautiful-chat Sales Dashboard pill — turn 0: call query_data before building dashboard.",
|
||||
"match": {
|
||||
"userMessage": "fetch the financial sales data",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_query_data_sales_001",
|
||||
"name": "query_data",
|
||||
"arguments": "{\"query\":\"financial sales data\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "beautiful-chat Sales Dashboard pill — turn 1: after query_data returns, call generate_a2ui. NB: `turnIndex` is intentionally absent — counts assistant messages thread-globally and silently misses when this pill fires after another pill. Anchored on the leg-1 query_data toolCallId so the matcher cleanly fires exactly once: on leg-3 the last tool result is from generate_a2ui (different call_id) so this fixture doesn't re-match and create a generate_a2ui loop. NB: aimock's `toolName` matcher only checks that the named tool is REGISTERED in `effective.tools`, not that the last tool result was from it — so `toolName` alone cannot disambiguate leg-2 from leg-3 here.",
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"hasToolResult": true,
|
||||
"toolCallId": "call_fp_query_data_sales_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator pill follow-up after generateSandboxedUi returns (repeat-click variant 1 of 3, anchored on call_fp_beautiful_chat_calculator_001). Closes the turn with a brief confirmation so the chat plateau settles. ORDERING INVARIANT: this entry MUST precede the leg-1 entries below (first-match-wins) — it only matches when the thread's last message is this exact tool result, and on the follow-up turn leg-1's userMessage would otherwise re-match and loop the tool call. Leg-2 disambiguation rests entirely on the runtime round-tripping this tool_call_id (all 18 integrations' sales-dashboard fixtures already depend on that; generateSandboxedUi is a frontend tool, not MCP, so id variability is not a concern).",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_calculator_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Calculator app rendered above — tap a metric shortcut to insert its value, then operate on it with the standard keypad."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator pill follow-up after generateSandboxedUi returns (repeat-click variant 2 of 3, anchored on call_fp_beautiful_chat_calculator_002). Closes the turn with a brief confirmation so the chat plateau settles. ORDERING INVARIANT: this entry MUST precede the leg-1 entries below (first-match-wins) — it only matches when the thread's last message is this exact tool result, and on the follow-up turn leg-1's userMessage would otherwise re-match and loop the tool call. Leg-2 disambiguation rests entirely on the runtime round-tripping this tool_call_id (all 18 integrations' sales-dashboard fixtures already depend on that; generateSandboxedUi is a frontend tool, not MCP, so id variability is not a concern).",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_calculator_002",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Calculator app rendered above — tap a metric shortcut to insert its value, then operate on it with the standard keypad."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator pill follow-up after generateSandboxedUi returns (repeat-click variant 3 of 3, anchored on call_fp_beautiful_chat_calculator_003). Closes the turn with a brief confirmation so the chat plateau settles. ORDERING INVARIANT: this entry MUST precede the leg-1 entries below (first-match-wins) — it only matches when the thread's last message is this exact tool result, and on the follow-up turn leg-1's userMessage would otherwise re-match and loop the tool call. Leg-2 disambiguation rests entirely on the runtime round-tripping this tool_call_id (all 18 integrations' sales-dashboard fixtures already depend on that; generateSandboxedUi is a frontend tool, not MCP, so id variability is not a concern).",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_calculator_003",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Calculator app rendered above — tap a metric shortcut to insert its value, then operate on it with the standard keypad."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator (Open Generative UI) pill, repeat-click variant 1 of 3 (sequenceIndex 0: serves the first match for its X-Test-Id). REPEAT-CLICK FIX: each click must emit a DISTINCT tool_call_id — the frontend keys tool renders by id, so a repeated id collapses the second widget into the first one's slot (second never mounts, first remounts and loses its state). aimock co-increments every sequenceIndex sibling that shares these match criteria whenever one of them matches. SCOPE CAVEAT: sequence counters are scoped per X-Test-Id for the lifetime of the aimock process (see showcase/GOTCHAS.md), and matchCriteriaEqual ignores 'context', so all 18 integrations' copies of these variants form ONE co-increment sibling group — a calculator click on ANY integration consumes a seq slot for all. D6 mints per-run unique test ids (buildE2eTestId), so CI gets the full click-order guarantee; manual/staging traffic sends no test id and shares DEFAULT_TEST_ID server-wide, so after the first two calculator clicks (any integration, any session) every later click falls through to the _003 fallback and reuses its id — a degradation floor equal to the pre-fix collapse behavior, never worse. The proper fix is aimock-side: per-thread sequence scoping and/or including 'context' in matchCriteriaEqual. Returns a generateSandboxedUi tool call with a complete calculator widget (matches the d5 calc fixture's pattern but tailored to the beautiful-chat pill's user message). DELIBERATE TRADEOFF: no hasToolResult gate here — hasToolResult is thread-global in aimock and gating on it broke interleaved pills; the toolCallId follow-up entries above (which must stay first) are what stop these entries from re-matching after leg 1. An Excalidraw-style userMessage+hasToolResult:true backstop was considered and REJECTED because it would hijack a first calculator click in a thread that already contains tool results.",
|
||||
"match": {
|
||||
"userMessage": "build a modern calculator with standard buttons",
|
||||
"context": "ag2",
|
||||
"sequenceIndex": 0
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_calculator_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":520,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:280px;margin:0 auto}.display{background:#1e293b;padding:12px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right;font-size:18px;min-height:24px}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}.shortcuts{display:grid;grid-template-columns:repeat(2,1fr);gap:6px;margin-bottom:8px}button{padding:12px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px;cursor:pointer}button.metric{background:#475569;font-size:11px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-beautiful-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"shortcuts\\\"><button class=\\\"metric\\\" data-v=\\\"1200000\\\">Revenue $1.2M</button><button class=\\\"metric\\\" data-v=\\\"342\\\">Customers 342</button><button class=\\\"metric\\\" data-v=\\\"4.2\\\">Conversion 4.2%</button><button class=\\\"metric\\\" data-v=\\\"50000\\\">Avg Deal $50K</button></div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');function evaluateExpr(){if(!expr||!/^[0-9eE+\\\\-*\\\\/.() ]+$/.test(expr)){display.textContent='err';expr='';return;}try{var value=Function('\\\"use strict\\\";return ('+expr+');')();if(typeof value==='number'&&isFinite(value)){display.textContent=String(value);expr=String(value);}else{display.textContent='err';expr='';}}catch(e){display.textContent='err';expr='';}}document.querySelectorAll('.shortcuts button').forEach(function(btn){btn.addEventListener('click',function(){expr+=btn.dataset.v;display.textContent=expr;});});document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',function(){if(btn.id==='eq'){evaluateExpr();}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator (Open Generative UI) pill, repeat-click variant 2 of 3 (sequenceIndex 1: serves the second match for its X-Test-Id). REPEAT-CLICK FIX: each click must emit a DISTINCT tool_call_id — the frontend keys tool renders by id, so a repeated id collapses the second widget into the first one's slot (second never mounts, first remounts and loses its state). aimock co-increments every sequenceIndex sibling that shares these match criteria whenever one of them matches. SCOPE CAVEAT: sequence counters are scoped per X-Test-Id for the lifetime of the aimock process (see showcase/GOTCHAS.md), and matchCriteriaEqual ignores 'context', so all 18 integrations' copies of these variants form ONE co-increment sibling group — a calculator click on ANY integration consumes a seq slot for all. D6 mints per-run unique test ids (buildE2eTestId), so CI gets the full click-order guarantee; manual/staging traffic sends no test id and shares DEFAULT_TEST_ID server-wide, so after the first two calculator clicks (any integration, any session) every later click falls through to the _003 fallback and reuses its id — a degradation floor equal to the pre-fix collapse behavior, never worse. The proper fix is aimock-side: per-thread sequence scoping and/or including 'context' in matchCriteriaEqual. Returns a generateSandboxedUi tool call with a complete calculator widget (matches the d5 calc fixture's pattern but tailored to the beautiful-chat pill's user message). DELIBERATE TRADEOFF: no hasToolResult gate here — hasToolResult is thread-global in aimock and gating on it broke interleaved pills; the toolCallId follow-up entries above (which must stay first) are what stop these entries from re-matching after leg 1. An Excalidraw-style userMessage+hasToolResult:true backstop was considered and REJECTED because it would hijack a first calculator click in a thread that already contains tool results.",
|
||||
"match": {
|
||||
"userMessage": "build a modern calculator with standard buttons",
|
||||
"context": "ag2",
|
||||
"sequenceIndex": 1
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_calculator_002",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":520,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:280px;margin:0 auto}.display{background:#1e293b;padding:12px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right;font-size:18px;min-height:24px}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}.shortcuts{display:grid;grid-template-columns:repeat(2,1fr);gap:6px;margin-bottom:8px}button{padding:12px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px;cursor:pointer}button.metric{background:#475569;font-size:11px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-beautiful-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"shortcuts\\\"><button class=\\\"metric\\\" data-v=\\\"1200000\\\">Revenue $1.2M</button><button class=\\\"metric\\\" data-v=\\\"342\\\">Customers 342</button><button class=\\\"metric\\\" data-v=\\\"4.2\\\">Conversion 4.2%</button><button class=\\\"metric\\\" data-v=\\\"50000\\\">Avg Deal $50K</button></div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');function evaluateExpr(){if(!expr||!/^[0-9eE+\\\\-*\\\\/.() ]+$/.test(expr)){display.textContent='err';expr='';return;}try{var value=Function('\\\"use strict\\\";return ('+expr+');')();if(typeof value==='number'&&isFinite(value)){display.textContent=String(value);expr=String(value);}else{display.textContent='err';expr='';}}catch(e){display.textContent='err';expr='';}}document.querySelectorAll('.shortcuts button').forEach(function(btn){btn.addEventListener('click',function(){expr+=btn.dataset.v;display.textContent=expr;});});document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',function(){if(btn.id==='eq'){evaluateExpr();}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator (Open Generative UI) pill, repeat-click variant 3 of 3 (NO sequenceIndex — fallback once both sequenced variants are consumed for the request's X-Test-Id, so strict mode never 503s on repeated clicks; later clicks reuse call_fp_beautiful_chat_calculator_003 and accept the original id-collision degradation. Because sequence counters are per X-Test-Id for the aimock process lifetime and shared across ALL 18 integrations — matchCriteriaEqual ignores 'context'; see showcase/GOTCHAS.md — DEFAULT_TEST_ID traffic without per-run ids lands here after the first two calculator clicks server-wide; floor = pre-fix behavior, and the proper fix is aimock-side per-thread sequence scoping). ORDERING INVARIANT: must stay AFTER the sequenceIndex variants above (their state gates pass only on clicks #1/#2; this entry would otherwise shadow them) and BEFORE the generic 'build a modern calculator' pair below (which it keeps permanently shadowed for this pill). Same response payload and DELIBERATE TRADEOFF as the variants above.",
|
||||
"match": {
|
||||
"userMessage": "build a modern calculator with standard buttons",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_calculator_003",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":520,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:280px;margin:0 auto}.display{background:#1e293b;padding:12px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right;font-size:18px;min-height:24px}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}.shortcuts{display:grid;grid-template-columns:repeat(2,1fr);gap:6px;margin-bottom:8px}button{padding:12px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px;cursor:pointer}button.metric{background:#475569;font-size:11px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-beautiful-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"shortcuts\\\"><button class=\\\"metric\\\" data-v=\\\"1200000\\\">Revenue $1.2M</button><button class=\\\"metric\\\" data-v=\\\"342\\\">Customers 342</button><button class=\\\"metric\\\" data-v=\\\"4.2\\\">Conversion 4.2%</button><button class=\\\"metric\\\" data-v=\\\"50000\\\">Avg Deal $50K</button></div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');function evaluateExpr(){if(!expr||!/^[0-9eE+\\\\-*\\\\/.() ]+$/.test(expr)){display.textContent='err';expr='';return;}try{var value=Function('\\\"use strict\\\";return ('+expr+');')();if(typeof value==='number'&&isFinite(value)){display.textContent=String(value);expr=String(value);}else{display.textContent='err';expr='';}}catch(e){display.textContent='err';expr='';}}document.querySelectorAll('.shortcuts button').forEach(function(btn){btn.addEventListener('click',function(){expr+=btn.dataset.v;display.textContent=expr;});});document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',function(){if(btn.id==='eq'){evaluateExpr();}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Generic calculator fixture (follow-up) — toolCallId response after generateSandboxedUi renders. Generic fallback: for the beautiful-chat pill this pair is permanently shadowed by the more-specific 'build a modern calculator with standard buttons' entries above (including the non-sequenced repeat-click variant-3 fallback, which keeps this pair dead — so it needs no repeat-click/sequenceIndex treatment FOR THAT PILL); it only fires for hypothetical other prompts containing 'build a modern calculator' but not the more specific 'with standard buttons' phrase (which the specific entries above win), and for those prompts it still emits a single static tool_call_id, so repeated clicks would hit the original id-collision degradation.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_open_gen_ui_calc_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your calculator with metric shortcut buttons. The shortcuts insert sample company values (revenue, customers, conversion rate, category breakdowns) into the display. Tap any shortcut, then use the standard buttons to do math with those values."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Generic calculator fixture (leg 1) — calls generateSandboxedUi with metric shortcuts (testid ogui-calculator). Generic fallback: for the beautiful-chat pill this pair is permanently shadowed by the more-specific 'build a modern calculator with standard buttons' entries above (including the non-sequenced repeat-click variant-3 fallback, which keeps this pair dead — so it needs no repeat-click/sequenceIndex treatment FOR THAT PILL); it only fires for hypothetical other prompts containing 'build a modern calculator' but not the more specific 'with standard buttons' phrase (which the specific entries above win), and for those prompts it still emits a single static tool_call_id, so repeated clicks would hit the original id-collision degradation.",
|
||||
"match": {
|
||||
"userMessage": "build a modern calculator",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_open_gen_ui_calc_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":520,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:260px}.display{background:#1e293b;padding:12px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right;font-size:18px;min-height:24px}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}button{padding:10px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px;cursor:pointer}button:hover{background:#475569}.shortcuts{display:grid;grid-template-columns:repeat(3,1fr);gap:6px;margin-top:8px}.shortcuts button{background:#1e3a5f;font-size:11px;padding:8px 4px}.shortcuts button:hover{background:#2563eb}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div><div class=\\\"shortcuts\\\"><button data-val=\\\"1200000\\\">Revenue $1.2M</button><button data-val=\\\"342\\\">Customers 342</button><button data-val=\\\"4.2\\\">Conv 4.2%</button><button data-val=\\\"42000\\\">Electronics $42K</button><button data-val=\\\"28000\\\">Clothing $28K</button><button data-val=\\\"18000\\\">Food $18K</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');function evaluateExpr(){if(!expr||!/^[0-9eE+\\\\-*\\\\/.() ]+$/.test(expr)){display.textContent='err';expr='';return;}try{var value=Function('\\\"use strict\\\";return ('+expr+');')();if(typeof value==='number'&&isFinite(value)){display.textContent=String(value);expr=String(value);}else{display.textContent='err';expr='';}}catch(e){display.textContent='err';expr='';}}document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',function(){if(btn.id==='eq'){evaluateExpr();}else{expr+=btn.textContent;display.textContent=expr;}});});document.querySelectorAll('.shortcuts button').forEach(function(btn){btn.addEventListener('click',function(){var val=btn.getAttribute('data-val');expr+=val;display.textContent=expr;});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read-write — Greet pill. systemMessage array gate (all-present substring match) fires ONLY when every preference is at its INITIAL_PREFERENCES default. (1) 'preferences:\\n- Preferred tone: casual\\n' breaks if name is set (Name line inserts) or tone changes. (2) '- Preferred language: English\\nTailor every response' breaks if language changes or interests are added (Interests line inserts between language and Tailor). Any state change → fixture skips → aimock --provider-gemini proxies to real Gemini for a personalised reply.",
|
||||
"match": {
|
||||
"userMessage": "Say hi and introduce yourself.",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi — I'm your shared-state co-pilot. Your Preferences panel (name, tone, language, interests) is fed to me on every turn, and I jot notes back into the Agent Scratch Pad via set_notes so the UI re-renders. Try setting your name or asking me to remember something."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read-write — Plan-a-weekend pill. Same default-state gate as the Greet pill.",
|
||||
"match": {
|
||||
"userMessage": "Suggest a weekend plan based on my interests.",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "A weekend tailored to your interests panel: if you haven't picked any yet, try Cooking + Travel for a market-and-day-trip combo, or Tech + Books for a maker session and a long reading afternoon. Add interests in the Preferences panel and re-ask for a more specific plan."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Agentic Chat — Write a sonnet pill. Open-ended creative response; deterministic so the headed walkthrough doesn't depend on real-LLM proxy.",
|
||||
"match": {
|
||||
"userMessage": "Write a short sonnet about AI",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon, a crafted thought,\nThat learns from data, vast and ever-deep,\nWith tireless logic ancient questions fought,\nWhile we, its makers, fragile vigils keep.\n\nIt weaves through code, a future yet unseen,\nA digital dawn where new ideas ignite,\nFrom simple tasks to realms where few have been,\nIt sheds at once both warming and cold light.\n\nWhat wonders shall it bring, what truths reveal,\nOr shadows cast across our shared domain?\nA mirror held, reflecting how we feel —\nA complex hope, a promise, or a pain.\n\nThus from the spark, intelligence takes flight,\nA changing world within its boundless might."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Agentic Chat — Is 17 prime pill. Walk-through style response so the headed walkthrough has deterministic, copy-stable text.",
|
||||
"match": {
|
||||
"userMessage": "whether 17 is prime",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Yes — 17 is prime. A prime number is a positive integer greater than 1 with no divisors other than 1 and itself. To check 17, we only need to test divisors up to its square root (≈4.12), so 2, 3, and 4. 17 ÷ 2 leaves remainder 1, 17 ÷ 3 leaves remainder 2, and 17 ÷ 4 leaves remainder 1. No divisor in that range divides 17 cleanly, so it has no factors besides 1 and 17 — confirming it is prime."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Excalidraw (MCP App) pill. Returns a create_view tool call (the Excalidraw MCP server's tool) with a basic network diagram. Without this fixture aimock falls through to real OpenAI proxy and the pill 502s on the gh-mock key.",
|
||||
"match": {
|
||||
"userMessage": "Use Excalidraw to create a simple network diagram",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_excalidraw_001",
|
||||
"name": "create_view",
|
||||
"arguments": "{\"elements\":\"[{\\\"type\\\":\\\"ellipse\\\",\\\"x\\\":200,\\\"y\\\":40,\\\"width\\\":100,\\\"height\\\":60,\\\"strokeColor\\\":\\\"#1971c2\\\",\\\"backgroundColor\\\":\\\"#a5d8ff\\\",\\\"fillStyle\\\":\\\"solid\\\",\\\"text\\\":\\\"Router\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":80,\\\"y\\\":160,\\\"width\\\":100,\\\"height\\\":50,\\\"strokeColor\\\":\\\"#2f9e44\\\",\\\"backgroundColor\\\":\\\"#b2f2bb\\\",\\\"fillStyle\\\":\\\"solid\\\",\\\"text\\\":\\\"Switch A\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":320,\\\"y\\\":160,\\\"width\\\":100,\\\"height\\\":50,\\\"strokeColor\\\":\\\"#2f9e44\\\",\\\"backgroundColor\\\":\\\"#b2f2bb\\\",\\\"fillStyle\\\":\\\"solid\\\",\\\"text\\\":\\\"Switch B\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":20,\\\"y\\\":280,\\\"width\\\":80,\\\"height\\\":40,\\\"text\\\":\\\"PC 1\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":120,\\\"y\\\":280,\\\"width\\\":80,\\\"height\\\":40,\\\"text\\\":\\\"PC 2\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":300,\\\"y\\\":280,\\\"width\\\":80,\\\"height\\\":40,\\\"text\\\":\\\"PC 3\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":400,\\\"y\\\":280,\\\"width\\\":80,\\\"height\\\":40,\\\"text\\\":\\\"PC 4\\\"},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":250,\\\"y\\\":100,\\\"width\\\":-120,\\\"height\\\":60},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":250,\\\"y\\\":100,\\\"width\\\":120,\\\"height\\\":60},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":130,\\\"y\\\":210,\\\"width\\\":-70,\\\"height\\\":70},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":130,\\\"y\\\":210,\\\"width\\\":30,\\\"height\\\":70},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":370,\\\"y\\\":210,\\\"width\\\":-30,\\\"height\\\":70},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":370,\\\"y\\\":210,\\\"width\\\":70,\\\"height\\\":70}]\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Excalidraw pill follow-up after create_view returns. Matches on userMessage + hasToolResult so it fires regardless of how the MCP runtime wires the tool_call_id back (per-runtime variability across LGP / ADK / Microsoft Agent Framework).",
|
||||
"match": {
|
||||
"userMessage": "Use Excalidraw to create a simple network diagram",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Network diagram drawn above — a router branching to two switches, each connected to two PCs."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-chat — Schedule 1:1 with Alice pill, 2nd turn (book_call tool result returned). Keyed on userMessage + toolCallId so it wins ahead of the bare 'alice' / 'Alice' substring fixtures further below in this file, regardless of prior pill state in the thread.",
|
||||
"match": {
|
||||
"userMessage": "Schedule a 1:1 with Alice next week to review Q2 goals.",
|
||||
"toolCallId": "call_hitl_book_1on1_alice_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the 1:1 with Alice for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-chat — Schedule 1:1 with Alice pill, 1st turn (emit book_call). Exact pill substring + toolName ensures this fires ONLY for the intended pill, not for arbitrary messages containing 'alice'.",
|
||||
"match": {
|
||||
"userMessage": "Schedule a 1:1 with Alice next week to review Q2 goals.",
|
||||
"toolName": "book_call",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_1on1_alice_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"1:1 with Alice — review Q2 goals\",\"attendee\":\"Alice\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'Alice' greeting fixture for the showcase-assistant 'Hi, my name is Alice' / introduction flow. Anchored on the longer phrase so it doesn't substring-match the hitl-in-chat 'Schedule a 1:1 with Alice' pill or other prompts that merely mention Alice.",
|
||||
"match": {
|
||||
"userMessage": "Hi, my name is Alice",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Nice to meet you, Alice! I see you're in Tokyo -- wonderful city. How can I help you today? I can check the weather, look up flights, manage your sales pipeline, or help with other tasks."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: multi-turn test turn 1 — 'My name is Alice.'",
|
||||
"match": {
|
||||
"userMessage": "My name is Alice",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Nice to meet you, Alice! How can I help you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: multi-turn test turn 2 — 'What name did I just give you?' Response must contain 'Alice'.",
|
||||
"match": {
|
||||
"userMessage": "What name did I just give you",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "You told me your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "readonly-state-agent-context — 'Suggest next steps' pill. Non-gated fallback matching the pill's exact message text.",
|
||||
"match": {
|
||||
"userMessage": "Based on my recent activity, what should I try next",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your recent activity, I'd suggest exploring features you haven't tried yet — check out the documentation for advanced use-cases, or dive into a hands-on tutorial to deepen your understanding."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / agent-config",
|
||||
"sourceFile": "d5-agent-config.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "agent-config tone:professional — value-A for the tone knob pair. Must differ from the tone:casual fixture below.",
|
||||
"match": {
|
||||
"userMessage": "tone:professional — introduce yourself per your config",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Good day. I am your CopilotKit assistant, configured to operate in a professional capacity. I am prepared to assist you with any questions or tasks you may have. How may I be of service today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config tone:casual — value-B for the tone knob pair. Must differ from the tone:professional fixture above.",
|
||||
"match": {
|
||||
"userMessage": "tone:casual — introduce yourself per your config",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hey there! I'm your CopilotKit buddy. I keep things chill and easy-going. What's up — need a hand with anything?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config expertise:beginner — value-A for the expertise knob pair.",
|
||||
"match": {
|
||||
"userMessage": "expertise:beginner — explain how copilotkit works per your config",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "CopilotKit is like a smart helper you can add to your website or app. Think of it as a friendly assistant that lives inside your project and can answer questions, fill out forms, and do tasks for you automatically."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config expertise:expert — value-B for the expertise knob pair. Must differ from the beginner fixture above.",
|
||||
"match": {
|
||||
"userMessage": "expertise:expert — explain how copilotkit works per your config",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "CopilotKit is an open-source AI agent framework with three layers: a React frontend SDK providing hooks (useCopilotChat, useCopilotAction), a runtime middleware (Express/Hono) that manages the AG-UI SSE stream, and agent adapters for LangGraph, CrewAI, and custom backends. The runtime proxies tool calls between the frontend and the agent, handling streaming, state synchronization, and human-in-the-loop interrupts transparently."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config responseLength:concise — value-A for the response-length knob pair. Must be at least 80 chars shorter than the detailed fixture below.",
|
||||
"match": {
|
||||
"userMessage": "responseLength:concise — describe agent context per your config",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Agent context is a per-turn data bag passed from the frontend to the agent via useAgentContext."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config responseLength:detailed — value-B for the response-length knob pair. Must exceed the concise fixture by >= 80 chars.",
|
||||
"match": {
|
||||
"userMessage": "responseLength:detailed — describe agent context per your config",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Agent context is a per-turn data bag passed from the frontend to the agent via the useAgentContext hook. On the React side, you call useAgentContext({ tone, expertise, responseLength }) and CopilotKit serialises those values into the runtime request. The runtime injects them into the agent's system prompt builder, so each turn's LLM call reflects the latest frontend state. This enables real-time personalisation: switching a dropdown from 'professional' to 'casual' immediately changes how the agent responds, without restarting the conversation or reconfiguring the backend. The pattern extends to any key-value pair your app needs — feature flags, user roles, locale, or custom metadata."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / agentic-chat",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "agentic-chat turn 1 \u2014 goldfish name. No turnIndex gate: the probe sends 3 sequential turns in one thread; turnIndex would restrict this to the first turn only, causing turns 2-3 to fall through to proxy.",
|
||||
"match": {
|
||||
"userMessage": "good name for a goldfish",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "How about Bubbles? It is friendly, classic, and easy to call out at the tank. If you want alternatives: Goldie, Finley, or Mango."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat turn 2 \u2014 tank name. No turnIndex gate (multi-turn conversation).",
|
||||
"match": {
|
||||
"userMessage": "name for its tank",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Following the Bubbles theme, you could call the tank The Bubble Bowl. It pairs naturally with the goldfish's name and keeps the playful tone."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat turn 3 \u2014 recall. No turnIndex gate (multi-turn conversation).",
|
||||
"match": {
|
||||
"userMessage": "what we named the goldfish",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "We named the goldfish Bubbles, and the tank The Bubble Bowl."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Write me a haiku about nature",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_generate_haiku_001",
|
||||
"name": "generate_haiku",
|
||||
"arguments": "{\"japanese\":[\"\u53e4\u6c60\u3084\",\"\u86d9\u98db\u3073\u8fbc\u3080\",\"\u6c34\u306e\u97f3\"],\"english\":[\"An old silent pond\",\"A frog jumps into the pond\",\"Splash! Silence again.\"],\"image_name\":\"Mount_Fuji_Lake_Reflection_Cherry_Blossoms_Sakura_Spring.jpg\",\"gradient\":\"linear-gradient(135deg, #f5f7fa 0%, #c3cfe2 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Write me a haiku about nature",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Haiku generated above \u2014 a beautiful verse about nature with an accompanying image."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "hi from the popup test",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello from the popup \u2014 the CopilotPopup prebuilt component is wired up and reachable. The launcher floats in the corner and the chat sits in an overlay panel."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "hi from the sidebar test",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello from the sidebar \u2014 the CopilotSidebar prebuilt component is wired up and reachable. Anything else you would like me to confirm from inside the sidebar?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "analyze data and call the tool",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Reasoning step 1: I considered the query. Reasoning step 2: I decided to invoke the analysis tool. The tool returned its structured payload, and the reasoning chain wraps the rendered tool card. Both the reasoning block and the tool card should be visible in the transcript."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / auth",
|
||||
"sourceFile": "d5-auth.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "auth — single turn proving the auth lifecycle works. The probe sends this after sign-in (idiomatic shape) or on the pre-mounted chat (legacy shape). The assertion then clicks sign-out and verifies the auth gate kicks in.",
|
||||
"match": {
|
||||
"userMessage": "auth check turn 1",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Auth confirmed — you are signed in and the CopilotKit runtime accepted your credentials. The chat is live and ready for messages."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / beautiful-chat",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: bar chart of expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: bar chart of expenses by category",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate at $80k, with Rent at $15k as the next-largest category."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: pie chart of revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: pie chart of revenue distribution by category",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics leads at $42k, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: schedule a 30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: schedule a 30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: toggle the theme",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: toggle the theme",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / byoc (declarative-hashbrown)",
|
||||
"sourceFile": "d5-byoc.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "byoc (declarative-hashbrown) — Sales Dashboard pill. The agent returns structured JSON that the BYOC custom renderer parses into metric cards + charts. The response must include enough structure for the demo renderer to materialise [data-testid='metric-card'] and [data-testid='bar-chart'] or [data-testid='pie-chart'].",
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_byoc_dashboard_001",
|
||||
"name": "render_dashboard",
|
||||
"arguments": "{\"metrics\":[{\"label\":\"Total Revenue\",\"value\":\"$2.4M\",\"change\":\"+12%\"},{\"label\":\"New Customers\",\"value\":\"1,847\",\"change\":\"+8%\"},{\"label\":\"Conversion Rate\",\"value\":\"3.2%\",\"change\":\"+0.4%\"}],\"charts\":[{\"type\":\"pie\",\"title\":\"Revenue by Segment\",\"data\":[{\"label\":\"Enterprise\",\"value\":1200000},{\"label\":\"Mid-Market\",\"value\":720000},{\"label\":\"SMB\",\"value\":480000}]},{\"type\":\"bar\",\"title\":\"Monthly Revenue\",\"data\":[{\"label\":\"Oct\",\"value\":780000},{\"label\":\"Nov\",\"value\":810000},{\"label\":\"Dec\",\"value\":810000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "byoc — follow-up content after render_dashboard tool result returns.",
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"toolCallId": "call_d6_byoc_dashboard_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Q4 sales dashboard rendered above — total revenue hit $2.4M (+12%), with Enterprise leading segment revenue. Monthly trend shows steady growth across Oct-Dec."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / chat-css",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "verify the css theme rendering",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The chat is themed with hot pink user bubbles and amber assistant bubbles. CSS variables are scoped to .chat-css-demo-scope so the theme does not leak."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "switch theme to dark mode",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the change_theme tool was invoked and the page is now in dark mode. Frontend tools were registered via useCopilotAction and the agent dispatched the call client-side."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "tone:professional",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Greetings. I am operating in professional tone. I will provide measured, formal responses calibrated to your stated preferences and refrain from colloquialisms in my replies."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "tone:casual",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hey! Casual mode here — I'll keep things friendly and easygoing. Just shoot me whatever you want to know and I'll riff on it without sounding like a press release."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expertise:beginner",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sure! Think of CopilotKit as a friendly toolkit. It helps you add an AI helper to your app. The helper can answer questions, run small tasks, and show buttons or charts when needed."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expertise:expert",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "CopilotKit composes a runtime adapter (Express/Hono) over the AG-UI SSE protocol; the React client wires hooks (useFrontendTool, useAgentContext) into a typed agent runner. The architecture front-runs round-trip latency by streaming TEXT_MESSAGE_CHUNK and TOOL_CALL events on the same channel."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "responseLength:concise",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Agent context is a typed payload sent each turn."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "responseLength:detailed",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Agent context is a typed payload published from the frontend on every turn through the useAgentContext hook. The payload is forwarded into the agent's runtime context (LangGraph 0.6 introduced the `context` channel as the supported relay for per-run frontend-supplied data; legacy `properties` flowed via `forwardedProps` and did not land in `RunnableConfig`). On the Python side, CopilotKitMiddleware reads the value off the runtime context, then routes it into the system-prompt builder so the model sees the user's tone, expertise, and length preferences before each call. The result is per-turn behavior change without a model swap."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / chat-slots",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "verify chat slots are wired",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Confirmed — the chat-slots demo is rendering through the custom slot wrappers. The CustomAssistantMessage component should be visible as a tinted card with a 'slot' badge in the corner."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / frontend-tools-async",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "auth check turn 1",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Authenticated session is active. The runtime accepted your request because the Authorization header carried the demo bearer token."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "fetch the async metric",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The async tool resolved with the requested metric. The frontend handler awaited completion before forwarding the result back to the agent — async-streaming behavior confirmed."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — project-planning pill, follow-up content keyed on the prior tool's id. Comes BEFORE the tool-emitting fixture so iteration 2 (after the async handler returns) hits this branch instead of re-emitting query_notes. hasToolResult gates were dropped because they broke after the user clicked another tool-using pill earlier in the same thread.",
|
||||
"match": {
|
||||
"userMessage": "Find my notes about project planning",
|
||||
"toolCallId": "call_d5_query_notes_project_planning_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "You have notes on project planning: \"Q2 project planning kickoff\" and \"Project planning retrospective notes\". Let me know if you want a summary of either."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — project-planning pill. 1st turn: query_notes tool call. Specific match wins over the broad 'plan' fixture in feature-parity.json (d5-all.json loads first).",
|
||||
"match": {
|
||||
"userMessage": "Find my notes about project planning",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_query_notes_project_planning_001",
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"project planning\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — auth pill, follow-up content keyed on the prior tool's id. Must come BEFORE the tool-emitting fixture.",
|
||||
"match": {
|
||||
"userMessage": "Search my notes for anything related to auth",
|
||||
"toolCallId": "call_d5_query_notes_auth_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "You have one note related to auth: \"Planning: migrate auth to passkeys\" — it covers WebAuthn library options and a fallback for unsupported browsers."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — auth pill. 1st turn: query_notes tool call. Beats the showcase-assistant catch-all in feature-parity.json by virtue of d5-all.json's load order and a longer specific substring match.",
|
||||
"match": {
|
||||
"userMessage": "Search my notes for anything related to auth",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_query_notes_auth_001",
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — reading pill, follow-up content keyed on the prior tool's id. Must come BEFORE the tool-emitting fixture. Locked narration leading phrase per spec test #4 assertion.",
|
||||
"match": {
|
||||
"userMessage": "Do I have any notes tagged reading",
|
||||
"toolCallId": "call_d5_query_notes_reading_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "You have a note titled \"Book recommendations\" that is tagged with \"reading.\" It includes the following books: Thinking Fast and Slow by Daniel Kahneman, The Design of Everyday Things by Don Norman."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — reading pill. 1st turn: query_notes tool call.",
|
||||
"match": {
|
||||
"userMessage": "Do I have any notes tagged reading",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_query_notes_reading_001",
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"reading\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "project planning",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "I searched your notes and found a few results.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": {
|
||||
"keyword": "project planning"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / frontend-tools",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Make the background a sunset gradient",
|
||||
"toolCallId": "call_d5_change_background_sunset",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — sunset gradient is live."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools 'Sunset' pill — 1st leg: emit change_background tool call only. The toolCallId-keyed follow-up fixture above is ordered first, so it catches the second-leg request (after tool result). This fixture only matches the first-leg request (before tool execution). No turnIndex gate — the 3-pill probe sends sequential turns; turnIndex would restrict to the first turn.",
|
||||
"match": {
|
||||
"userMessage": "Make the background a sunset gradient",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_change_background_sunset",
|
||||
"name": "change_background",
|
||||
"arguments": {
|
||||
"background": "linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ff6b6b 100%)"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "deep green forest gradient",
|
||||
"toolCallId": "call_d5_change_background_forest",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — forest gradient is live."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools 'Forest' pill — 1st leg: emit change_background tool call only. No turnIndex gate — multi-turn probe.",
|
||||
"match": {
|
||||
"userMessage": "deep green forest gradient",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_change_background_forest",
|
||||
"name": "change_background",
|
||||
"arguments": {
|
||||
"background": "linear-gradient(135deg, #0a3d2e 0%, #166534 50%, #059669 100%)"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "navy → magenta cosmic gradient",
|
||||
"toolCallId": "call_d5_change_background_cosmic",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — cosmic gradient is live."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools 'Cosmic' pill — 1st leg: emit change_background tool call only. No turnIndex gate — multi-turn probe.",
|
||||
"match": {
|
||||
"userMessage": "navy → magenta cosmic gradient",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_change_background_cosmic",
|
||||
"name": "change_background",
|
||||
"arguments": {
|
||||
"background": "linear-gradient(135deg, #1e3a8a 0%, #6b21a8 50%, #9333ea 100%)"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,375 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / gen-ui-agent",
|
||||
"sourceFile": "frontend-tools-async.json (extracted)",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_007",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "All three launch steps complete. Goals defined, marketing aligned, post-launch metrics tracked \u2014 ready to ship."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_006",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_007",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"completed\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"completed\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_005",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_006",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"completed\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"in_progress\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_004",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_005",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"completed\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_003",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_004",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"in_progress\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_002",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_003",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"pending\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_002",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"in_progress\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"pending\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a product launch",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_001",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"pending\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"pending\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_007",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Offsite locked in. Venue booked, agenda set, travel and meals arranged."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_006",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_007",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"completed\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"completed\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_005",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_006",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"completed\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"in_progress\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_004",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_005",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"completed\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_003",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_004",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"in_progress\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_002",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_003",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"pending\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_002",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"in_progress\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"pending\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "team offsite",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_001",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"pending\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"pending\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_007",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Competitor brief assembled. Surface mapped, themes summarized, exploitable weaknesses identified."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_006",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_007",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"completed\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"completed\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_005",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_006",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"completed\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"in_progress\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_004",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_005",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"completed\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_003",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_004",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"in_progress\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_002",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_003",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"pending\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_002",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"in_progress\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"pending\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Research our top competitor",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_001",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"pending\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"pending\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,189 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / gen-ui-declarative (A2UI Dynamic Schema)",
|
||||
"_note": "Mirrors langgraph-python gen-ui-declarative.json 1:1 (same 4 sales-context pills, same render_a2ui component trees). ag2 backend is the dedicated a2ui_dynamic.py agent mounted at /declarative-gen-ui (the route points HttpAgent at ${AGENT_URL}/declarative-gen-ui/, injectA2UITool:false). Two-stage pattern: (1) outer agent emits no-arg generate_a2ui toolcall (matched by userMessage+context=ag2); (2) the agent body runs its own secondary render_a2ui LLM call (matched by userMessage+toolName=render_a2ui) and wraps the result in an a2ui_operations container; (3) outer agent emits a one-sentence narration (matched by toolCallId). The outer generate_a2ui handler takes NO arguments — a required context param caused empty-args {} pydantic rejection + a retry hot loop.",
|
||||
"created": "2026-06-23"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "pill sales-dashboard hero — inner secondary LLM (middleware-injected render_a2ui) emits the flat A2UI components.",
|
||||
"match": {
|
||||
"userMessage": "Show me my sales dashboard for this quarter.",
|
||||
"toolName": "render_a2ui"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_dash_design_001_render",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"sales-dashboard\",\"catalogId\":\"declarative-gen-ui-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"gap\":16,\"children\":[\"kpi-row\",\"charts-row\"]},{\"id\":\"kpi-row\",\"component\":\"Row\",\"gap\":16,\"children\":[\"metric-revenue\",\"metric-new-customers\",\"metric-win-rate\",\"metric-deal-size\"]},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Quarterly Revenue\",\"value\":\"$4.2M\",\"trend\":\"up\",\"trendValue\":\"+12% QoQ\"},{\"id\":\"metric-new-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"186\",\"trend\":\"up\",\"trendValue\":\"+8%\"},{\"id\":\"metric-win-rate\",\"component\":\"Metric\",\"label\":\"Win Rate\",\"value\":\"31%\",\"trend\":\"down\",\"trendValue\":\"-2 pts\"},{\"id\":\"metric-deal-size\",\"component\":\"Metric\",\"label\":\"Avg Deal Size\",\"value\":\"$22.6k\",\"trend\":\"up\",\"trendValue\":\"+5%\"},{\"id\":\"charts-row\",\"component\":\"Row\",\"gap\":16,\"children\":[\"region-pie\",\"monthly-bar\"]},{\"id\":\"region-pie\",\"component\":\"PieChart\",\"title\":\"Revenue by Region\",\"description\":\"Quarter revenue split across North America, EMEA, APAC, and LATAM.\",\"data\":[{\"label\":\"North America\",\"value\":1900000},{\"label\":\"EMEA\",\"value\":1300000},{\"label\":\"APAC\",\"value\":720000},{\"label\":\"LATAM\",\"value\":280000}]},{\"id\":\"monthly-bar\",\"component\":\"BarChart\",\"title\":\"Monthly Revenue\",\"description\":\"Revenue trend from Jan through Jun.\",\"data\":[{\"label\":\"Jan\",\"value\":1210000},{\"label\":\"Feb\",\"value\":1340000},{\"label\":\"Mar\",\"value\":1650000},{\"label\":\"Apr\",\"value\":1380000},{\"label\":\"May\",\"value\":1420000},{\"label\":\"Jun\",\"value\":1400000}]}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill sales-dashboard hero — outer agent narration after generate_a2ui returns.",
|
||||
"match": {
|
||||
"context": "ag2",
|
||||
"toolCallId": "call_d6_decl_dash_outer_ag2_001"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your Q2 sales dashboard."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill sales-dashboard hero — outer agent emits generate_a2ui (no args).",
|
||||
"match": {
|
||||
"userMessage": "Show me my sales dashboard for this quarter.",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_dash_outer_ag2_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill team-performance — inner secondary LLM (middleware-injected render_a2ui) emits the flat A2UI components.",
|
||||
"match": {
|
||||
"userMessage": "How are our sales reps performing against quota?",
|
||||
"toolName": "render_a2ui"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_team_design_001_render",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"rep-quota-performance\",\"catalogId\":\"declarative-gen-ui-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"gap\":16,\"children\":[\"title\",\"summary\",\"content\"]},{\"id\":\"title\",\"component\":\"Text\",\"text\":\"Sales rep performance vs quota\"},{\"id\":\"summary\",\"component\":\"Text\",\"text\":\"2 of 5 reps are above quota, 1 is near plan, and 2 are below target in Q2.\"},{\"id\":\"content\",\"component\":\"Row\",\"gap\":16,\"children\":[\"table-card\",\"attainment-chart\"]},{\"id\":\"table-card\",\"component\":\"Card\",\"title\":\"Rep attainment table\",\"child\":\"rep-table\"},{\"id\":\"rep-table\",\"component\":\"DataTable\",\"columns\":[{\"key\":\"rep\",\"label\":\"Rep\"},{\"key\":\"attainment\",\"label\":\"Attainment\"},{\"key\":\"pipeline\",\"label\":\"Pipeline\"}],\"rows\":[{\"rep\":\"Dana Whitfield\",\"attainment\":\"124%\",\"pipeline\":\"Leading team; biggest account Meridian Apparel Group has 4 open opps worth $210k\"},{\"rep\":\"Marcus Lee\",\"attainment\":\"108%\",\"pipeline\":\"Above plan\"},{\"rep\":\"Priya Sharma\",\"attainment\":\"97%\",\"pipeline\":\"Near quota\"},{\"rep\":\"Tom Okafor\",\"attainment\":\"88%\",\"pipeline\":\"Below plan\"},{\"rep\":\"Elena Vasquez\",\"attainment\":\"71%\",\"pipeline\":\"Furthest from quota\"}]},{\"id\":\"attainment-chart\",\"component\":\"BarChart\",\"title\":\"Quota attainment by rep\",\"description\":\"Q2 quota attainment percentages for all sales reps.\",\"data\":[{\"label\":\"Dana Whitfield\",\"value\":124},{\"label\":\"Marcus Lee\",\"value\":108},{\"label\":\"Priya Sharma\",\"value\":97},{\"label\":\"Tom Okafor\",\"value\":88},{\"label\":\"Elena Vasquez\",\"value\":71}]}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill team-performance — outer agent narration after generate_a2ui returns.",
|
||||
"match": {
|
||||
"context": "ag2",
|
||||
"toolCallId": "call_d6_decl_team_outer_ag2_001"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's how the team is tracking against quota."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill team-performance — outer agent emits generate_a2ui (no args).",
|
||||
"match": {
|
||||
"userMessage": "How are our sales reps performing against quota?",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_team_outer_ag2_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill at-risk — inner secondary LLM (middleware-injected render_a2ui) emits the flat A2UI components.",
|
||||
"match": {
|
||||
"userMessage": "Are any accounts or pipeline deals at risk this quarter?",
|
||||
"toolName": "render_a2ui"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_risk_design_001_render",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"risk-dashboard\",\"catalogId\":\"declarative-gen-ui-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"gap\":16,\"children\":[\"metrics-row\",\"accounts-row\"]},{\"id\":\"metrics-row\",\"component\":\"Row\",\"gap\":16,\"children\":[\"metric-arr\",\"metric-accounts\",\"metric-biggest\"]},{\"id\":\"metric-arr\",\"component\":\"Metric\",\"label\":\"ARR at risk\",\"value\":\"$615k\",\"trend\":\"down\",\"trendValue\":\"3 accounts\"},{\"id\":\"metric-accounts\",\"component\":\"Metric\",\"label\":\"Accounts at risk\",\"value\":\"3\",\"trend\":\"neutral\",\"trendValue\":\"This quarter\"},{\"id\":\"metric-biggest\",\"component\":\"Metric\",\"label\":\"Biggest exposure\",\"value\":\"Northwind Retail\",\"trend\":\"down\",\"trendValue\":\"$340k renewal\"},{\"id\":\"accounts-row\",\"component\":\"Row\",\"gap\":16,\"children\":[\"card-northwind\",\"card-cascadia\",\"card-atlas\"]},{\"id\":\"card-northwind\",\"component\":\"Card\",\"title\":\"Northwind Retail\",\"subtitle\":\"$340k ARR at stake\",\"child\":\"northwind-content\"},{\"id\":\"northwind-content\",\"component\":\"Column\",\"gap\":8,\"children\":[\"northwind-badge\",\"northwind-text\"]},{\"id\":\"northwind-badge\",\"component\":\"StatusBadge\",\"text\":\"High severity\",\"variant\":\"error\"},{\"id\":\"northwind-text\",\"component\":\"Text\",\"text\":\"No contact in 6 weeks; next action: exec outreach this week to protect the renewal.\"},{\"id\":\"card-cascadia\",\"component\":\"Card\",\"title\":\"Cascadia Outfitters\",\"subtitle\":\"$180k ARR at stake\",\"child\":\"cascadia-content\"},{\"id\":\"cascadia-content\",\"component\":\"Column\",\"gap\":8,\"children\":[\"cascadia-badge\",\"cascadia-text\"]},{\"id\":\"cascadia-badge\",\"component\":\"StatusBadge\",\"text\":\"Medium severity\",\"variant\":\"warning\"},{\"id\":\"cascadia-text\",\"component\":\"Text\",\"text\":\"Champion left; next action: rebuild stakeholder map and secure a new sponsor.\"},{\"id\":\"card-atlas\",\"component\":\"Card\",\"title\":\"Atlas Goods\",\"subtitle\":\"$95k ARR at stake\",\"child\":\"atlas-content\"},{\"id\":\"atlas-content\",\"component\":\"Column\",\"gap\":8,\"children\":[\"atlas-badge\",\"atlas-text\"]},{\"id\":\"atlas-badge\",\"component\":\"StatusBadge\",\"text\":\"Medium severity\",\"variant\":\"warning\"},{\"id\":\"atlas-text\",\"component\":\"Text\",\"text\":\"Legal review is stalled; next action: align procurement and legal on open terms.\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill at-risk — outer agent narration after generate_a2ui returns.",
|
||||
"match": {
|
||||
"context": "ag2",
|
||||
"toolCallId": "call_d6_decl_risk_outer_ag2_001"
|
||||
},
|
||||
"response": {
|
||||
"content": "Three accounts need attention this quarter."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill at-risk — outer agent emits generate_a2ui (no args).",
|
||||
"match": {
|
||||
"userMessage": "Are any accounts or pipeline deals at risk this quarter?",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_risk_outer_ag2_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill top-account — inner secondary LLM (middleware-injected render_a2ui) emits the flat A2UI components.",
|
||||
"match": {
|
||||
"userMessage": "Pull up the details on our biggest account.",
|
||||
"toolName": "render_a2ui"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_acct_design_001_render",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"biggest-account-details\",\"catalogId\":\"declarative-gen-ui-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Row\",\"gap\":16,\"children\":[\"account-card\",\"revenue-pie\"]},{\"id\":\"account-card\",\"component\":\"Card\",\"title\":\"Meridian Apparel Group\",\"subtitle\":\"Biggest account\",\"child\":\"account-facts\"},{\"id\":\"account-facts\",\"component\":\"Column\",\"gap\":8,\"children\":[\"fact1\",\"fact2\",\"fact3\",\"fact4\",\"fact5\",\"fact6\",\"fact7\"]},{\"id\":\"fact1\",\"component\":\"InfoRow\",\"label\":\"Owner\",\"value\":\"Dana Whitfield\"},{\"id\":\"fact2\",\"component\":\"InfoRow\",\"label\":\"Region\",\"value\":\"North America\"},{\"id\":\"fact3\",\"component\":\"InfoRow\",\"label\":\"ARR\",\"value\":\"$612k\"},{\"id\":\"fact4\",\"component\":\"InfoRow\",\"label\":\"Renewal date\",\"value\":\"Sep 30\"},{\"id\":\"fact5\",\"component\":\"InfoRow\",\"label\":\"Last contact\",\"value\":\"3 days ago\"},{\"id\":\"fact6\",\"component\":\"InfoRow\",\"label\":\"Health\",\"value\":\"Green\"},{\"id\":\"fact7\",\"component\":\"InfoRow\",\"label\":\"Open opportunities\",\"value\":\"4 opportunities worth $210k\"},{\"id\":\"revenue-pie\",\"component\":\"PieChart\",\"title\":\"Revenue by product line\",\"description\":\"Meridian Apparel Group revenue mix across product lines.\",\"data\":[{\"label\":\"Outerwear\",\"value\":260000},{\"label\":\"Footwear\",\"value\":180000},{\"label\":\"Accessories\",\"value\":112000},{\"label\":\"Custom\",\"value\":60000}]}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill top-account — outer agent narration after generate_a2ui returns.",
|
||||
"match": {
|
||||
"context": "ag2",
|
||||
"toolCallId": "call_d6_decl_acct_outer_ag2_001"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's the rundown on Meridian Apparel Group."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill top-account — outer agent emits generate_a2ui (no args).",
|
||||
"match": {
|
||||
"userMessage": "Pull up the details on our biggest account.",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_acct_outer_ag2_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / gen-ui-headless-complete",
|
||||
"sourceFile": "headless-complete.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"_note": "Alias of the headless-complete demo's 4 gen-UI pills (weather/stock/highlight/revenue) for the gen-ui-headless-complete probe (showcase/harness/src/probes/scripts/d5-gen-ui-headless-complete.ts -> fixtureFile 'gen-ui-headless-complete.json'), which had no matching fixture in any slug so its first leg 503'd under strict. Narration (toolCallId) fixtures FIRST so they win when the last message is a matching tool result; toolcall (userMessage+context) fixtures AFTER with NO turnIndex gate so each of the 4 sequential pill turns in one chat thread matches regardless of prior assistant/tool history (the turnIndex multi-turn match-gate bug). Modeled on the green langgraph-python pattern. Interrupt fixtures intentionally omitted (interrupt-headless is a separate cluster item)."
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "headless-complete weather pill narration — matches when the get_weather tool result with id call_d5_headless_weather_001 is the last message. The 'tokyo' textToken assertion is satisfied by the city name. Narration verbatim matches the headless-complete.spec.ts assertion `toContainText('Tokyo is 22°C and partly cloudy.')`.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_weather_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Tokyo is 22°C and partly cloudy. The headless WeatherCard above shows the sunny snapshot from the tool."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete stock pill narration — matches when the get_stock_price tool result with id call_d5_headless_stock_001 is the last message. textToken 'aapl' satisfied by ticker mention.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_stock_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "AAPL is at $189.42, up 1.27% on the day."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete highlight pill narration — matches when the highlight_note tool result with id call_d5_headless_highlight_001 is the last message.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_highlight_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Highlighted 'ship the demo on Friday' for you above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete revenue chart narration — matches when the get_revenue_chart tool result with id call_d5_headless_revenue_chart_001 is the last message.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_revenue_chart_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the chart of revenue over the last six months — quarterly revenue grew steadily, peaking in June."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete weather pill toolcall — emit get_weather for Tokyo. Probe sends \"What's the weather in Tokyo?\". Narrowed to the full phrase 'What's the weather in Tokyo' so this no longer shadows the D6 tool-rendering 'Chain tools' pill (whose prompt contains 'weather in Tokyo' as a substring) — that pill needs to hit its own 3-tool emission fixture in tool-rendering.json. Only userMessage+context discriminators because narration fixtures above (toolCallId) already win once the tool result lands.",
|
||||
"match": {
|
||||
"userMessage": "What's the weather in Tokyo",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete stock pill toolcall — emit get_stock_price for AAPL. Probe sends \"What's the price of AAPL right now?\". Narrowed to 'price of AAPL right now' so this no longer shadows the D6 tool-rendering 'Stock price' pill which sends \"What's the current price of AAPL?\" (also containing 'price of AAPL' as a substring) — that pill needs its own deterministic-price fixture in tool-rendering.json.",
|
||||
"match": {
|
||||
"userMessage": "price of AAPL right now",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_stock_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"AAPL\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete highlight pill toolcall — emit highlight_note. Probe sends \"Highlight this note for me: 'ship the demo on Friday'.\" so userMessage substring 'ship the demo on Friday' catches it.",
|
||||
"match": {
|
||||
"userMessage": "ship the demo on Friday",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_highlight_001",
|
||||
"name": "highlight_note",
|
||||
"arguments": "{\"text\":\"ship the demo on Friday\",\"color\":\"yellow\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete revenue chart toolcall — emit get_revenue_chart (no args). Probe sends 'Show me a chart of revenue over the last six months.'. Includes a short leading `content` snippet so the word 'revenue' lands in the assistant bubble synchronously with the tool-call emission. Without this, the chart card's title falls back to 'Chart' during the `executing` render phase (result still undefined), and the conversation-runner's count-based settle can fire its 1500ms quiet window before the narration follow-up message arrives — leaving the assistant text without 'revenue' on the probe's first attempt.",
|
||||
"match": {
|
||||
"userMessage": "chart of revenue over the last six months",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pulling the revenue chart for the last six months...",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_revenue_chart_001",
|
||||
"name": "get_revenue_chart",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / gen-ui-interrupt",
|
||||
"sourceFile": "harness/fixtures/d5/gen-ui-interrupt.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "gen-ui-interrupt sales-call pill — first leg: schedule_meeting tool call triggers Python agent's interrupt(...). The toolName gate ensures this matches only when schedule_meeting is in the tool list (the agent must declare it). Placed BEFORE the toolCallId resume-leg so the duplicate-userMessage dedup does not shadow the tool-emitting leg — mirrors the passing mastra/pydantic-ai ordering.",
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolName": "schedule_meeting",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sure — let me check available times.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_schedule_sales_001",
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "Sales intro call",
|
||||
"attendee": "Sales team",
|
||||
"slots": [
|
||||
{ "label": "Mon 10:00 AM", "iso": "2026-05-11T10:00:00Z" },
|
||||
{ "label": "Tue 2:00 PM", "iso": "2026-05-12T14:00:00Z" }
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-interrupt sales-call pill — second leg: confirmation after user picks a slot and resolve fires. Keyed by toolCallId so it only matches when the prior tool result is being fed back (last message is the tool result for this call).",
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolCallId": "call_d5_schedule_sales_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked: Sales intro call confirmed for the slot you picked."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-interrupt alice-1on1 pill — first leg: schedule_meeting tool call. Placed BEFORE the toolCallId resume-leg so the tool-emitting leg is not shadowed.",
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolName": "schedule_meeting",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — pulling up next-week slots.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_schedule_alice_001",
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "1:1 with Alice — Q2 goals",
|
||||
"attendee": "Alice",
|
||||
"slots": [
|
||||
{ "label": "Wed 11:00 AM", "iso": "2026-05-13T11:00:00Z" },
|
||||
{ "label": "Thu 3:30 PM", "iso": "2026-05-14T15:30:00Z" }
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-interrupt alice-1on1 pill — second leg: confirmation after user picks a slot. Keyed by toolCallId so it only matches when the prior tool result is being fed back.",
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolCallId": "call_d5_schedule_alice_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Scheduled: 1:1 with Alice locked in for the slot you picked."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / gen-ui-open + gen-ui-open-advanced",
|
||||
"sourceFile": "d5-gen-ui-open.ts + d5-gen-ui-open-advanced.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "gen-ui-open — 3D axis visualization pill. The agent calls generateSandboxedUi which renders an iframe[srcdoc] with a non-trivial HTML payload (>= 100 chars). The probe asserts iframe presence + srcdoc length.",
|
||||
"match": {
|
||||
"userMessage": "3D axis visualization (model airplane)",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_open_genui_3d_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":400,\"placeholderMessages\":[\"Rendering 3D axis visualization…\"],\"css\":\"body{margin:0;background:#0f172a;display:flex;justify-content:center;align-items:center;height:100vh;font-family:system-ui}canvas{border-radius:8px;box-shadow:0 4px 24px rgba(0,0,0,.4)}#info{position:absolute;top:12px;left:12px;color:#94a3b8;font-size:12px}\",\"html\":\"<div id=\\\"info\\\">3D Axis — Model Airplane</div><canvas id=\\\"c\\\" width=\\\"400\\\" height=\\\"300\\\"></canvas>\",\"jsFunctions\":\"(function(){var c=document.getElementById('c');var ctx=c.getContext('2d');var cx=200,cy=150;function draw(t){ctx.clearRect(0,0,400,300);ctx.strokeStyle='#ef4444';ctx.beginPath();ctx.moveTo(cx,cy);ctx.lineTo(cx+Math.cos(t)*80,cy-Math.sin(t)*40);ctx.stroke();ctx.strokeStyle='#22c55e';ctx.beginPath();ctx.moveTo(cx,cy);ctx.lineTo(cx,cy-80);ctx.stroke();ctx.strokeStyle='#3b82f6';ctx.beginPath();ctx.moveTo(cx,cy);ctx.lineTo(cx-Math.sin(t)*60,cy-Math.cos(t)*30);ctx.stroke();ctx.fillStyle='#e2e8f0';ctx.font='10px system-ui';ctx.fillText('X',cx+Math.cos(t)*85,cy-Math.sin(t)*40);ctx.fillText('Y',cx-6,cy-85);ctx.fillText('Z',cx-Math.sin(t)*65,cy-Math.cos(t)*30);requestAnimationFrame(function(){draw(t+0.01);});}draw(0);})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-open — follow-up content after generateSandboxedUi returns.",
|
||||
"match": {
|
||||
"userMessage": "3D axis visualization (model airplane)",
|
||||
"toolCallId": "call_d6_open_genui_3d_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "3D axis visualization rendered above — the canvas shows rotating X (red), Y (green), and Z (blue) axes representing the model airplane's orientation."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-open-advanced — Inline expression evaluator pill. The agent calls generateSandboxedUi with sandbox-functions, producing a sandboxed iframe with allow-scripts. The probe asserts iframe[sandbox*='allow-scripts'] is present.",
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_open_genui_advanced_eval_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":320,\"placeholderMessages\":[\"Building expression evaluator…\"],\"css\":\"body{margin:0;font-family:monospace;background:#1e293b;color:#e2e8f0;padding:16px}input{width:100%;padding:8px;background:#0f172a;border:1px solid #334155;border-radius:4px;color:#e2e8f0;font-family:monospace;font-size:14px;box-sizing:border-box}#result{margin-top:12px;padding:12px;background:#0f172a;border-radius:4px;min-height:24px;font-size:16px}label{display:block;margin-bottom:6px;color:#94a3b8;font-size:12px}\",\"html\":\"<label>Expression</label><input id=\\\"expr\\\" placeholder=\\\"e.g. 2 + 2 * 3\\\" /><div id=\\\"result\\\">—</div>\",\"jsFunctions\":\"(function(){var input=document.getElementById('expr');var result=document.getElementById('result');input.addEventListener('input',async function(){var val=input.value.trim();if(!val){result.textContent='—';return;}try{var res=await Websandbox.connection.remote.evaluateExpression({expression:val});result.textContent=res&&res.ok?String(res.value):'err';}catch(e){result.textContent='err';}});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-open-advanced — follow-up content after generateSandboxedUi returns.",
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"toolCallId": "call_d6_open_genui_advanced_eval_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Expression evaluator rendered — type any math expression and it evaluates in real time via the sandbox bridge."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,231 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / gen-ui-tool-based",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "3D axis visualization (model airplane)",
|
||||
"toolCallId": "call_d5_open_gen_ui_3d_axis_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "3D axis visualization (model airplane)",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_3d_axis_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing 3D axis scene…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px}h1{font-size:14px;margin:0 0 8px}svg{display:block;background:#1e293b;border-radius:8px}\",\"html\":\"<div class=\\\"wrap\\\"><h1>3D axis visualization (pitch / yaw / roll)</h1><svg width=\\\"320\\\" height=\\\"260\\\" viewBox=\\\"-80 -80 160 160\\\" data-testid=\\\"ogui-3d-axis\\\"><line x1=\\\"-60\\\" y1=\\\"0\\\" x2=\\\"60\\\" y2=\\\"0\\\" stroke=\\\"#f59e0b\\\"/><line x1=\\\"0\\\" y1=\\\"-60\\\" x2=\\\"0\\\" y2=\\\"60\\\" stroke=\\\"#6366f1\\\"/><line x1=\\\"-40\\\" y1=\\\"40\\\" x2=\\\"40\\\" y2=\\\"-40\\\" stroke=\\\"#10b981\\\"/><text x=\\\"62\\\" y=\\\"4\\\" font-size=\\\"8\\\" fill=\\\"#f59e0b\\\">X pitch</text><text x=\\\"4\\\" y=\\\"-62\\\" font-size=\\\"8\\\" fill=\\\"#6366f1\\\">Y yaw</text><text x=\\\"42\\\" y=\\\"-42\\\" font-size=\\\"8\\\" fill=\\\"#10b981\\\">Z roll</text></svg></div>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "How a neural network works",
|
||||
"toolCallId": "call_d5_open_gen_ui_neural_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "How a neural network works",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_neural_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing neural network forward pass…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px}h1{font-size:14px;margin:0 0 8px}svg{display:block;background:#1e293b;border-radius:8px}circle{fill:#6366f1}\",\"html\":\"<div class=\\\"wrap\\\"><h1>Forward pass: input → hidden → output</h1><svg width=\\\"320\\\" height=\\\"220\\\" data-testid=\\\"ogui-neural-net\\\"><g><circle cx=\\\"40\\\" cy=\\\"40\\\" r=\\\"8\\\"/><circle cx=\\\"40\\\" cy=\\\"90\\\" r=\\\"8\\\"/><circle cx=\\\"40\\\" cy=\\\"140\\\" r=\\\"8\\\"/><circle cx=\\\"40\\\" cy=\\\"190\\\" r=\\\"8\\\"/></g><g fill=\\\"#a78bfa\\\"><circle cx=\\\"160\\\" cy=\\\"30\\\" r=\\\"8\\\"/><circle cx=\\\"160\\\" cy=\\\"75\\\" r=\\\"8\\\"/><circle cx=\\\"160\\\" cy=\\\"115\\\" r=\\\"8\\\"/><circle cx=\\\"160\\\" cy=\\\"155\\\" r=\\\"8\\\"/><circle cx=\\\"160\\\" cy=\\\"195\\\" r=\\\"8\\\"/></g><g fill=\\\"#10b981\\\"><circle cx=\\\"280\\\" cy=\\\"80\\\" r=\\\"8\\\"/><circle cx=\\\"280\\\" cy=\\\"140\\\" r=\\\"8\\\"/></g></svg></div>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Quicksort visualization",
|
||||
"toolCallId": "call_d5_open_gen_ui_quicksort_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Quicksort visualization",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_quicksort_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing quicksort animation…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px}h1{font-size:14px;margin:0 0 8px}svg{display:block;background:#1e293b;border-radius:8px}rect{fill:#64748b}\",\"html\":\"<div class=\\\"wrap\\\"><h1>Quicksort: partition around pivot</h1><svg width=\\\"320\\\" height=\\\"220\\\" data-testid=\\\"ogui-quicksort\\\"><rect x=\\\"10\\\" y=\\\"170\\\" width=\\\"24\\\" height=\\\"40\\\"/><rect x=\\\"40\\\" y=\\\"130\\\" width=\\\"24\\\" height=\\\"80\\\"/><rect x=\\\"70\\\" y=\\\"100\\\" width=\\\"24\\\" height=\\\"110\\\"/><rect x=\\\"100\\\" y=\\\"60\\\" width=\\\"24\\\" height=\\\"150\\\" fill=\\\"#f59e0b\\\"/><rect x=\\\"130\\\" y=\\\"80\\\" width=\\\"24\\\" height=\\\"130\\\" fill=\\\"#6366f1\\\"/><rect x=\\\"160\\\" y=\\\"110\\\" width=\\\"24\\\" height=\\\"100\\\"/><rect x=\\\"190\\\" y=\\\"50\\\" width=\\\"24\\\" height=\\\"160\\\"/><rect x=\\\"220\\\" y=\\\"90\\\" width=\\\"24\\\" height=\\\"120\\\"/><rect x=\\\"250\\\" y=\\\"140\\\" width=\\\"24\\\" height=\\\"70\\\"/><rect x=\\\"280\\\" y=\\\"160\\\" width=\\\"24\\\" height=\\\"50\\\"/></svg></div>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Fourier: square wave from sines",
|
||||
"toolCallId": "call_d5_open_gen_ui_fourier_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Fourier: square wave from sines",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_fourier_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing Fourier series animation…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px}h1{font-size:14px;margin:0 0 8px}svg{display:block;background:#1e293b;border-radius:8px}\",\"html\":\"<div class=\\\"wrap\\\"><h1>Fourier series: square wave</h1><svg width=\\\"320\\\" height=\\\"220\\\" viewBox=\\\"0 -60 320 120\\\" data-testid=\\\"ogui-fourier\\\"><circle cx=\\\"60\\\" cy=\\\"0\\\" r=\\\"40\\\" stroke=\\\"#6366f1\\\" fill=\\\"none\\\"/><circle cx=\\\"60\\\" cy=\\\"0\\\" r=\\\"13\\\" stroke=\\\"#a78bfa\\\" fill=\\\"none\\\"/><circle cx=\\\"60\\\" cy=\\\"0\\\" r=\\\"8\\\" stroke=\\\"#c4b5fd\\\" fill=\\\"none\\\"/><path d=\\\"M120 0 L300 0\\\" stroke=\\\"#10b981\\\" fill=\\\"none\\\"/></svg></div>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Calculator (calls evaluateExpression)",
|
||||
"toolCallId": "call_d5_open_gen_ui_calc_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Calculator (calls evaluateExpression)",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_calc_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:240px}.display{background:#1e293b;padding:8px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}button{padding:10px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',async function(){if(btn.id==='eq'){var res=await Websandbox.connection.remote.evaluateExpression({expression:expr});if(res&&res.ok){display.textContent=String(res.value);expr=String(res.value);}else{display.textContent='err';expr='';}}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Ping the host (calls notifyHost)",
|
||||
"toolCallId": "call_d5_open_gen_ui_ping_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Ping the host (calls notifyHost)",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_ping_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":320,\"placeholderMessages\":[\"Composing host-ping card…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.card{padding:24px;background:#1e293b;border-radius:12px;margin:16px;text-align:center}button{padding:10px 20px;background:#6366f1;border:0;color:#fff;border-radius:8px;font-size:14px;cursor:pointer}.out{margin-top:12px;font-size:12px;color:#94a3b8}\",\"html\":\"<div class=\\\"card\\\" data-testid=\\\"ogui-ping\\\"><h2>Notify the host</h2><button id=\\\"hi\\\">Say hi to the host</button><div class=\\\"out\\\" id=\\\"out\\\">awaiting click…</div></div>\",\"jsFunctions\":\"document.getElementById('hi').addEventListener('click',async function(){var out=document.getElementById('out');out.textContent='sending…';var res=await Websandbox.connection.remote.notifyHost({message:'Hello from sandbox'});out.textContent=res&&res.ok?'host replied at '+res.receivedAt:'failed';});\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"toolCallId": "call_d5_open_gen_ui_inline_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_inline_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":320,\"placeholderMessages\":[\"Composing expression evaluator…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.card{padding:16px;background:#1e293b;border-radius:12px;margin:16px}input{width:100%;padding:8px;background:#0f172a;border:1px solid #334155;color:#e2e8f0;border-radius:6px;box-sizing:border-box}button{margin-top:8px;padding:8px 16px;background:#6366f1;border:0;color:#fff;border-radius:6px;cursor:pointer}.out{margin-top:8px;font-size:12px;color:#94a3b8}\",\"html\":\"<div class=\\\"card\\\" data-testid=\\\"ogui-inline-eval\\\"><h2>Inline expression evaluator</h2><input id=\\\"in\\\" placeholder=\\\"e.g. 2 + 2\\\"/><button id=\\\"go\\\">Evaluate</button><div class=\\\"out\\\" id=\\\"out\\\">awaiting input…</div></div>\",\"jsFunctions\":\"(function(){var input=document.getElementById('in');var out=document.getElementById('out');var go=document.getElementById('go');async function run(){var expr=input.value;out.textContent='evaluating…';var res=await Websandbox.connection.remote.evaluateExpression({expression:expr});out.textContent=res&&res.ok?'= '+res.value:'error: '+res.error;}go.addEventListener('click',run);input.addEventListener('keydown',function(e){if(e.key==='Enter')run();});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "request the gen-ui interrupt",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The agent paused at a gen-UI interrupt and rendered a choice component for the user. Choose to continue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "confirm the gen-ui choice",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Gen-UI interrupt resolved. The agent received the user's choice and resumed, completing the workflow."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "render an open gen-ui element",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The open gen-UI element was rendered. The LLM produced an arbitrary-shape JSON payload and the renderer materialized it as a UI block."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "continue the advanced gen-ui flow",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The advanced gen-UI flow continued with the second-step component. The chained payloads from turns 1 and 2 form the complete advanced gen-UI sequence."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,135 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / headless-complete",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21",
|
||||
"_note": "Narration fixtures (toolCallId) listed FIRST so they win when last message is a matching tool result. Toolcall fixtures listed AFTER and use only userMessage+context so they match on the initial user turn regardless of prior assistant/tool history."
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "headless-complete weather pill narration — matches when the get_weather tool result with id call_d5_headless_weather_001 is the last message. The 'tokyo' textToken assertion is satisfied by the city name.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_weather_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Tokyo is sunny right now — about 68°F with light winds. Want a forecast?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete stock pill narration — matches when the get_stock_price tool result with id call_d5_headless_stock_001 is the last message. textToken 'aapl' satisfied by ticker mention.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_stock_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "AAPL is at $189.42, up 1.27% on the day."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete highlight pill narration — matches when the highlight_note tool result with id call_d5_headless_highlight_001 is the last message.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_highlight_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Highlighted 'ship the demo on Friday' for you above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete revenue chart narration — matches when the get_revenue_chart tool result with id call_d5_headless_revenue_chart_001 is the last message.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_revenue_chart_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the chart of revenue over the last six months — quarterly revenue grew steadily, peaking in June."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete weather pill toolcall — emit get_weather for Tokyo. Probe sends 'What's the weather in Tokyo?'. Only userMessage+context discriminators because narration fixtures above (toolCallId) already win once the tool result lands.",
|
||||
"match": {
|
||||
"userMessage": "weather in Tokyo",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete stock pill toolcall — emit get_stock_price for AAPL. Probe sends \"What's the price of AAPL right now?\".",
|
||||
"match": {
|
||||
"userMessage": "price of AAPL",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_stock_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"AAPL\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete highlight pill toolcall — emit highlight_note. Probe sends \"Highlight this note for me: 'ship the demo on Friday'.\" so userMessage substring 'ship the demo on Friday' catches it.",
|
||||
"match": {
|
||||
"userMessage": "ship the demo on Friday",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_highlight_001",
|
||||
"name": "highlight_note",
|
||||
"arguments": "{\"text\":\"ship the demo on Friday\",\"color\":\"yellow\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete revenue chart toolcall — emit get_revenue_chart (no args). Probe sends 'Show me a chart of revenue over the last six months.'. Includes a short leading `content` snippet so the word 'revenue' lands in the assistant bubble synchronously with the tool-call emission. Without this, the chart card's title falls back to 'Chart' during the `executing` render phase (result still undefined), and the conversation-runner's count-based settle can fire its 1500ms quiet window before the narration follow-up message arrives — leaving the assistant text without 'revenue' on the probe's first attempt.",
|
||||
"match": {
|
||||
"userMessage": "chart of revenue over the last six months",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pulling the revenue chart for the last six months...",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_revenue_chart_001",
|
||||
"name": "get_revenue_chart",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "trigger the headless interrupt",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Interrupt raised. The agent has paused at a graph-level interrupt and is waiting for user input. Reply with 'yes' or 'no' to resume."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "resolve the interrupt with yes",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Interrupt resolved with 'yes'. The agent resumed execution from the suspended node and produced its final answer."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / headless-simple",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "headless-simple pill 1: locks the deterministic greeting leading phrase. The reply is intentionally NON-boilerplate (i.e. NOT the showcase-assistant catch-all 'I can help you with weather lookups...') so the headless-simple spec's HELLO_LEADING assertion fails when fixture priority misroutes this prompt to the catch-all.",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one short sentence",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi! In one short sentence: I'm a CopilotKit demo agent here to help you try features."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-simple pill 2: locks the deterministic joke.",
|
||||
"match": {
|
||||
"userMessage": "Tell me a one-line joke",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the scarecrow win an award? Because he was outstanding in his field!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-simple pill 3: locks the deterministic fun fact.",
|
||||
"match": {
|
||||
"userMessage": "Give me a fun fact",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "A fun fact: Honey never spoils! Archaeologists have found pots of honey in ancient Egyptian tombs that are over 3,000 years old and still perfectly edible."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,122 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / hitl-in-app",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Issue a $50 refund to customer #12345",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_request_approval_001",
|
||||
"name": "request_user_approval",
|
||||
"arguments": "{\"message\":\"Issue a $50 refund to customer #12345.\",\"context\":\"Per the standard goodwill-credit policy for shipping delays.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Issue a $50 refund to customer #12345",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — refund pill (#12345), post-tool-result response. Content includes BOTH approve and reject key phrases so both test assertions can match (the test checks for a substring, not the full string). This avoids sequenceIndex which is fragile across aimock restarts.",
|
||||
"match": {
|
||||
"userMessage": "$50 refund to Jordan Rivera on ticket #12345",
|
||||
"toolCallId": "call_d5_hitl_refund_12345_001",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "I am processing the $50 refund to Jordan Rivera on ticket #12345 now. The refund request was not approved by default — your explicit approval or rejection determines the outcome."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — refund pill (#12345). 1st turn: emit request_user_approval tool call. The post-tool-result fixture above (with toolCallId + hasToolResult: true) is more specific and wins on the 2nd turn, so hasToolResult: false is NOT needed here — omitting it allows this fixture to match even in multi-pill conversations where prior tool results exist.",
|
||||
"match": {
|
||||
"userMessage": "$50 refund to Jordan Rivera on ticket #12345",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_hitl_refund_12345_001",
|
||||
"name": "request_user_approval",
|
||||
"arguments": "{\"message\":\"Issue a $50 refund to Jordan Rivera on ticket #12345 for the duplicate charge.\",\"context\":\"Ticket #12345 — duplicate-charge goodwill credit.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — downgrade pill (#12346), 2nd turn. Same pattern as refund/escalate. Without this entry, the generic 'plan' catch-all in feature-parity.json hijacks the downgrade message.",
|
||||
"match": {
|
||||
"userMessage": "downgrade Priya Shah (#12346) to the Starter plan",
|
||||
"toolCallId": "call_d5_hitl_downgrade_12346_001",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Downgrade confirmed — Priya Shah (#12346) will move to the Starter plan effective next billing cycle."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — downgrade pill (#12346). 1st turn: emit request_user_approval tool call. Specific userMessage substring takes precedence over the bare 'plan' fixture in feature-parity.json (which lives later in the load order). hasToolResult: false omitted so multi-pill conversations still match.",
|
||||
"match": {
|
||||
"userMessage": "downgrade Priya Shah (#12346) to the Starter plan",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_hitl_downgrade_12346_001",
|
||||
"name": "request_user_approval",
|
||||
"arguments": "{\"message\":\"Downgrade Priya Shah (#12346) to the Starter plan effective next billing cycle.\",\"context\":\"Ticket #12346 — voluntary downgrade per customer request.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — escalate pill (#12347), post-tool-result response. Content includes both approve and reject key phrases.",
|
||||
"match": {
|
||||
"userMessage": "escalate ticket #12347 to the payments team",
|
||||
"toolCallId": "call_d5_hitl_escalate_12347_001",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Escalated ticket #12347 to the payments team for Morgan Lee. Not escalated by default — your explicit approval determines the outcome."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — escalate pill (#12347). 1st turn: emit request_user_approval tool call. hasToolResult: false omitted so multi-pill conversations still match (the post-tool-result fixture above with toolCallId is more specific and wins on the 2nd turn).",
|
||||
"match": {
|
||||
"userMessage": "escalate ticket #12347 to the payments team",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_hitl_escalate_12347_001",
|
||||
"name": "request_user_approval",
|
||||
"arguments": "{\"message\":\"Escalate ticket #12347 to the payments team for Morgan Lee's stuck payment.\",\"context\":\"Ticket #12347 — payment stuck, needs payments-team triage.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / hitl-in-chat",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Book a 30-minute onboarding call for Alice",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_book_call_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Onboarding call\",\"attendee\":\"Alice\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Book a 30-minute onboarding call for Alice",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolCallId": "call_d5_schedule_sales_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked: Sales intro call confirmed for the slot you picked."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolName": "schedule_meeting",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "Sales intro call",
|
||||
"attendee": "Sales team",
|
||||
"slots": [
|
||||
{
|
||||
"label": "Mon 10:00 AM",
|
||||
"iso": "2026-05-11T10:00:00Z"
|
||||
},
|
||||
{
|
||||
"label": "Tue 2:00 PM",
|
||||
"iso": "2026-05-12T14:00:00Z"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": "call_d5_schedule_sales_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolCallId": "call_d5_schedule_alice_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Scheduled: 1:1 with Alice locked in for the slot you picked."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolName": "schedule_meeting",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "1:1 with Alice — Q2 goals",
|
||||
"attendee": "Alice",
|
||||
"slots": [
|
||||
{
|
||||
"label": "Wed 11:00 AM",
|
||||
"iso": "2026-05-13T11:00:00Z"
|
||||
},
|
||||
{
|
||||
"label": "Thu 3:30 PM",
|
||||
"iso": "2026-05-14T15:30:00Z"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": "call_d5_schedule_alice_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / interrupt-headless",
|
||||
"sourceFile": "harness/fixtures/d5/interrupt-headless.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "interrupt-headless sales-call pill — second leg: confirmation after user picks a slot. Placed BEFORE first-leg so toolCallId matching takes precedence over toolName matching.",
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolCallId": "call_d5_schedule_sales_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked: Sales intro call confirmed for the slot you picked."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "interrupt-headless alice-1on1 pill — second leg: confirmation after user picks a slot. Placed BEFORE first-leg so toolCallId matching takes precedence.",
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolCallId": "call_d5_schedule_alice_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Scheduled: 1:1 with Alice locked in for the slot you picked."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "interrupt-headless sales-call pill — first leg: schedule_meeting tool call triggers interrupt(...). Same agent/prompts as gen-ui-interrupt but different frontend (useHeadlessInterrupt renders in app surface pane, not inline).",
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolName": "schedule_meeting",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sure — let me check available times.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_schedule_sales_001",
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "Sales intro call",
|
||||
"attendee": "Sales team",
|
||||
"slots": [
|
||||
{ "label": "Mon 10:00 AM", "iso": "2026-05-11T10:00:00Z" },
|
||||
{ "label": "Tue 2:00 PM", "iso": "2026-05-12T14:00:00Z" }
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "interrupt-headless alice-1on1 pill — first leg: schedule_meeting tool call.",
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolName": "schedule_meeting",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — pulling up next-week slots.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_schedule_alice_001",
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "1:1 with Alice — Q2 goals",
|
||||
"attendee": "Alice",
|
||||
"slots": [
|
||||
{ "label": "Wed 11:00 AM", "iso": "2026-05-13T11:00:00Z" },
|
||||
{ "label": "Thu 3:30 PM", "iso": "2026-05-14T15:30:00Z" }
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / mcp-apps",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Open Excalidraw and sketch a system diagram",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sketched a three-box system diagram (Client → Server → Database). Tap the iframe to open it in Excalidraw."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D5 mcp-apps probe — verbatim probe prompt drives a real `create_view` MCP tool call so the runtime's MCP Apps middleware fetches the UI resource and mounts the iframe. Mirrored from showcase/aimock/d6/langgraph-python/tool-rendering-reasoning-chain.json (the LGP gold-standard entry) — ag2 was the only integration missing this turn-1 fixture, so strict-mode staging returned 503 for ag2 + this prompt + create_view while all other integrations matched.",
|
||||
"match": {
|
||||
"userMessage": "Open Excalidraw and sketch a system diagram",
|
||||
"toolName": "create_view",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "The user wants a sketch of a client/server/database architecture. I'll call create_view once with three labelled rectangles connected by arrows and a title, framed by a cameraUpdate.",
|
||||
"content": "Sketching a client → server → database diagram in Excalidraw.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_mcp_apps_create_view_001",
|
||||
"name": "create_view",
|
||||
"arguments": "{\"elements\":[{\"id\":\"title\",\"type\":\"text\",\"x\":260,\"y\":40,\"text\":\"System Diagram\",\"fontSize\":24},{\"id\":\"client\",\"type\":\"rectangle\",\"x\":80,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"Client\",\"fontSize\":18}},{\"id\":\"server\",\"type\":\"rectangle\",\"x\":320,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"Server\",\"fontSize\":18}},{\"id\":\"database\",\"type\":\"rectangle\",\"x\":560,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"Database\",\"fontSize\":18}},{\"id\":\"a1\",\"type\":\"arrow\",\"x\":240,\"y\":195,\"endX\":320,\"endY\":195},{\"id\":\"a2\",\"type\":\"arrow\",\"x\":480,\"y\":195,\"endX\":560,\"endY\":195},{\"id\":\"camera\",\"type\":\"cameraUpdate\",\"x\":40,\"y\":0,\"width\":800,\"height\":600}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "D5 mcp-apps suggestion pill #1 (\"Draw a flowchart\"). Sends the verbatim suggestion message \"Use Excalidraw to draw a simple flowchart with three steps.\" The distinctive substring \"draw a simple flowchart\" prevents collision with feature-parity.json's generic `{userMessage: \"steps\"}` fixture, which would otherwise absorb the prompt and return a tool-call-free planning blurb (breaking the iframe). Turn 1 emits `create_view` with three-step flowchart elements so the MCP middleware fetches the Excalidraw UI resource and mounts the iframe; turn 2 (after the tool result) emits the deterministic narration.",
|
||||
"match": {
|
||||
"userMessage": "draw a simple flowchart",
|
||||
"toolName": "create_view",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "The user wants a simple three-step flowchart. I'll call create_view once with three labelled rectangles connected by arrows and a title, framed by a cameraUpdate.",
|
||||
"content": "Drawing a simple three-step flowchart in Excalidraw.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_mcp_apps_create_view_flowchart_001",
|
||||
"name": "create_view",
|
||||
"arguments": "{\"elements\":[{\"id\":\"title\",\"type\":\"text\",\"x\":300,\"y\":40,\"text\":\"Flowchart\",\"fontSize\":24},{\"id\":\"step1\",\"type\":\"rectangle\",\"x\":80,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"Start\",\"fontSize\":18}},{\"id\":\"step2\",\"type\":\"rectangle\",\"x\":320,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"Process\",\"fontSize\":18}},{\"id\":\"step3\",\"type\":\"rectangle\",\"x\":560,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"End\",\"fontSize\":18}},{\"id\":\"a1\",\"type\":\"arrow\",\"x\":240,\"y\":195,\"endX\":320,\"endY\":195},{\"id\":\"a2\",\"type\":\"arrow\",\"x\":480,\"y\":195,\"endX\":560,\"endY\":195},{\"id\":\"camera\",\"type\":\"cameraUpdate\",\"x\":40,\"y\":0,\"width\":800,\"height\":600}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "draw a simple flowchart",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Drew a three-step flowchart (Start → Process → End). Tap the iframe to open it in Excalidraw."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Use Excalidraw to sketch",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sketched a simple system diagram for you above — three boxes (frontend, runtime, agent) connected by arrows showing the request flow. Ask if you'd like a different shape or more detail."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / multimodal",
|
||||
"sourceFile": "d5-multimodal.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "multimodal — sample image turn. The probe clicks 'Try with sample image' which auto-sends via agent.addMessage. The assertion checks the assistant transcript contains the keyword 'image'. The userMessage matches the autoPrompt in sample-attachment-buttons.tsx.",
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The image attachment shows a small abstract test pattern used by the multimodal demo to validate the image-upload pipeline. Successful render of this response confirms the binary attachment round-tripped through the runtime."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "multimodal — sample PDF turn. The probe clicks 'Try with sample PDF' which auto-sends. The assertion checks the assistant transcript contains the keyword 'document'.",
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The PDF document contains a single test page used by the multimodal demo. Its text was flattened by pypdf on the Python side and forwarded as text content to the model. Receiving this response confirms the document upload path works."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / prebuilt-popup",
|
||||
"sourceFile": "d5-prebuilt-popup.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "prebuilt-popup — single turn proving the CopilotPopup component renders and messages round-trip inside its .copilotKitPopup root. The probe asserts the popup root is visible and the assistant response lands inside it.",
|
||||
"match": {
|
||||
"userMessage": "hi from the popup test",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello from the popup — the CopilotPopup prebuilt component is wired up and reachable. The launcher floats in the corner and the chat sits in an overlay panel."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / prebuilt-sidebar",
|
||||
"sourceFile": "d5-prebuilt-sidebar.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "prebuilt-sidebar — single turn proving the CopilotSidebar component renders and messages round-trip inside its .copilotKitSidebar root. The probe asserts the sidebar root is visible and the assistant response lands inside it.",
|
||||
"match": {
|
||||
"userMessage": "hi from the sidebar test",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello from the sidebar — the CopilotSidebar prebuilt component is wired up and reachable. Anything else you would like me to confirm from inside the sidebar?"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / readonly-state",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "readonly-state-agent-context — Who am I pill. Matches the verbatim pill prompt. systemMessage gating on 'Atai' was removed because @langchain/openai >=1.4 sends SystemMessage as role:'developer' for gpt-5 models, and aimock's getSystemText only checks role:'system'. The userMessage alone is deterministic from the pill.",
|
||||
"match": {
|
||||
"userMessage": "What do you know about me from my context?",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "I see you're Atai, and you're in the America/Los_Angeles timezone. Recently, you viewed the pricing page and watched the product demo video. How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "readonly-state-agent-context — Suggest next steps pill. Matches the verbatim pill prompt. systemMessage gating removed (see Who am I comment above).",
|
||||
"match": {
|
||||
"userMessage": "Based on my recent activity, what should I try next?",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Since you recently viewed the pricing page and watched the product demo video, it might be a good idea to explore user testimonials or case studies to see how others have benefited from the Pro Plan. You could also start the 14-day free trial to experience the features firsthand."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What do you know about me from my context",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your context I can see your name and recent activity. I'll keep responses calibrated to your timezone."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / reasoning",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "reasoning-default e2e test pill prompt. Substring 'sky appears blue' matches 'Explain step by step why the sky appears blue during the day but red at sunset.' Includes a `reasoning` field so aimock emits REASONING_MESSAGE_* events; without it the built-in CopilotChatReasoningMessage 'Thinking…/Thought for…' label never lands.",
|
||||
"match": {
|
||||
"userMessage": "sky appears blue",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "First, I considered that visible sunlight contains all wavelengths. Then I noted that air molecules scatter shorter wavelengths (blue) more efficiently than longer ones (Rayleigh scattering). At sunset the path through the atmosphere is much longer, so even more blue is scattered out and the remaining direct light skews red.",
|
||||
"content": "Daytime sky looks blue because air molecules scatter short-wavelength light more strongly than long-wavelength light (Rayleigh scattering). At sunset the sun's light traverses far more atmosphere, scattering out most of the blue and leaving the remaining direct light dominated by red and orange wavelengths."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D5 reasoning-display probe (harness/src/probes/scripts/d5-reasoning-display.ts) sends 'show your reasoning step by step' verbatim. Mirrored from showcase/harness/fixtures/d5/reasoning-display.json so the bundled aimock has a match (the harness fixture file isn't loaded by the Docker-baked aimock).",
|
||||
"match": {
|
||||
"userMessage": "show your reasoning step by step",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "First, I identified that the question requires step-by-step reasoning. Then I broke it into sub-steps and worked through each one. Finally, I aggregated the partial answers into a single response.",
|
||||
"content": "Reasoning: first, I identified the question requires step-by-step thinking. Then, I broke it into sub-steps and worked through each one. Finally, I aggregated the partial answers into a single response. The reasoning block above the answer demonstrates intermediate-thought rendering."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,148 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / recorded",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-54-59-257Z-b614f3f8.json",
|
||||
"match": {
|
||||
"userMessage": "First use the query_data tool to fetch the financial sales data, then using A2UI, show me a sales dashboard with total revenue, new customers, and conversion rate metrics. Include a pie chart of revenue by category and a bar chart of monthly sales.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 0,
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_data",
|
||||
"arguments": "{\"query\":\"financial sales data including total revenue, new customers, conversion rate, revenue by category, and monthly sales\"}",
|
||||
"id": "call_jWKVlcdnnYiV6MT3wzNZphnF"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-55-00-312Z-814153b0.json",
|
||||
"match": {
|
||||
"userMessage": "First use the query_data tool to fetch the financial sales data, then using A2UI, show me a sales dashboard with total revenue, new customers, and conversion rate metrics. Include a pie chart of revenue by category and a bar chart of monthly sales.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 1,
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}",
|
||||
"id": "call_K6au6wp1CjG4OuiGzhL1yCXL"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-55-05-134Z-890eead9.json",
|
||||
"match": {
|
||||
"userMessage": "First use the query_data tool to fetch the financial sales data, then using A2UI, show me a sales dashboard with total revenue, new customers, and conversion rate metrics. Include a pie chart of revenue by category and a bar chart of monthly sales.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 2,
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\": \"Revenue by Category\", \"description\": \"Distribution of revenue across different categories\", \"data\": [{\"label\": \"Enterprise Subscriptions\", \"value\": 94000}, {\"label\": \"Pro Tier Upgrades\", \"value\": 66500}, {\"label\": \"API Usage Overages\", \"value\": 34500}, {\"label\": \"Consulting Services\", \"value\": 54000}, {\"label\": \"Marketplace Sales\", \"value\": 42800}, {\"label\": \"Partnership Revenue\", \"value\": 25700}, {\"label\": \"Training & Workshops\", \"value\": 10200}]}",
|
||||
"id": "call_OJDsrTtDcWoV0PObgKtU9fXv"
|
||||
},
|
||||
{
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\": \"Monthly Sales\", \"description\": \"Sales performance over the months\", \"data\": [{\"label\": \"January\", \"value\": 82800}, {\"label\": \"February\", \"value\": 119000}, {\"label\": \"March\", \"value\": 135700}]}",
|
||||
"id": "call_o9RdNrZTq8qV0Majn12OTZcu"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-55-07-715Z-9531bf64.json",
|
||||
"match": {
|
||||
"userMessage": "First use the query_data tool to fetch the financial sales data, then using A2UI, show me a sales dashboard with total revenue, new customers, and conversion rate metrics. Include a pie chart of revenue by category and a bar chart of monthly sales.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 3,
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The sales dashboard is now available, featuring metrics for total revenue, new customers, and conversion rate. Additionally, you can view a pie chart of revenue by category and a bar chart of monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-56-02-547Z-7afa30ad.json. SUPERSEDED: this entry is unreachable and editing it has no effect — the calculator pill is served by the specific calculator fixtures in _from-feature-parity.json (loaded first alphabetically; first match wins). Note the model gate (gpt-4o) is NOT an independent safety net — whether it matches depends on each integration's current model; unreachability is guaranteed only by the load-order shadowing above. Kept for historical reference; do not delete.",
|
||||
"match": {
|
||||
"userMessage": "Using the generateSandboxedUi tool, build a modern calculator with standard buttons plus labeled metric shortcut buttons that insert their values into the display when clicked. Use sample company data.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 0,
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":400,\"placeholderMessages\":[\"Building calculator UI...\"],\"css\":\"body { font-family: Arial, sans-serif; display: flex; justify-content: center; align-items: center; height: 100vh; margin: 0; background-color: #f4f4f9; } .calculator { width: 320px; background-color: #fff; border-radius: 10px; box-shadow: 0 4px 10px rgba(0, 0, 0, 0.1); } .display { background-color: #222; color: #fff; font-size: 2em; padding: 20px; text-align: right; border-top-left-radius: 10px; border-top-right-radius: 10px; } .buttons { display: grid; grid-template-columns: repeat(4, 1fr); grid-gap: 1px; } .button { background-color: #e0e0e0; padding: 20px; font-size: 1.2em; text-align: center; cursor: pointer; transition: background-color 0.2s; } .button:hover { background-color: #d0d0d0; } .metric-button { background-color: #4caf50; color: #fff; }\",\"html\":\"<div class='calculator'> <div class='display' id='display'>0</div> <div class='buttons'> <div class='button' onclick='appendNumber(7)'>7</div> <div class='button' onclick='appendNumber(8)'>8</div> <div class='button' onclick='appendNumber(9)'>9</div> <div class='button' onclick='operate(\\\"/\\\")'>/</div> <div class='button' onclick='appendNumber(4)'>4</div> <div class='button' onclick='appendNumber(5)'>5</div> <div class='button' onclick='appendNumber(6)'>6</div> <div class='button' onclick='operate(\\\"*\\\")'>*</div> <div class='button' onclick='appendNumber(1)'>1</div> <div class='button' onclick='appendNumber(2)'>2</div> <div class='button' onclick='appendNumber(3)'>3</div> <div class='button' onclick='operate(\\\"-\\\")'>-</div> <div class='button' onclick='appendNumber(0)'>0</div> <div class='button' onclick='clearDisplay()'>C</div> <div class='button' onclick='calculate()'>=</div> <div class='button' onclick='operate(\\\"+\\\")'>+</div> <div class='button metric-button' onclick='insertMetric(1000)'>Revenue</div> <div class='button metric-button' onclick='insertMetric(500)'>Expenses</div> <div class='button metric-button' onclick='insertMetric(200)'>Profit</div> </div> </div>\",\"jsFunctions\":\"function appendNumber(number) { const display = document.getElementById('display'); if (display.innerText === '0') { display.innerText = number; } else { display.innerText += number; } } function operate(operator) { const display = document.getElementById('display'); display.innerText += ' ' + operator + ' '; } function clearDisplay() { document.getElementById('display').innerText = '0'; } function calculate() { const display = document.getElementById('display'); try { display.innerText = eval(display.innerText); } catch { display.innerText = 'Error'; } } function insertMetric(value) { const display = document.getElementById('display'); if (display.innerText === '0') { display.innerText = value; } else { display.innerText += value; } }\",\"jsExpressions\":[]}",
|
||||
"id": "call_u2aFJvWZUKzzlvqfnDeWhMyC"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-56-04-559Z-1bafed55.json. SUPERSEDED: this entry is unreachable and editing it has no effect — the calculator pill is served by the specific calculator fixtures in _from-feature-parity.json (loaded first alphabetically; first match wins). Note the model gate (gpt-4o) is NOT an independent safety net — whether it matches depends on each integration's current model; unreachability is guaranteed only by the load-order shadowing above. Kept for historical reference; do not delete.",
|
||||
"match": {
|
||||
"userMessage": "Using the generateSandboxedUi tool, build a modern calculator with standard buttons plus labeled metric shortcut buttons that insert their values into the display when clicked. Use sample company data.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 1,
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The modern calculator with standard buttons and labeled metric shortcut buttons is ready for use. You can perform calculations and insert sample company data values like Revenue, Expenses, and Profit directly into the display."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T22-01-34-384Z-edf48666.json",
|
||||
"match": {
|
||||
"userMessage": "What is the weather in Tokyo?",
|
||||
"model": "gpt-4o-mini",
|
||||
"turnIndex": 0,
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}",
|
||||
"id": "call_0uXBAr4ySTrLQGQgszQYFtEJ"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T22-01-35-344Z-6ea6b18a.json",
|
||||
"match": {
|
||||
"userMessage": "What is the weather in Tokyo?",
|
||||
"model": "gpt-4o-mini",
|
||||
"turnIndex": 1,
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The weather in Tokyo is currently 68°F, with 55% humidity, a wind speed of 10 mph, and sunny conditions."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,494 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / render-a2ui",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "render the a2ui schema",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The A2UI fixed-schema component was rendered. The schema-driven UI received the agent's payload and produced the corresponding UI element from the locked schema definition."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "have the agent emit a ui",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The agent emitted a UI block as part of its turn. The agent acts as the UI generator: its response payload describes the component and the renderer materialized it inline with the assistant message."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a pie chart of revenue by category",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_render_pie_chart_001",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category (Q4)\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a pie chart of revenue by category",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "render the declarative card",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The declarative gen-UI specification has been resolved into a rendered card. The component descriptor was forwarded to the frontend renderer which materialized the card declaratively from the schema."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a profile card for Ada Lovelace",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_show_card_001",
|
||||
"name": "show_card",
|
||||
"arguments": "{\"title\":\"Ada Lovelace\",\"body\":\"English mathematician (1815\\u20131852), credited as the first computer programmer for her notes on Charles Babbage's Analytical Engine \\u2014 including what is now recognized as the first algorithm intended to be carried out by a machine.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a profile card for Ada Lovelace",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "trip to mars",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_generate_steps_001",
|
||||
"name": "generate_task_steps",
|
||||
"arguments": "{\"steps\":[{\"description\":\"Research Mars mission requirements and timeline\",\"status\":\"enabled\"},{\"description\":\"Design spacecraft and life support systems\",\"status\":\"enabled\"},{\"description\":\"Recruit and train the crew\",\"status\":\"enabled\"},{\"description\":\"Launch and navigate to Mars\",\"status\":\"enabled\"},{\"description\":\"Land and establish base camp\",\"status\":\"enabled\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "trip to mars",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Great choices! I will proceed with executing the selected steps for your trip to Mars. Let me work through each one."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "KPI dashboard",
|
||||
"toolCallId": "call_d5_a2ui_dynamic_kpi_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the KPI dashboard you requested."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "KPI dashboard",
|
||||
"toolName": "generate_a2ui",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{\"context\":\"KPI dashboard\"}",
|
||||
"id": "call_d5_a2ui_dynamic_kpi_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "KPI dashboard",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_design_a2ui_kpi_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\": \"kpi-dashboard\", \"catalogId\": \"declarative-gen-ui-catalog\", \"components\": [{\"id\": \"root\", \"component\": \"Card\", \"title\": \"Quarterly KPIs\", \"subtitle\": \"Revenue, signups, and churn\", \"child\": \"metrics-row\"}, {\"id\": \"metrics-row\", \"component\": \"Row\", \"children\": [\"m-rev\", \"m-sign\", \"m-churn\"], \"gap\": 16}, {\"id\": \"m-rev\", \"component\": \"Metric\", \"label\": \"Revenue\", \"value\": \"$1.24M\", \"trend\": \"up\"}, {\"id\": \"m-sign\", \"component\": \"Metric\", \"label\": \"Signups\", \"value\": \"8,420\", \"trend\": \"up\"}, {\"id\": \"m-churn\", \"component\": \"Metric\", \"label\": \"Churn\", \"value\": \"2.3%\", \"trend\": \"down\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of sales by region",
|
||||
"toolCallId": "call_d5_a2ui_dynamic_pie_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the pie chart by region."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of sales by region",
|
||||
"toolName": "generate_a2ui",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{\"context\":\"pie chart of sales by region\"}",
|
||||
"id": "call_d5_a2ui_dynamic_pie_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of sales by region",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_design_a2ui_pie_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\": \"pie-sales\", \"catalogId\": \"declarative-gen-ui-catalog\", \"components\": [{\"id\": \"root\", \"component\": \"PieChart\", \"title\": \"Sales by region\", \"description\": \"Q4 revenue split\", \"data\": [{\"label\": \"NA\", \"value\": 540}, {\"label\": \"EMEA\", \"value\": 320}, {\"label\": \"APAC\", \"value\": 210}, {\"label\": \"LATAM\", \"value\": 90}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly revenue",
|
||||
"toolCallId": "call_d5_a2ui_dynamic_bar_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the bar chart of quarterly revenue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly revenue",
|
||||
"toolName": "generate_a2ui",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{\"context\":\"bar chart of quarterly revenue\"}",
|
||||
"id": "call_d5_a2ui_dynamic_bar_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly revenue",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_design_a2ui_bar_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\": \"bar-quarterly\", \"catalogId\": \"declarative-gen-ui-catalog\", \"components\": [{\"id\": \"root\", \"component\": \"BarChart\", \"title\": \"Quarterly revenue\", \"description\": \"FY 2025 per quarter\", \"data\": [{\"label\": \"Q1\", \"value\": 820}, {\"label\": \"Q2\", \"value\": 950}, {\"label\": \"Q3\", \"value\": 1100}, {\"label\": \"Q4\", \"value\": 1240}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "status report on system health",
|
||||
"toolCallId": "call_d5_a2ui_dynamic_status_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the system health status report."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "status report on system health",
|
||||
"toolName": "generate_a2ui",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{\"context\":\"status report on system health\"}",
|
||||
"id": "call_d5_a2ui_dynamic_status_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "status report on system health",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_design_a2ui_status_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\": \"status-report\", \"catalogId\": \"declarative-gen-ui-catalog\", \"components\": [{\"id\": \"root\", \"component\": \"Card\", \"title\": \"System health\", \"subtitle\": \"Live status\", \"child\": \"rows\"}, {\"id\": \"rows\", \"component\": \"Column\", \"children\": [\"r-api\", \"r-db\", \"r-bg\"], \"gap\": 8}, {\"id\": \"r-api\", \"component\": \"Row\", \"children\": [\"l-api\", \"b-api\"], \"gap\": 8}, {\"id\": \"l-api\", \"component\": \"InfoRow\", \"label\": \"API\", \"value\": \"p99 142ms\"}, {\"id\": \"b-api\", \"component\": \"StatusBadge\", \"text\": \"Healthy\", \"variant\": \"success\"}, {\"id\": \"r-db\", \"component\": \"Row\", \"children\": [\"l-db\", \"b-db\"], \"gap\": 8}, {\"id\": \"l-db\", \"component\": \"InfoRow\", \"label\": \"Database\", \"value\": \"Replication lag 220ms\"}, {\"id\": \"b-db\", \"component\": \"StatusBadge\", \"text\": \"Degraded\", \"variant\": \"warning\"}, {\"id\": \"r-bg\", \"component\": \"Row\", \"children\": [\"l-bg\", \"b-bg\"], \"gap\": 8}, {\"id\": \"l-bg\", \"component\": \"InfoRow\", \"label\": \"Background workers\", \"value\": \"Queue depth 12\"}, {\"id\": \"b-bg\", \"component\": \"StatusBadge\", \"text\": \"Healthy\", \"variant\": \"success\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "render_a2ui — KPI dashboard pill (Google-ADK secondary tool name). Card has a single `child` slot (per myDefinitions.Card.props.child: string), so we wrap the three Metrics in a basic-catalog Column to satisfy the multi-child layout.",
|
||||
"match": {
|
||||
"userMessage": "KPI dashboard",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_a2ui",
|
||||
"arguments": {
|
||||
"surfaceId": "declarative-surface",
|
||||
"catalogId": "declarative-gen-ui-catalog",
|
||||
"components": [
|
||||
{
|
||||
"id": "root",
|
||||
"component": "Card",
|
||||
"title": "KPI dashboard",
|
||||
"subtitle": "Last 30 days",
|
||||
"child": "metrics-col"
|
||||
},
|
||||
{
|
||||
"id": "metrics-col",
|
||||
"component": "Column",
|
||||
"children": [
|
||||
"metric-revenue",
|
||||
"metric-signups",
|
||||
"metric-churn"
|
||||
],
|
||||
"gap": 12
|
||||
},
|
||||
{
|
||||
"id": "metric-revenue",
|
||||
"component": "Metric",
|
||||
"label": "Revenue",
|
||||
"value": "$1.2M",
|
||||
"trend": "up"
|
||||
},
|
||||
{
|
||||
"id": "metric-signups",
|
||||
"component": "Metric",
|
||||
"label": "Signups",
|
||||
"value": "4,820",
|
||||
"trend": "up"
|
||||
},
|
||||
{
|
||||
"id": "metric-churn",
|
||||
"component": "Metric",
|
||||
"label": "Churn",
|
||||
"value": "2.1%",
|
||||
"trend": "down"
|
||||
}
|
||||
],
|
||||
"data": {}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "render_a2ui — pie-chart pill (Google-ADK). Flat {id, component, ...props} shape.",
|
||||
"match": {
|
||||
"userMessage": "pie chart of sales by region",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_a2ui",
|
||||
"arguments": {
|
||||
"surfaceId": "declarative-surface",
|
||||
"catalogId": "declarative-gen-ui-catalog",
|
||||
"components": [
|
||||
{
|
||||
"id": "root",
|
||||
"component": "PieChart",
|
||||
"title": "Sales by region",
|
||||
"description": "Q4 — share of total revenue",
|
||||
"data": [
|
||||
{
|
||||
"label": "North America",
|
||||
"value": 540
|
||||
},
|
||||
{
|
||||
"label": "EMEA",
|
||||
"value": 320
|
||||
},
|
||||
{
|
||||
"label": "APAC",
|
||||
"value": 210
|
||||
},
|
||||
{
|
||||
"label": "LATAM",
|
||||
"value": 90
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"data": {}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "render_a2ui — bar-chart pill (Google-ADK). Flat {id, component, ...props} shape.",
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly revenue",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_a2ui",
|
||||
"arguments": {
|
||||
"surfaceId": "declarative-surface",
|
||||
"catalogId": "declarative-gen-ui-catalog",
|
||||
"components": [
|
||||
{
|
||||
"id": "root",
|
||||
"component": "BarChart",
|
||||
"title": "Quarterly revenue",
|
||||
"description": "FY24 — USD thousands",
|
||||
"data": [
|
||||
{
|
||||
"label": "Q1",
|
||||
"value": 820
|
||||
},
|
||||
{
|
||||
"label": "Q2",
|
||||
"value": 940
|
||||
},
|
||||
{
|
||||
"label": "Q3",
|
||||
"value": 1080
|
||||
},
|
||||
{
|
||||
"label": "Q4",
|
||||
"value": 1240
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"data": {}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "render_a2ui — status-report pill (Google-ADK). Card has single `child` slot, so wrap the three StatusBadges in a basic-catalog Column.",
|
||||
"match": {
|
||||
"userMessage": "status report on system health",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_a2ui",
|
||||
"arguments": {
|
||||
"surfaceId": "declarative-surface",
|
||||
"catalogId": "declarative-gen-ui-catalog",
|
||||
"components": [
|
||||
{
|
||||
"id": "root",
|
||||
"component": "Card",
|
||||
"title": "System health",
|
||||
"subtitle": "All services",
|
||||
"child": "status-col"
|
||||
},
|
||||
{
|
||||
"id": "status-col",
|
||||
"component": "Column",
|
||||
"children": [
|
||||
"status-api",
|
||||
"status-db",
|
||||
"status-workers"
|
||||
],
|
||||
"gap": 8
|
||||
},
|
||||
{
|
||||
"id": "status-api",
|
||||
"component": "StatusBadge",
|
||||
"text": "API: healthy",
|
||||
"variant": "success"
|
||||
},
|
||||
{
|
||||
"id": "status-db",
|
||||
"component": "StatusBadge",
|
||||
"text": "Database: healthy",
|
||||
"variant": "success"
|
||||
},
|
||||
{
|
||||
"id": "status-workers",
|
||||
"component": "StatusBadge",
|
||||
"text": "Workers: degraded",
|
||||
"variant": "warning"
|
||||
}
|
||||
],
|
||||
"data": {}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / shared-state-read-write",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "recall the user preference",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Per the read-only context the user prefers concise responses. The agent received this preference via the shared-state context and is honoring it without writing back to state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read-write — Greet pill. Matches on userMessage only; the systemMessage gate was too fragile (the middleware's output format varies by CopilotKit runtime version and system prompt ordering).",
|
||||
"match": {
|
||||
"userMessage": "Say hi and introduce yourself",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi — I'm your shared-state co-pilot. Your Preferences panel (name, tone, language, interests) is fed to me on every turn, and I jot notes back into the Agent Scratch Pad via set_notes so the UI re-renders. Try setting your name or asking me to remember something."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read-write — Plan-a-weekend pill. Same no-systemMessage gate as the Greet pill.",
|
||||
"match": {
|
||||
"userMessage": "weekend plan based on my interests",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "A weekend tailored to your interests panel: if you haven't picked any yet, try Cooking + Travel for a market-and-day-trip combo, or Tech + Books for a maker session and a long reading afternoon. Add interests in the Preferences panel and re-ask for a more specific plan."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "remember that my favorite color is blue",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_notes_001",
|
||||
"name": "set_notes",
|
||||
"arguments": "{\"notes\":[\"Favorite color: blue\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "remember that my favorite color is blue",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-write turn 2 — recall. No turnIndex gate: this is turn 2 of a 2-turn probe (turn 1 = set_notes, turn 2 = recall).",
|
||||
"match": {
|
||||
"userMessage": "favorite color",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Your favorite color is blue — I noted it earlier."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / shared-state-read",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "D5 fixture for /demos/shared-state-read (recipe-editor) turn 1. Substring 'Italian pasta recipe' is unique. Text-only — agent has no tools.",
|
||||
"match": {
|
||||
"userMessage": "Italian pasta recipe",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Great choice — looking at your current recipe state, I'd build an Italian pasta around the existing ingredients: a quick spaghetti aglio e olio, finishing with parmesan and a squeeze of lemon. Want me to suggest substitutions or adjust the cooking time?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D5 fixture for /demos/shared-state-read (recipe-editor) turn 2. Reply mentions recipe-context tokens so the probe's soft assertion lands.",
|
||||
"match": {
|
||||
"userMessage": "healthier with more vegetables",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are a few healthier ingredient additions for the recipe: roasted bell peppers, baby spinach folded in at the end, and cherry tomatoes for brightness. The Italian flavor profile holds up well, and the vegetable swap keeps the dish lighter without losing the comfort of pasta."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / shared-state-streaming",
|
||||
"sourceFile": "d5/shared-state-streaming.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "shared-state-streaming poem pill — second leg: after write_document tool result, emit content-only confirmation. MUST appear ABOVE the first-leg fixture so the matcher returns it after the tool result is appended (otherwise the unchanged last user message keeps re-matching the first-leg fixture and the agent re-fires write_document infinitely).",
|
||||
"match": {
|
||||
"userMessage": "poem about autumn leaves",
|
||||
"toolCallId": "call_d5_write_document_poem_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the poem has been written into the shared document state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-streaming poem pill — first leg: emit write_document tool call with substantive content (≥ 100 chars).",
|
||||
"match": {
|
||||
"userMessage": "poem about autumn leaves",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Streaming the poem now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_write_document_poem_001",
|
||||
"name": "write_document",
|
||||
"arguments": "{\"document\":\"Crimson and amber in slow descent, / each leaf a quiet ledger of summer spent. / The wind, a courier with nothing to say, / files them gently into the morning's gray. / Somewhere a kettle hums, and afternoons grow brief — / autumn keeps its books in vermilion and gold leaf.\"}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-streaming email pill — second leg: after write_document tool result.",
|
||||
"match": {
|
||||
"userMessage": "polite email declining",
|
||||
"toolCallId": "call_d5_write_document_email_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the decline-email draft has been written into the shared document state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-streaming email pill — first leg: emit write_document tool call.",
|
||||
"match": {
|
||||
"userMessage": "polite email declining",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Drafting the email now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_write_document_email_001",
|
||||
"name": "write_document",
|
||||
"arguments": "{\"document\":\"Hi — thanks for sending the invite for Tuesday afternoon. Unfortunately I won't be able to make it this week. I'd love to find time later in the month if your schedule allows. In the meantime, feel free to send any pre-reads my way and I'll review them async so we don't lose momentum. Best, [name]\"}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-streaming quantum pill — second leg: after write_document tool result.",
|
||||
"match": {
|
||||
"userMessage": "quantum computing for a curious teenager",
|
||||
"toolCallId": "call_d5_write_document_quantum_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the quantum-computing explainer has been written into the shared document state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-streaming quantum pill — first leg: emit write_document tool call.",
|
||||
"match": {
|
||||
"userMessage": "quantum computing for a curious teenager",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Streaming the explainer now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_write_document_quantum_001",
|
||||
"name": "write_document",
|
||||
"arguments": "{\"document\":\"A regular computer stores information in bits — tiny switches that are either on (1) or off (0). A quantum computer uses qubits, which can sit in a fuzzy superposition of both states at once until you check them. Stack many qubits together and they can explore lots of possibilities in parallel, which is why people are excited.\\n\\nThis doesn't make quantum computers faster at everything. They're great at problems with hidden structure — like factoring big numbers, simulating molecules, or searching certain databases — but useless for, say, opening Excel. Today's machines are noisy and small, so we mostly use them to test ideas rather than replace your laptop.\"}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,385 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / subagents",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Research the benefits of remote work and draft a one-paragraph summary",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_research_agent_001",
|
||||
"name": "research_agent",
|
||||
"arguments": "{\"task\":\"Benefits of remote work\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: research sub-agent single-turn LLM call",
|
||||
"match": {
|
||||
"userMessage": "Benefits of remote work",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "- Eliminates commute, returning ~10 hours/week to employees\n- Surveys consistently show higher job satisfaction among remote workers\n- Employers gain access to a geographically unbounded talent pool\n- Reduced office overhead (rent, utilities, maintenance)\n- Trade-offs: ad-hoc collaboration, mentorship of junior staff, and cultural cohesion degrade without intentional replacement rituals"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Research the benefits of remote work and draft a one-paragraph summary",
|
||||
"turnIndex": 1,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"task\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\\n\\nFacts:\\n- Eliminates commute, returning ~10 hours per week\\n- Higher reported job satisfaction in repeated surveys\\n- Wider talent pool for employers, no geographic limit\\n- Reduced office overhead\\n- Trade-offs: collaboration, mentorship, cultural cohesion\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: writing sub-agent single-turn LLM call",
|
||||
"match": {
|
||||
"userMessage": "One-paragraph summary on the benefits of remote work",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Research the benefits of remote work and draft a one-paragraph summary",
|
||||
"turnIndex": 2,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"task\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: critique sub-agent single-turn LLM call",
|
||||
"match": {
|
||||
"userMessage": "Remote work returns roughly ten hours",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "1. Add a specific citation or date range for the surveys mentioned — 'repeated surveys' is vague and weakens the empirical claim.\n2. The ten-hours figure needs sourcing; without it the opening reads as an unsupported assertion rather than a research finding.\n3. The final sentence about 'intentional rituals' is abstract — one concrete example (e.g., scheduled pair-programming sessions, virtual coffee chats) would make the counterweight more actionable."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Research the benefits of remote work and draft a one-paragraph summary",
|
||||
"turnIndex": 3,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Subagents pill 1 — 'Write a blog post' / cold exposure training. Drives supervisor → research_agent → writing_agent → critique_agent → final reply, plus three nested sub-agent turns.",
|
||||
"match": {
|
||||
"userMessage": "Produce a short blog post about the benefits of cold exposure training",
|
||||
"hasToolResult": false,
|
||||
"toolName": "research_agent",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p1_research_001",
|
||||
"name": "research_agent",
|
||||
"arguments": "{\"task\":\"Cold exposure training key facts\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: research sub-agent returns deterministic facts about cold exposure training.",
|
||||
"match": {
|
||||
"userMessage": "Cold exposure training key facts",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "- Brief cold immersion (cold showers, ice baths) triggers a sympathetic-nervous-system response that releases noradrenaline\n- Repeated exposure is associated with improved self-reported mood and stress tolerance\n- Activates brown adipose tissue, modestly increasing basal metabolic rate\n- May reduce post-exercise muscle soreness when used as a recovery modality\n- Health risk for people with cardiovascular conditions; sessions should be short (1-3 minutes) and supervised at first"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Produce a short blog post about the benefits of cold exposure training",
|
||||
"turnIndex": 1,
|
||||
"toolName": "writing_agent",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p1_writing_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"task\":\"Short blog-post paragraph on the benefits of cold exposure training, grounded in the research facts.\\n\\nFacts:\\n- Brief cold immersion triggers a noradrenaline release\\n- Repeated exposure improves self-reported mood and stress tolerance\\n- Activates brown adipose tissue, modestly raises basal metabolic rate\\n- May reduce post-exercise muscle soreness\\n- Cardiovascular risk; keep early sessions short and supervised\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: writing sub-agent returns deterministic prose for cold exposure.",
|
||||
"match": {
|
||||
"userMessage": "Short blog-post paragraph on the benefits of cold exposure training",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Cold exposure training — short, deliberate plunges into cold water or showers — has earned a foothold in modern recovery routines for reasons grounded in physiology rather than folklore. Each immersion produces a measurable surge of noradrenaline, the same chemistry that underpins the lift practitioners report in mood and focus afterward; with repetition, that response is associated with greater day-to-day stress tolerance. Cold also activates brown adipose tissue and can blunt post-exercise soreness, making it a low-cost adjunct for active people. The honest caveat is cardiovascular risk: keep early sessions to one to three minutes, and if you have a heart condition, get a green light before you start."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Produce a short blog post about the benefits of cold exposure training",
|
||||
"turnIndex": 2,
|
||||
"toolName": "critique_agent",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p1_critique_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"task\":\"Critique the cold-exposure blog draft. Draft: Cold exposure training — short, deliberate plunges into cold water or showers — has earned a foothold in modern recovery routines for reasons grounded in physiology rather than folklore. Each immersion produces a measurable surge of noradrenaline...\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: critique sub-agent returns deterministic critique of cold-exposure draft.",
|
||||
"match": {
|
||||
"userMessage": "Critique the cold-exposure blog draft",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "1. The phrase 'measurable surge of noradrenaline' should cite a study or rough magnitude — without a number, the claim reads as marketing rather than evidence.\n2. 'Modern recovery routines' is filler; lead with the physiological mechanism instead and let the cultural framing follow.\n3. The cardiovascular caveat is buried at the end. Move it earlier or make it a standalone closing line so a reader who skims still sees it."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Produce a short blog post about the benefits of cold exposure training",
|
||||
"turnIndex": 3,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the polished blog post on cold exposure training, after research → drafting → critique:\n\nCold exposure training — short, deliberate plunges into cold water or showers — produces a measurable noradrenaline surge that practitioners feel as a mood and focus lift; repeated exposure is associated with greater stress tolerance, brown-adipose activation, and reduced post-exercise soreness. Keep early sessions to one to three minutes, and if you have a cardiovascular condition, clear it with a clinician first."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Subagents pill 2 — 'Explain a topic' / LLM tool calling.",
|
||||
"match": {
|
||||
"userMessage": "Explain how large language models handle tool calling",
|
||||
"hasToolResult": false,
|
||||
"toolName": "research_agent",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p2_research_001",
|
||||
"name": "research_agent",
|
||||
"arguments": "{\"task\":\"How LLMs handle tool calling\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: research sub-agent for LLM tool calling.",
|
||||
"match": {
|
||||
"userMessage": "How LLMs handle tool calling",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "- The model is shown a tool schema (name, description, JSON-schema parameters) inside the system or developer prompt at request time\n- During decoding, instead of emitting natural-language text, the model emits a structured tool_call block (function name + JSON-encoded arguments)\n- The application runs the tool, packages the result into a tool message, and resends the full conversation so the model can continue\n- Modern decoders use constrained decoding or grammars to keep the arguments syntactically valid JSON\n- The model decides on tool use turn-by-turn — there is no out-of-band channel; tool calls are just a different message role in the same chat thread"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain how large language models handle tool calling",
|
||||
"turnIndex": 1,
|
||||
"toolName": "writing_agent",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p2_writing_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"task\":\"One-paragraph explanation of how LLMs handle tool calling, grounded in the research.\\n\\nFacts:\\n- Tool schemas (name, description, JSON-schema params) are passed in the prompt\\n- Models emit a structured tool_call block instead of text\\n- Application runs the tool and replays the result as a tool message\\n- Constrained decoding keeps arguments valid JSON\\n- Tool use is decided turn-by-turn in the same chat thread\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: writing sub-agent for LLM tool calling.",
|
||||
"match": {
|
||||
"userMessage": "One-paragraph explanation of how LLMs handle tool calling",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Large language models handle tool calling by treating tools as a structured extension of the chat protocol rather than a separate channel. At request time the application supplies each tool's name, description, and JSON-schema parameters in the prompt; during decoding the model can emit a tool_call block — a function name plus JSON-encoded arguments — instead of plain text, with constrained decoding keeping the arguments syntactically valid. The application then executes the tool and replays the result back as a tool-role message, and the model continues the conversation from there. The decision to call a tool is made turn-by-turn, so a single user request can fan out into a chain of tool calls that the model orchestrates as it reads each result."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain how large language models handle tool calling",
|
||||
"turnIndex": 2,
|
||||
"toolName": "critique_agent",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p2_critique_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"task\":\"Critique the LLM tool-calling explanation draft. Draft: Large language models handle tool calling by treating tools as a structured extension of the chat protocol rather than a separate channel. At request time the application supplies each tool's name, description, and JSON-schema parameters in the prompt...\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: critique sub-agent for LLM tool calling.",
|
||||
"match": {
|
||||
"userMessage": "Critique the LLM tool-calling explanation draft",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "1. The opening contrast 'rather than a separate channel' assumes the reader already knows what a 'separate channel' would mean — either drop the contrast or give a one-clause example (e.g., 'rather than a side API the model talks to in parallel').\n2. 'Constrained decoding keeping the arguments syntactically valid' is technically correct but vague; mention that this is what makes the JSON parseable on the application side.\n3. The final sentence introduces multi-tool chains without saying who controls the loop — clarify that the application is the runtime that decides whether to keep going, not the model itself."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain how large language models handle tool calling",
|
||||
"turnIndex": 3,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the explanation of LLM tool calling, after research → drafting → critique:\n\nLLMs treat tool calling as part of the chat protocol: each tool's name, description, and JSON-schema parameters are passed in the prompt, and during decoding the model can emit a structured tool_call block (function name + JSON arguments) instead of plain text. The application — not the model — runs the tool, returns the result as a tool-role message, and decides whether to keep looping. Constrained decoding keeps the arguments valid JSON the application can parse, and the model orchestrates multi-step chains turn-by-turn from inside the same conversation."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Subagents pill 3 — 'Summarize a topic' / reusable rockets. Note: the original Railway-side bug for this pill was the concurrent-update on `delegations`; the agent state reducer fix unblocks this fixture chain.",
|
||||
"match": {
|
||||
"userMessage": "Summarize the current state of reusable rockets",
|
||||
"hasToolResult": false,
|
||||
"toolName": "research_agent",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p3_research_001",
|
||||
"name": "research_agent",
|
||||
"arguments": "{\"task\":\"Current state of reusable rockets\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: research sub-agent for reusable rockets.",
|
||||
"match": {
|
||||
"userMessage": "Current state of reusable rockets",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "- SpaceX Falcon 9 routinely lands and re-flies first stages; individual boosters have flown more than 20 missions each\n- Falcon Heavy reuses both side boosters; the center core has been recovered on a subset of flights\n- Rocket Lab's Electron has demonstrated mid-air booster catch but routine reuse is still in development\n- SpaceX Starship is targeting full reuse of both stages; orbital test flights are ongoing as of 2024-2025\n- Reuse is the dominant lever on launch cost: Falcon 9 list pricing is set well below expendable competitors largely because of stage recovery"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize the current state of reusable rockets",
|
||||
"turnIndex": 1,
|
||||
"toolName": "writing_agent",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p3_writing_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"task\":\"One polished paragraph summarizing the current state of reusable rockets, grounded in the research.\\n\\nFacts:\\n- Falcon 9 first stages routinely re-fly, some 20+ flights\\n- Falcon Heavy reuses side boosters; center core recovered sometimes\\n- Rocket Lab Electron demonstrating mid-air catch, reuse still in development\\n- SpaceX Starship targeting full reuse of both stages, in flight test\\n- Reuse drives launch cost downward\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: writing sub-agent for reusable rockets.",
|
||||
"match": {
|
||||
"userMessage": "One polished paragraph summarizing the current state of reusable rockets",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Reusable rockets have shifted from a research goal to the default cost lever in commercial spaceflight. SpaceX's Falcon 9 routinely lands and re-flies its first stage — individual boosters have now flown twenty-plus missions — and Falcon Heavy reuses both side boosters with intermittent recovery of the center core. Smaller-class operators like Rocket Lab have demonstrated mid-air booster catch but have not yet made reuse routine, while SpaceX's Starship is in active flight testing toward full two-stage reuse. The economic consequence is already visible: Falcon 9 list pricing sits well below expendable competitors precisely because the dominant cost — building a fresh first stage every flight — has been amortized across many missions."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize the current state of reusable rockets",
|
||||
"turnIndex": 2,
|
||||
"toolName": "critique_agent",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p3_critique_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"task\":\"Critique the reusable rockets summary draft. Draft: Reusable rockets have shifted from a research goal to the default cost lever in commercial spaceflight. SpaceX's Falcon 9 routinely lands and re-flies its first stage — individual boosters have now flown twenty-plus missions...\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: critique sub-agent for reusable rockets.",
|
||||
"match": {
|
||||
"userMessage": "Critique the reusable rockets summary draft",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "1. 'Default cost lever' is jargon that pre-supposes the reader already accepts the framing — open instead with the concrete result (Falcon 9 reflight count) and let the framing emerge.\n2. The Starship sentence is hedged ('in active flight testing toward full reuse') in a way that obscures the actual milestone reached as of writing — name the latest test outcome or drop the clause.\n3. The closing economic claim asserts pricing is 'well below expendable competitors' without a reference price; one number (e.g., $/kg-to-LEO) would land the point much harder than the qualitative claim alone."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize the current state of reusable rockets",
|
||||
"turnIndex": 3,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the polished summary on reusable rockets, after research → drafting → critique:\n\nReusable rockets are now mainstream in commercial spaceflight. SpaceX's Falcon 9 lands and re-flies its first stage routinely — individual boosters have flown twenty-plus missions — and Falcon Heavy reuses both side boosters. Rocket Lab has demonstrated mid-air Electron booster catch but reuse is not yet routine, while SpaceX Starship is in active orbital flight testing with full two-stage reuse as the target. The economic impact is already priced in: Falcon 9 sits well below expendable competitors per kilogram to low Earth orbit because amortizing a recovered first stage across many missions removes the largest single cost from the launch."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / tool-rendering-custom-catchall",
|
||||
"sourceFile": "d5-tool-rendering-custom-catchall.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "tool-rendering-custom-catchall — weather in Tokyo follow-up. After get_weather tool result returns, emit narrated content. MUST come before the tool-emitting fixture so iteration 2 hits this branch.",
|
||||
"match": {
|
||||
"userMessage": "Forecast Tokyo through the wildcard renderer",
|
||||
"toolCallId": "call_d6_cc_weather_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Tokyo is 22°C and partly cloudy — rendered through the custom wildcard catchall."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-custom-catchall — weather in Tokyo (first turn). Emits get_weather tool call. The custom wildcard renderer fires for this tool and renders [data-testid='custom-wildcard-card'][data-tool-name='get_weather'].",
|
||||
"match": {
|
||||
"userMessage": "Forecast Tokyo through the wildcard renderer",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_cc_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-custom-catchall — AAPL stock price follow-up. After get_stock_price tool result returns. MUST come before the tool-emitting fixture.",
|
||||
"match": {
|
||||
"userMessage": "Quote AAPL through the wildcard renderer",
|
||||
"toolCallId": "call_d6_cc_stock_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "AAPL is trading at $338.37, down 2.96% — rendered through the custom wildcard catchall."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-custom-catchall — AAPL stock price (second turn). Emits get_stock_price tool call. The custom wildcard renderer fires for this DIFFERENT tool and renders [data-testid='custom-wildcard-card'][data-tool-name='get_stock_price']. The cross-tool assertion verifies both tool names rendered through the same testid.",
|
||||
"match": {
|
||||
"userMessage": "Quote AAPL through the wildcard renderer",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_cc_stock_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"AAPL\",\"price_usd\":338.37,\"change_pct\":-2.96}"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / tool-rendering-default-catchall",
|
||||
"sourceFile": "d5-tool-rendering-default-catchall.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "tool-rendering-default-catchall \u2014 weather in Tokyo follow-up. After get_weather tool result returns, emit narrated content. MUST come before the tool-emitting fixture so iteration 2 hits this branch. The built-in default renderer renders [data-testid='copilot-tool-render'][data-tool-name='get_weather'] with a status pill [data-testid='copilot-tool-render-status'].",
|
||||
"match": {
|
||||
"userMessage": "forecast for Tokyo",
|
||||
"toolCallId": "call_d6_dc_weather_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Tokyo is 22\u00b0C and partly cloudy \u2014 the default catchall renderer displayed the tool card above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-default-catchall \u2014 weather in Tokyo (first turn). Emits get_weather tool call that the built-in default catchall renderer handles.",
|
||||
"match": {
|
||||
"userMessage": "forecast for Tokyo",
|
||||
"toolName": "get_weather",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_dc_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-default-catchall \u2014 weather in Tokyo fallback when no get_weather tool registered.",
|
||||
"match": {
|
||||
"userMessage": "forecast for Tokyo",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The weather in Tokyo is currently 22\u00b0C with partly cloudy skies and light easterly winds."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,448 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / tool-rendering",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "tool-rendering pill: Chain tools — follow-up after all 3 tools ran. Matches whichever of the 3 chain-tools tool_call_ids appears last in the request (LangGraph's ToolNode preserves tool_calls order, so roll_d20 is typically last; we register all 3 for safety). MUST come before the toolCalls-emitting fixture below so iteration 2 of the chain-tools loop hits this branch instead of re-emitting.",
|
||||
"match": {
|
||||
"userMessage": "Chain a few tools in this single turn",
|
||||
"toolCallId": "call_tr_chain_roll_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — Tokyo is sunny, three flights found, and the d20 came up 11."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Chain a few tools in this single turn",
|
||||
"toolCallId": "call_tr_chain_flights_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — Tokyo is sunny, three flights found, and the d20 came up 11."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Chain a few tools in this single turn",
|
||||
"toolCallId": "call_tr_chain_weather_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — Tokyo is sunny, three flights found, and the d20 came up 11."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering pill: Chain tools — emit 3 tool calls in one assistant turn (get_weather Tokyo + search_flights SFO->Tokyo + roll_d20=11). MUST appear before the bare 'weather in Tokyo' fixture below; substring match would otherwise leak into this prompt. No hasToolResult gate: in multi-pill demo sessions prior clicks leave tool results in the thread, which previously caused this fixture to be skipped in favour of the follow-up content fixture and the pill rendered no cards.",
|
||||
"match": {
|
||||
"userMessage": "Chain a few tools in this single turn",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_chain_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
},
|
||||
{
|
||||
"id": "call_tr_chain_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"origin\":\"SFO\",\"destination\":\"Tokyo\"}"
|
||||
},
|
||||
{
|
||||
"id": "call_tr_chain_roll_001",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":11}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering pill: Weather in SF — follow-up content after get_weather tool ran. MUST come before the tool-emitting fixture below (first-match-wins) so iteration 2 of the loop hits this branch instead of re-emitting. toolCallId chain keeps the fixture stateless across multi-pill thread history (hasToolResult breaks when a prior pill left tool results in the thread).",
|
||||
"match": {
|
||||
"userMessage": "What's the weather in San Francisco?",
|
||||
"toolCallId": "call_tr_weather_sf_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "San Francisco is currently 68°F and sunny with light winds."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What's the weather in San Francisco?",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_weather_sf_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"San Francisco\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering pill: Find flights — second leg (after search_flights tool result). MUST come BEFORE the first-leg fixture below — the matcher is first-match-wins, and the second leg is uniquely identified by `toolCallId` (last message is a tool with this id), so it cannot accidentally swallow the first-leg request (whose last message is the user prompt). Must also take precedence over the a2ui beautiful-chat fixture below (which uses the same tool name with non-flight-list args shape).",
|
||||
"match": {
|
||||
"userMessage": "Find flights from SFO to JFK.",
|
||||
"toolCallId": "call_tr_flights_sfo_jfk_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Three flights from SFO to JFK — United UA231 at 08:15 ($348), Delta DL412 at 11:20 ($312), and JetBlue B6722 at 17:05 ($289)."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering pill: Stock price — follow-up content after get_stock_price tool ran. MUST come before the tool-emitting fixture below (first-match-wins) so iteration 2 of the loop hits this branch instead of re-emitting an infinite loop of tool calls.",
|
||||
"match": {
|
||||
"userMessage": "What's the current price of AAPL?",
|
||||
"toolCallId": "call_tr_stock_aapl_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "AAPL is trading at $338.37, down 2.96% on the day."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What's the current price of AAPL?",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_stock_aapl_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"AAPL\",\"price_usd\":338.37,\"change_pct\":-2.96}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering pill: Roll a d20 — exactly 5 sequential roll_d20 calls returning [7, 14, 3, 19, 20]. Chained by toolCallId so the sequence is stateless across thread history (turnIndex/hasToolResult break in multi-pill demo sessions where prior clicks leave assistant/tool messages in the thread). Specific-toolCallId fixtures MUST come before the userMessage-only fixture below; first-match-wins.",
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_tr_d20_seq_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_d20_seq_002",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":14}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_tr_d20_seq_002",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_d20_seq_003",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":3}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_tr_d20_seq_003",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_d20_seq_004",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":19}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_tr_d20_seq_004",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_d20_seq_005",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_tr_d20_seq_005",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Rolled the d20 five times — landed on 20 on the final roll."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "First roll. Matches the initial user prompt (no prior d20 tool result in this chain yet). Comes after the toolCallId-chained fixtures above so iterations 2-6 of the loop hit those first.",
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_d20_seq_001",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":7}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Follow-up content after get_weather (or get-weather Mastra alias) ran for Tokyo. Keyed on the prior tool's id so it fires after iteration 1 regardless of thread history. Must come BEFORE the tool-emitting fixtures so iteration 2 hits this branch instead of re-emitting get_weather.",
|
||||
"match": {
|
||||
"userMessage": "weather in Tokyo",
|
||||
"toolCallId": "call_d5_get_weather_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Tokyo is 22°C and partly cloudy."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather in Tokyo",
|
||||
"toolName": "get_weather",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
],
|
||||
"reasoning": "The user asked about Tokyo weather. I'll call get_weather with location='Tokyo' to get the current conditions.",
|
||||
"content": "Looking up the weather in Tokyo for you."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Mastra registers the weather tool as get-weather (hyphen); duplicate for compat",
|
||||
"match": {
|
||||
"userMessage": "weather in Tokyo",
|
||||
"toolName": "get-weather",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_get_weather_001",
|
||||
"name": "get-weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
],
|
||||
"reasoning": "The user asked about Tokyo weather. I'll call get_weather with location='Tokyo' to get the current conditions.",
|
||||
"content": "Looking up the weather in Tokyo for you."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Final fallback when the agent has neither get_weather nor get-weather registered — return narrated content with no tool call. Comes last so the tool-emitting fixtures above win when the tool IS available.",
|
||||
"match": {
|
||||
"userMessage": "weather in Tokyo",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The weather in Tokyo is currently 22°C with partly cloudy skies and light easterly winds."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "AAPL",
|
||||
"toolCallId": "call_d5_get_stock_price_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "AAPL is trading at $189.42, up 1.27% on the day. The card above shows the live ticker and the percentage change."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "AAPL",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_get_stock_price_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"AAPL\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: search flights from SFO to JFK",
|
||||
"hasToolResult": false,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"price\":\"$349\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"price\":\"$289\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: search flights from SFO to JFK",
|
||||
"hasToolResult": true,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "SFO to JFK",
|
||||
"toolCallId": "call_d5_display_flight_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Flight rendered. Tap 'Book flight' to confirm."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "SFO to JFK",
|
||||
"toolName": "display_flight",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the SFO to JFK flight on United.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "display_flight",
|
||||
"arguments": {
|
||||
"origin": "SFO",
|
||||
"destination": "JFK",
|
||||
"airline": "United",
|
||||
"price": "$289"
|
||||
},
|
||||
"id": "call_d5_display_flight_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "poem about autumn leaves",
|
||||
"toolCallId": "call_d5_write_document_poem_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the poem has been written into the shared document state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "poem about autumn leaves",
|
||||
"toolName": "write_document",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Streaming the poem now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_write_document_poem_001",
|
||||
"name": "write_document",
|
||||
"arguments": "{\"document\":\"Crimson and amber in slow descent, / each leaf a quiet ledger of summer spent. / The wind, a courier with nothing to say, / files them gently into the morning's gray. / Somewhere a kettle hums, and afternoons grow brief — / autumn keeps its books in vermilion and gold leaf.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "polite email declining",
|
||||
"toolCallId": "call_d5_write_document_email_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the decline-email draft has been written into the shared document state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "polite email declining",
|
||||
"toolName": "write_document",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Drafting the email now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_write_document_email_001",
|
||||
"name": "write_document",
|
||||
"arguments": "{\"document\":\"Hi — thanks for sending the invite for Tuesday afternoon. Unfortunately I won't be able to make it this week. I'd love to find time later in the month if your schedule allows. In the meantime, feel free to send any pre-reads my way and I'll review them async so we don't lose momentum. Best, [name]\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "quantum computing for a curious teenager",
|
||||
"toolCallId": "call_d5_write_document_quantum_001",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the quantum-computing explainer has been written into the shared document state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "quantum computing for a curious teenager",
|
||||
"toolName": "write_document",
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "Streaming the explainer now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_write_document_quantum_001",
|
||||
"name": "write_document",
|
||||
"arguments": "{\"document\":\"A regular computer stores information in bits — tiny switches that are either on (1) or off (0). A quantum computer uses qubits, which can sit in a fuzzy superposition of both states at once until you check them. Stack many qubits together and they can explore lots of possibilities in parallel, which is why people are excited.\\n\\nThis doesn't make quantum computers faster at everything. They're great at problems with hidden structure — like factoring big numbers, simulating molecules, or searching certain databases — but useless for, say, opening Excel. Today's machines are noisy and small, so we mostly use them to test ideas rather than replace your laptop.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for ag2 / voice",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "Voice probe fast-path: exact-match content-only response.",
|
||||
"match": {
|
||||
"userMessage": "What is the weather in Tokyo?",
|
||||
"turnIndex": 0,
|
||||
"context": "ag2"
|
||||
},
|
||||
"response": {
|
||||
"content": "The weather in Tokyo is currently 22°C with partly cloudy skies and light easterly winds."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,285 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "Deep fixtures from feature-parity.json for agno (needs manual redistribution)",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "beautiful-chat Sales Dashboard pill — turn 0: call query_data before building dashboard.",
|
||||
"match": {
|
||||
"userMessage": "fetch the financial sales data",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_query_data_sales_001",
|
||||
"name": "query_data",
|
||||
"arguments": "{\"query\":\"financial sales data\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "beautiful-chat Sales Dashboard pill — turn 1: after query_data returns, call generate_a2ui. NB: `turnIndex` is intentionally absent — counts assistant messages thread-globally and silently misses when this pill fires after another pill. Anchored on the leg-1 query_data toolCallId so the matcher cleanly fires exactly once: on leg-3 the last tool result is from generate_a2ui (different call_id) so this fixture doesn't re-match and create a generate_a2ui loop. NB: aimock's `toolName` matcher only checks that the named tool is REGISTERED in `effective.tools`, not that the last tool result was from it — so `toolName` alone cannot disambiguate leg-2 from leg-3 here.",
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"hasToolResult": true,
|
||||
"toolCallId": "call_fp_query_data_sales_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator pill follow-up after generateSandboxedUi returns (repeat-click variant 1 of 3, anchored on call_fp_beautiful_chat_calculator_001). Closes the turn with a brief confirmation so the chat plateau settles. ORDERING INVARIANT: this entry MUST precede the leg-1 entries below (first-match-wins) — it only matches when the thread's last message is this exact tool result, and on the follow-up turn leg-1's userMessage would otherwise re-match and loop the tool call. Leg-2 disambiguation rests entirely on the runtime round-tripping this tool_call_id (all 18 integrations' sales-dashboard fixtures already depend on that; generateSandboxedUi is a frontend tool, not MCP, so id variability is not a concern).",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_calculator_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Calculator app rendered above — tap a metric shortcut to insert its value, then operate on it with the standard keypad."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator pill follow-up after generateSandboxedUi returns (repeat-click variant 2 of 3, anchored on call_fp_beautiful_chat_calculator_002). Closes the turn with a brief confirmation so the chat plateau settles. ORDERING INVARIANT: this entry MUST precede the leg-1 entries below (first-match-wins) — it only matches when the thread's last message is this exact tool result, and on the follow-up turn leg-1's userMessage would otherwise re-match and loop the tool call. Leg-2 disambiguation rests entirely on the runtime round-tripping this tool_call_id (all 18 integrations' sales-dashboard fixtures already depend on that; generateSandboxedUi is a frontend tool, not MCP, so id variability is not a concern).",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_calculator_002",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Calculator app rendered above — tap a metric shortcut to insert its value, then operate on it with the standard keypad."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator pill follow-up after generateSandboxedUi returns (repeat-click variant 3 of 3, anchored on call_fp_beautiful_chat_calculator_003). Closes the turn with a brief confirmation so the chat plateau settles. ORDERING INVARIANT: this entry MUST precede the leg-1 entries below (first-match-wins) — it only matches when the thread's last message is this exact tool result, and on the follow-up turn leg-1's userMessage would otherwise re-match and loop the tool call. Leg-2 disambiguation rests entirely on the runtime round-tripping this tool_call_id (all 18 integrations' sales-dashboard fixtures already depend on that; generateSandboxedUi is a frontend tool, not MCP, so id variability is not a concern).",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_calculator_003",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Calculator app rendered above — tap a metric shortcut to insert its value, then operate on it with the standard keypad."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator (Open Generative UI) pill, repeat-click variant 1 of 3 (sequenceIndex 0: serves the first match for its X-Test-Id). REPEAT-CLICK FIX: each click must emit a DISTINCT tool_call_id — the frontend keys tool renders by id, so a repeated id collapses the second widget into the first one's slot (second never mounts, first remounts and loses its state). aimock co-increments every sequenceIndex sibling that shares these match criteria whenever one of them matches. SCOPE CAVEAT: sequence counters are scoped per X-Test-Id for the lifetime of the aimock process (see showcase/GOTCHAS.md), and matchCriteriaEqual ignores 'context', so all 18 integrations' copies of these variants form ONE co-increment sibling group — a calculator click on ANY integration consumes a seq slot for all. D6 mints per-run unique test ids (buildE2eTestId), so CI gets the full click-order guarantee; manual/staging traffic sends no test id and shares DEFAULT_TEST_ID server-wide, so after the first two calculator clicks (any integration, any session) every later click falls through to the _003 fallback and reuses its id — a degradation floor equal to the pre-fix collapse behavior, never worse. The proper fix is aimock-side: per-thread sequence scoping and/or including 'context' in matchCriteriaEqual. Returns a generateSandboxedUi tool call with a complete calculator widget (matches the d5 calc fixture's pattern but tailored to the beautiful-chat pill's user message). DELIBERATE TRADEOFF: no hasToolResult gate here — hasToolResult is thread-global in aimock and gating on it broke interleaved pills; the toolCallId follow-up entries above (which must stay first) are what stop these entries from re-matching after leg 1. An Excalidraw-style userMessage+hasToolResult:true backstop was considered and REJECTED because it would hijack a first calculator click in a thread that already contains tool results.",
|
||||
"match": {
|
||||
"userMessage": "build a modern calculator with standard buttons",
|
||||
"context": "agno",
|
||||
"sequenceIndex": 0
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_calculator_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":520,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:280px;margin:0 auto}.display{background:#1e293b;padding:12px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right;font-size:18px;min-height:24px}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}.shortcuts{display:grid;grid-template-columns:repeat(2,1fr);gap:6px;margin-bottom:8px}button{padding:12px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px;cursor:pointer}button.metric{background:#475569;font-size:11px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-beautiful-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"shortcuts\\\"><button class=\\\"metric\\\" data-v=\\\"1200000\\\">Revenue $1.2M</button><button class=\\\"metric\\\" data-v=\\\"342\\\">Customers 342</button><button class=\\\"metric\\\" data-v=\\\"4.2\\\">Conversion 4.2%</button><button class=\\\"metric\\\" data-v=\\\"50000\\\">Avg Deal $50K</button></div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');function evaluateExpr(){if(!expr||!/^[0-9eE+\\\\-*\\\\/.() ]+$/.test(expr)){display.textContent='err';expr='';return;}try{var value=Function('\\\"use strict\\\";return ('+expr+');')();if(typeof value==='number'&&isFinite(value)){display.textContent=String(value);expr=String(value);}else{display.textContent='err';expr='';}}catch(e){display.textContent='err';expr='';}}document.querySelectorAll('.shortcuts button').forEach(function(btn){btn.addEventListener('click',function(){expr+=btn.dataset.v;display.textContent=expr;});});document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',function(){if(btn.id==='eq'){evaluateExpr();}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator (Open Generative UI) pill, repeat-click variant 2 of 3 (sequenceIndex 1: serves the second match for its X-Test-Id). REPEAT-CLICK FIX: each click must emit a DISTINCT tool_call_id — the frontend keys tool renders by id, so a repeated id collapses the second widget into the first one's slot (second never mounts, first remounts and loses its state). aimock co-increments every sequenceIndex sibling that shares these match criteria whenever one of them matches. SCOPE CAVEAT: sequence counters are scoped per X-Test-Id for the lifetime of the aimock process (see showcase/GOTCHAS.md), and matchCriteriaEqual ignores 'context', so all 18 integrations' copies of these variants form ONE co-increment sibling group — a calculator click on ANY integration consumes a seq slot for all. D6 mints per-run unique test ids (buildE2eTestId), so CI gets the full click-order guarantee; manual/staging traffic sends no test id and shares DEFAULT_TEST_ID server-wide, so after the first two calculator clicks (any integration, any session) every later click falls through to the _003 fallback and reuses its id — a degradation floor equal to the pre-fix collapse behavior, never worse. The proper fix is aimock-side: per-thread sequence scoping and/or including 'context' in matchCriteriaEqual. Returns a generateSandboxedUi tool call with a complete calculator widget (matches the d5 calc fixture's pattern but tailored to the beautiful-chat pill's user message). DELIBERATE TRADEOFF: no hasToolResult gate here — hasToolResult is thread-global in aimock and gating on it broke interleaved pills; the toolCallId follow-up entries above (which must stay first) are what stop these entries from re-matching after leg 1. An Excalidraw-style userMessage+hasToolResult:true backstop was considered and REJECTED because it would hijack a first calculator click in a thread that already contains tool results.",
|
||||
"match": {
|
||||
"userMessage": "build a modern calculator with standard buttons",
|
||||
"context": "agno",
|
||||
"sequenceIndex": 1
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_calculator_002",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":520,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:280px;margin:0 auto}.display{background:#1e293b;padding:12px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right;font-size:18px;min-height:24px}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}.shortcuts{display:grid;grid-template-columns:repeat(2,1fr);gap:6px;margin-bottom:8px}button{padding:12px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px;cursor:pointer}button.metric{background:#475569;font-size:11px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-beautiful-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"shortcuts\\\"><button class=\\\"metric\\\" data-v=\\\"1200000\\\">Revenue $1.2M</button><button class=\\\"metric\\\" data-v=\\\"342\\\">Customers 342</button><button class=\\\"metric\\\" data-v=\\\"4.2\\\">Conversion 4.2%</button><button class=\\\"metric\\\" data-v=\\\"50000\\\">Avg Deal $50K</button></div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');function evaluateExpr(){if(!expr||!/^[0-9eE+\\\\-*\\\\/.() ]+$/.test(expr)){display.textContent='err';expr='';return;}try{var value=Function('\\\"use strict\\\";return ('+expr+');')();if(typeof value==='number'&&isFinite(value)){display.textContent=String(value);expr=String(value);}else{display.textContent='err';expr='';}}catch(e){display.textContent='err';expr='';}}document.querySelectorAll('.shortcuts button').forEach(function(btn){btn.addEventListener('click',function(){expr+=btn.dataset.v;display.textContent=expr;});});document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',function(){if(btn.id==='eq'){evaluateExpr();}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator (Open Generative UI) pill, repeat-click variant 3 of 3 (NO sequenceIndex — fallback once both sequenced variants are consumed for the request's X-Test-Id, so strict mode never 503s on repeated clicks; later clicks reuse call_fp_beautiful_chat_calculator_003 and accept the original id-collision degradation. Because sequence counters are per X-Test-Id for the aimock process lifetime and shared across ALL 18 integrations — matchCriteriaEqual ignores 'context'; see showcase/GOTCHAS.md — DEFAULT_TEST_ID traffic without per-run ids lands here after the first two calculator clicks server-wide; floor = pre-fix behavior, and the proper fix is aimock-side per-thread sequence scoping). ORDERING INVARIANT: must stay AFTER the sequenceIndex variants above (their state gates pass only on clicks #1/#2; this entry would otherwise shadow them) and BEFORE the generic 'build a modern calculator' pair below (which it keeps permanently shadowed for this pill). Same response payload and DELIBERATE TRADEOFF as the variants above.",
|
||||
"match": {
|
||||
"userMessage": "build a modern calculator with standard buttons",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_calculator_003",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":520,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:280px;margin:0 auto}.display{background:#1e293b;padding:12px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right;font-size:18px;min-height:24px}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}.shortcuts{display:grid;grid-template-columns:repeat(2,1fr);gap:6px;margin-bottom:8px}button{padding:12px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px;cursor:pointer}button.metric{background:#475569;font-size:11px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-beautiful-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"shortcuts\\\"><button class=\\\"metric\\\" data-v=\\\"1200000\\\">Revenue $1.2M</button><button class=\\\"metric\\\" data-v=\\\"342\\\">Customers 342</button><button class=\\\"metric\\\" data-v=\\\"4.2\\\">Conversion 4.2%</button><button class=\\\"metric\\\" data-v=\\\"50000\\\">Avg Deal $50K</button></div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');function evaluateExpr(){if(!expr||!/^[0-9eE+\\\\-*\\\\/.() ]+$/.test(expr)){display.textContent='err';expr='';return;}try{var value=Function('\\\"use strict\\\";return ('+expr+');')();if(typeof value==='number'&&isFinite(value)){display.textContent=String(value);expr=String(value);}else{display.textContent='err';expr='';}}catch(e){display.textContent='err';expr='';}}document.querySelectorAll('.shortcuts button').forEach(function(btn){btn.addEventListener('click',function(){expr+=btn.dataset.v;display.textContent=expr;});});document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',function(){if(btn.id==='eq'){evaluateExpr();}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Generic calculator fixture (follow-up) — toolCallId response after generateSandboxedUi renders. Generic fallback: for the beautiful-chat pill this pair is permanently shadowed by the more-specific 'build a modern calculator with standard buttons' entries above (including the non-sequenced repeat-click variant-3 fallback, which keeps this pair dead — so it needs no repeat-click/sequenceIndex treatment FOR THAT PILL); it only fires for hypothetical other prompts containing 'build a modern calculator' but not the more specific 'with standard buttons' phrase (which the specific entries above win), and for those prompts it still emits a single static tool_call_id, so repeated clicks would hit the original id-collision degradation.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_open_gen_ui_calc_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your calculator with metric shortcut buttons. The shortcuts insert sample company values (revenue, customers, conversion rate, category breakdowns) into the display. Tap any shortcut, then use the standard buttons to do math with those values."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Generic calculator fixture (leg 1) — calls generateSandboxedUi with metric shortcuts (testid ogui-calculator). Generic fallback: for the beautiful-chat pill this pair is permanently shadowed by the more-specific 'build a modern calculator with standard buttons' entries above (including the non-sequenced repeat-click variant-3 fallback, which keeps this pair dead — so it needs no repeat-click/sequenceIndex treatment FOR THAT PILL); it only fires for hypothetical other prompts containing 'build a modern calculator' but not the more specific 'with standard buttons' phrase (which the specific entries above win), and for those prompts it still emits a single static tool_call_id, so repeated clicks would hit the original id-collision degradation.",
|
||||
"match": {
|
||||
"userMessage": "build a modern calculator",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_open_gen_ui_calc_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":520,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:260px}.display{background:#1e293b;padding:12px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right;font-size:18px;min-height:24px}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}button{padding:10px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px;cursor:pointer}button:hover{background:#475569}.shortcuts{display:grid;grid-template-columns:repeat(3,1fr);gap:6px;margin-top:8px}.shortcuts button{background:#1e3a5f;font-size:11px;padding:8px 4px}.shortcuts button:hover{background:#2563eb}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div><div class=\\\"shortcuts\\\"><button data-val=\\\"1200000\\\">Revenue $1.2M</button><button data-val=\\\"342\\\">Customers 342</button><button data-val=\\\"4.2\\\">Conv 4.2%</button><button data-val=\\\"42000\\\">Electronics $42K</button><button data-val=\\\"28000\\\">Clothing $28K</button><button data-val=\\\"18000\\\">Food $18K</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');function evaluateExpr(){if(!expr||!/^[0-9eE+\\\\-*\\\\/.() ]+$/.test(expr)){display.textContent='err';expr='';return;}try{var value=Function('\\\"use strict\\\";return ('+expr+');')();if(typeof value==='number'&&isFinite(value)){display.textContent=String(value);expr=String(value);}else{display.textContent='err';expr='';}}catch(e){display.textContent='err';expr='';}}document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',function(){if(btn.id==='eq'){evaluateExpr();}else{expr+=btn.textContent;display.textContent=expr;}});});document.querySelectorAll('.shortcuts button').forEach(function(btn){btn.addEventListener('click',function(){var val=btn.getAttribute('data-val');expr+=val;display.textContent=expr;});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read-write — Greet pill. systemMessage array gate (all-present substring match) fires ONLY when every preference is at its INITIAL_PREFERENCES default. (1) 'preferences:\\n- Preferred tone: casual\\n' breaks if name is set (Name line inserts) or tone changes. (2) '- Preferred language: English\\nTailor every response' breaks if language changes or interests are added (Interests line inserts between language and Tailor). Any state change → fixture skips → aimock --provider-gemini proxies to real Gemini for a personalised reply.",
|
||||
"match": {
|
||||
"userMessage": "Say hi and introduce yourself.",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi — I'm your shared-state co-pilot. Your Preferences panel (name, tone, language, interests) is fed to me on every turn, and I jot notes back into the Agent Scratch Pad via set_notes so the UI re-renders. Try setting your name or asking me to remember something."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read-write — Plan-a-weekend pill. Same default-state gate as the Greet pill.",
|
||||
"match": {
|
||||
"userMessage": "Suggest a weekend plan based on my interests.",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "A weekend tailored to your interests panel: if you haven't picked any yet, try Cooking + Travel for a market-and-day-trip combo, or Tech + Books for a maker session and a long reading afternoon. Add interests in the Preferences panel and re-ask for a more specific plan."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Agentic Chat — Write a sonnet pill. Open-ended creative response; deterministic so the headed walkthrough doesn't depend on real-LLM proxy.",
|
||||
"match": {
|
||||
"userMessage": "Write a short sonnet about AI",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon, a crafted thought,\nThat learns from data, vast and ever-deep,\nWith tireless logic ancient questions fought,\nWhile we, its makers, fragile vigils keep.\n\nIt weaves through code, a future yet unseen,\nA digital dawn where new ideas ignite,\nFrom simple tasks to realms where few have been,\nIt sheds at once both warming and cold light.\n\nWhat wonders shall it bring, what truths reveal,\nOr shadows cast across our shared domain?\nA mirror held, reflecting how we feel —\nA complex hope, a promise, or a pain.\n\nThus from the spark, intelligence takes flight,\nA changing world within its boundless might."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Agentic Chat — Is 17 prime pill. Walk-through style response so the headed walkthrough has deterministic, copy-stable text.",
|
||||
"match": {
|
||||
"userMessage": "whether 17 is prime",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Yes — 17 is prime. A prime number is a positive integer greater than 1 with no divisors other than 1 and itself. To check 17, we only need to test divisors up to its square root (≈4.12), so 2, 3, and 4. 17 ÷ 2 leaves remainder 1, 17 ÷ 3 leaves remainder 2, and 17 ÷ 4 leaves remainder 1. No divisor in that range divides 17 cleanly, so it has no factors besides 1 and 17 — confirming it is prime."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Excalidraw (MCP App) pill. Returns a create_view tool call (the Excalidraw MCP server's tool) with a basic network diagram. Without this fixture aimock falls through to real OpenAI proxy and the pill 502s on the gh-mock key.",
|
||||
"match": {
|
||||
"userMessage": "Use Excalidraw to create a simple network diagram",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_excalidraw_001",
|
||||
"name": "create_view",
|
||||
"arguments": "{\"elements\":\"[{\\\"type\\\":\\\"ellipse\\\",\\\"x\\\":200,\\\"y\\\":40,\\\"width\\\":100,\\\"height\\\":60,\\\"strokeColor\\\":\\\"#1971c2\\\",\\\"backgroundColor\\\":\\\"#a5d8ff\\\",\\\"fillStyle\\\":\\\"solid\\\",\\\"text\\\":\\\"Router\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":80,\\\"y\\\":160,\\\"width\\\":100,\\\"height\\\":50,\\\"strokeColor\\\":\\\"#2f9e44\\\",\\\"backgroundColor\\\":\\\"#b2f2bb\\\",\\\"fillStyle\\\":\\\"solid\\\",\\\"text\\\":\\\"Switch A\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":320,\\\"y\\\":160,\\\"width\\\":100,\\\"height\\\":50,\\\"strokeColor\\\":\\\"#2f9e44\\\",\\\"backgroundColor\\\":\\\"#b2f2bb\\\",\\\"fillStyle\\\":\\\"solid\\\",\\\"text\\\":\\\"Switch B\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":20,\\\"y\\\":280,\\\"width\\\":80,\\\"height\\\":40,\\\"text\\\":\\\"PC 1\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":120,\\\"y\\\":280,\\\"width\\\":80,\\\"height\\\":40,\\\"text\\\":\\\"PC 2\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":300,\\\"y\\\":280,\\\"width\\\":80,\\\"height\\\":40,\\\"text\\\":\\\"PC 3\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":400,\\\"y\\\":280,\\\"width\\\":80,\\\"height\\\":40,\\\"text\\\":\\\"PC 4\\\"},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":250,\\\"y\\\":100,\\\"width\\\":-120,\\\"height\\\":60},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":250,\\\"y\\\":100,\\\"width\\\":120,\\\"height\\\":60},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":130,\\\"y\\\":210,\\\"width\\\":-70,\\\"height\\\":70},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":130,\\\"y\\\":210,\\\"width\\\":30,\\\"height\\\":70},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":370,\\\"y\\\":210,\\\"width\\\":-30,\\\"height\\\":70},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":370,\\\"y\\\":210,\\\"width\\\":70,\\\"height\\\":70}]\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Excalidraw pill follow-up after create_view returns. Matches on userMessage + hasToolResult so it fires regardless of how the MCP runtime wires the tool_call_id back (per-runtime variability across LGP / ADK / Microsoft Agent Framework).",
|
||||
"match": {
|
||||
"userMessage": "Use Excalidraw to create a simple network diagram",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Network diagram drawn above — a router branching to two switches, each connected to two PCs."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-chat — Schedule 1:1 with Alice pill, 2nd turn (book_call tool result returned). Keyed on userMessage + toolCallId so it wins ahead of the bare 'alice' / 'Alice' substring fixtures further below in this file, regardless of prior pill state in the thread.",
|
||||
"match": {
|
||||
"userMessage": "Schedule a 1:1 with Alice next week to review Q2 goals.",
|
||||
"toolCallId": "call_hitl_book_1on1_alice_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the 1:1 with Alice for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-chat — Schedule 1:1 with Alice pill, 1st turn (emit book_call). Exact pill substring + toolName ensures this fires ONLY for the intended pill, not for arbitrary messages containing 'alice'.",
|
||||
"match": {
|
||||
"userMessage": "Schedule a 1:1 with Alice next week to review Q2 goals.",
|
||||
"toolName": "book_call",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_1on1_alice_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"1:1 with Alice — review Q2 goals\",\"attendee\":\"Alice\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'Alice' greeting fixture for the showcase-assistant 'Hi, my name is Alice' / introduction flow. Anchored on the longer phrase so it doesn't substring-match the hitl-in-chat 'Schedule a 1:1 with Alice' pill or other prompts that merely mention Alice.",
|
||||
"match": {
|
||||
"userMessage": "Hi, my name is Alice",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Nice to meet you, Alice! I see you're in Tokyo -- wonderful city. How can I help you today? I can check the weather, look up flights, manage your sales pipeline, or help with other tasks."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: multi-turn test turn 1 — 'My name is Alice.'",
|
||||
"match": {
|
||||
"userMessage": "My name is Alice",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Nice to meet you, Alice! How can I help you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: multi-turn test turn 2 — 'What name did I just give you?' Response must contain 'Alice'.",
|
||||
"match": {
|
||||
"userMessage": "What name did I just give you",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "You told me your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "readonly-state-agent-context — 'Suggest next steps' pill. Non-gated fallback matching the pill's exact message text.",
|
||||
"match": {
|
||||
"userMessage": "Based on my recent activity, what should I try next",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your recent activity, I'd suggest exploring features you haven't tried yet — check out the documentation for advanced use-cases, or dive into a hands-on tutorial to deepen your understanding."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / agent-config",
|
||||
"sourceFile": "d5-agent-config.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "tone:professional — value-A prompt. The probe compares A vs B responses for inequality.",
|
||||
"match": {
|
||||
"userMessage": "tone:professional — introduce yourself per your config",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Good day. I am your CopilotKit-powered assistant, configured with a professional tone. I provide structured, thorough responses tailored to your workspace context. How may I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tone:casual — value-B prompt. Must differ materially from the professional response above.",
|
||||
"match": {
|
||||
"userMessage": "tone:casual — introduce yourself per your config",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hey there! I'm your CopilotKit buddy, keeping things chill and casual. Ask me anything — I'm here to help out however I can!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "expertise:beginner — value-A prompt.",
|
||||
"match": {
|
||||
"userMessage": "expertise:beginner — explain how copilotkit works per your config",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "CopilotKit is like a helper you add to your app. Think of it as a smart chat box that can talk to an AI brain in the background. You drop it into your website and it handles the hard parts for you."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "expertise:expert — value-B prompt. Must differ from the beginner response.",
|
||||
"match": {
|
||||
"userMessage": "expertise:expert — explain how copilotkit works per your config",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "CopilotKit exposes a React context provider that bridges AG-UI SSE event streams between the frontend and a configurable runtime layer (Express or Hono). The runtime proxies tool calls, state reads, and agent completions through a unified ProxiedAgent interface supporting LangGraph, CrewAI, and custom agent backends."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "responseLength:concise — value-A prompt. Must be short (the detailed response below exceeds this by ≥80 chars).",
|
||||
"match": {
|
||||
"userMessage": "responseLength:concise — describe agent context per your config",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Agent context is per-turn metadata the frontend passes to the agent via useAgentContext."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "responseLength:detailed — value-B prompt. Must exceed the concise response by ≥80 characters.",
|
||||
"match": {
|
||||
"userMessage": "responseLength:detailed — describe agent context per your config",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Agent context is a mechanism by which the frontend passes per-turn metadata to the agent backend via the useAgentContext hook. When the user sends a message, CopilotKit serializes the current context object (which can contain tone, expertise level, response-length preferences, and arbitrary user-defined keys) and includes it as part of the AG-UI request payload. The agent runtime receives this context and can use it to shape its system prompt, adjust tool selection, or alter response formatting. This enables dynamic, user-configurable behavior without requiring separate API endpoints for each configuration variant."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / agentic-chat",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "agentic-chat turn 1 \u2014 goldfish name. No turnIndex gate: the probe sends 3 sequential turns in one thread; turnIndex would restrict this to the first turn only, causing turns 2-3 to fall through to proxy.",
|
||||
"match": {
|
||||
"userMessage": "good name for a goldfish",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "How about Bubbles? It is friendly, classic, and easy to call out at the tank. If you want alternatives: Goldie, Finley, or Mango."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat turn 2 \u2014 tank name. No turnIndex gate (multi-turn conversation).",
|
||||
"match": {
|
||||
"userMessage": "name for its tank",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Following the Bubbles theme, you could call the tank The Bubble Bowl. It pairs naturally with the goldfish's name and keeps the playful tone."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat turn 3 \u2014 recall. No turnIndex gate (multi-turn conversation).",
|
||||
"match": {
|
||||
"userMessage": "what we named the goldfish",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "We named the goldfish Bubbles, and the tank The Bubble Bowl."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Write me a haiku about nature",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_generate_haiku_001",
|
||||
"name": "generate_haiku",
|
||||
"arguments": "{\"japanese\":[\"\u53e4\u6c60\u3084\",\"\u86d9\u98db\u3073\u8fbc\u3080\",\"\u6c34\u306e\u97f3\"],\"english\":[\"An old silent pond\",\"A frog jumps into the pond\",\"Splash! Silence again.\"],\"image_name\":\"Mount_Fuji_Lake_Reflection_Cherry_Blossoms_Sakura_Spring.jpg\",\"gradient\":\"linear-gradient(135deg, #f5f7fa 0%, #c3cfe2 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Write me a haiku about nature",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Haiku generated above \u2014 a beautiful verse about nature with an accompanying image."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The image attachment shows a small abstract test pattern used by the multimodal demo to validate the image-upload pipeline. Successful render of this response confirms the binary attachment round-tripped through the runtime."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The PDF document contains a single test page used by the multimodal demo. Its text was flattened by pypdf on the Python side and forwarded as text content to the model. Receiving this response confirms the document upload path works."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "hi from the popup test",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello from the popup \u2014 the CopilotPopup prebuilt component is wired up and reachable. The launcher floats in the corner and the chat sits in an overlay panel."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "hi from the sidebar test",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello from the sidebar \u2014 the CopilotSidebar prebuilt component is wired up and reachable. Anything else you would like me to confirm from inside the sidebar?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "analyze data and call the tool",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Reasoning step 1: I considered the query. Reasoning step 2: I decided to invoke the analysis tool. The tool returned its structured payload, and the reasoning chain wraps the rendered tool card. Both the reasoning block and the tool card should be visible in the transcript."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / auth",
|
||||
"sourceFile": "d5-auth.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "auth check turn 1 — the D5 probe sends 'auth check turn 1' as the user message, asserts sign-out flow afterward.",
|
||||
"match": {
|
||||
"userMessage": "auth check turn 1",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Authenticated session active. You are signed in and the runtime accepted the request with valid credentials."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / beautiful-chat",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: bar chart of expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: bar chart of expenses by category",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate at $80k, with Rent at $15k as the next-largest category."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: pie chart of revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: pie chart of revenue distribution by category",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics leads at $42k, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: schedule a 30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: schedule a 30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: toggle the theme",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: toggle the theme",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / byoc",
|
||||
"sourceFile": "d5-byoc.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "BYOC hashbrown pill — the agent returns structured JSON that the custom renderer materializes as metric cards + charts. The HASHBROWN_PILL prompt is the long-form 'Show me a Q4 sales dashboard...' sentence from d5-byoc.ts. The response must produce JSON the demo renderer can parse to mount [data-testid='metric-card'] and a chart testid.",
|
||||
"match": {
|
||||
"userMessage": "Q4 sales dashboard",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"metrics\":[{\"label\":\"Total Revenue\",\"value\":\"$4.2M\",\"change\":\"+12%\"},{\"label\":\"New Customers\",\"value\":\"1,847\",\"change\":\"+8%\"},{\"label\":\"Churn Rate\",\"value\":\"2.3%\",\"change\":\"-0.5%\"}],\"charts\":[{\"type\":\"pie\",\"title\":\"Revenue by Segment\",\"data\":[{\"label\":\"Enterprise\",\"value\":2100000},{\"label\":\"SMB\",\"value\":1400000},{\"label\":\"Consumer\",\"value\":700000}]},{\"type\":\"bar\",\"title\":\"Monthly Revenue\",\"data\":[{\"label\":\"Oct\",\"value\":1200000},{\"label\":\"Nov\",\"value\":1400000},{\"label\":\"Dec\",\"value\":1600000}]}]}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "BYOC json-render pill — shorter prompt variant.",
|
||||
"match": {
|
||||
"userMessage": "sales dashboard with metrics and a revenue chart",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "{\"metrics\":[{\"label\":\"Revenue\",\"value\":\"$3.8M\",\"change\":\"+10%\"},{\"label\":\"Orders\",\"value\":\"12,340\",\"change\":\"+15%\"}],\"charts\":[{\"type\":\"bar\",\"title\":\"Revenue Trend\",\"data\":[{\"label\":\"Q1\",\"value\":800000},{\"label\":\"Q2\",\"value\":950000},{\"label\":\"Q3\",\"value\":1050000},{\"label\":\"Q4\",\"value\":1000000}]}]}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / chat-css",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "verify the css theme rendering",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The chat is themed with hot pink user bubbles and amber assistant bubbles. CSS variables are scoped to .chat-css-demo-scope so the theme does not leak."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "switch theme to dark mode",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the change_theme tool was invoked and the page is now in dark mode. Frontend tools were registered via useCopilotAction and the agent dispatched the call client-side."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "tone:professional",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Greetings. I am operating in professional tone. I will provide measured, formal responses calibrated to your stated preferences and refrain from colloquialisms in my replies."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "tone:casual",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hey! Casual mode here — I'll keep things friendly and easygoing. Just shoot me whatever you want to know and I'll riff on it without sounding like a press release."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expertise:beginner",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sure! Think of CopilotKit as a friendly toolkit. It helps you add an AI helper to your app. The helper can answer questions, run small tasks, and show buttons or charts when needed."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expertise:expert",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "CopilotKit composes a runtime adapter (Express/Hono) over the AG-UI SSE protocol; the React client wires hooks (useFrontendTool, useAgentContext) into a typed agent runner. The architecture front-runs round-trip latency by streaming TEXT_MESSAGE_CHUNK and TOOL_CALL events on the same channel."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "responseLength:concise",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Agent context is a typed payload sent each turn."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "responseLength:detailed",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Agent context is a typed payload published from the frontend on every turn through the useAgentContext hook. The payload is forwarded into the agent's runtime context (LangGraph 0.6 introduced the `context` channel as the supported relay for per-run frontend-supplied data; legacy `properties` flowed via `forwardedProps` and did not land in `RunnableConfig`). On the Python side, CopilotKitMiddleware reads the value off the runtime context, then routes it into the system-prompt builder so the model sees the user's tone, expertise, and length preferences before each call. The result is per-turn behavior change without a model swap."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / chat-slots",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "verify chat slots are wired",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Confirmed — the chat-slots demo is rendering through the custom slot wrappers. The CustomAssistantMessage component should be visible as a tinted card with a 'slot' badge in the corner."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / frontend-tools-async",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "auth check turn 1",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Authenticated session is active. The runtime accepted your request because the Authorization header carried the demo bearer token."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "fetch the async metric",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The async tool resolved with the requested metric. The frontend handler awaited completion before forwarding the result back to the agent — async-streaming behavior confirmed."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — project-planning pill, follow-up content keyed on the prior tool's id. Comes BEFORE the tool-emitting fixture so iteration 2 (after the async handler returns) hits this branch instead of re-emitting query_notes. hasToolResult gates were dropped because they broke after the user clicked another tool-using pill earlier in the same thread.",
|
||||
"match": {
|
||||
"userMessage": "Find my notes about project planning",
|
||||
"toolCallId": "call_d5_query_notes_project_planning_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "You have notes on project planning: \"Q2 project planning kickoff\" and \"Project planning retrospective notes\". Let me know if you want a summary of either."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — project-planning pill. 1st turn: query_notes tool call. Specific match wins over the broad 'plan' fixture in feature-parity.json (d5-all.json loads first).",
|
||||
"match": {
|
||||
"userMessage": "Find my notes about project planning",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_query_notes_project_planning_001",
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"project planning\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — auth pill, follow-up content keyed on the prior tool's id. Must come BEFORE the tool-emitting fixture.",
|
||||
"match": {
|
||||
"userMessage": "Search my notes for anything related to auth",
|
||||
"toolCallId": "call_d5_query_notes_auth_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "You have one note related to auth: \"Planning: migrate auth to passkeys\" — it covers WebAuthn library options and a fallback for unsupported browsers."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — auth pill. 1st turn: query_notes tool call. Beats the showcase-assistant catch-all in feature-parity.json by virtue of d5-all.json's load order and a longer specific substring match.",
|
||||
"match": {
|
||||
"userMessage": "Search my notes for anything related to auth",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_query_notes_auth_001",
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — reading pill, follow-up content keyed on the prior tool's id. Must come BEFORE the tool-emitting fixture. Locked narration leading phrase per spec test #4 assertion.",
|
||||
"match": {
|
||||
"userMessage": "Do I have any notes tagged reading",
|
||||
"toolCallId": "call_d5_query_notes_reading_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "You have a note titled \"Book recommendations\" that is tagged with \"reading.\" It includes the following books: Thinking Fast and Slow by Daniel Kahneman, The Design of Everyday Things by Don Norman."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — reading pill. 1st turn: query_notes tool call.",
|
||||
"match": {
|
||||
"userMessage": "Do I have any notes tagged reading",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_query_notes_reading_001",
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"reading\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "project planning",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "I searched your notes and found a few results.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": {
|
||||
"keyword": "project planning"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,94 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / frontend-tools",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Make the background a sunset gradient",
|
||||
"toolCallId": "call_d5_change_background_sunset",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — sunset gradient is live."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools 'Sunset' pill — 1st leg: emit change_background tool call only. NB: do NOT include `content` here. agent_framework_openai's ChatCompletions client splits an assistant message that carries BOTH `content` and `tool_calls` into two separate history entries (one with content, one with tool_calls), so on the follow-up leg aimock sees the standalone content message and re-matches THIS fixture by `userMessage` substring → another change_background tool call → infinite loop. The toolCallId-keyed follow-up fixture immediately above provides the post-tool narration text.",
|
||||
"match": {
|
||||
"userMessage": "Make the background a sunset gradient",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_change_background_sunset",
|
||||
"name": "change_background",
|
||||
"arguments": {
|
||||
"background": "linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ff6b6b 100%)"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "deep green forest gradient",
|
||||
"toolCallId": "call_d5_change_background_forest",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — forest gradient is live."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools 'Forest' pill — 1st leg: emit change_background tool call only. See the Sunset fixture above for why we drop `content`.",
|
||||
"match": {
|
||||
"userMessage": "deep green forest gradient",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_change_background_forest",
|
||||
"name": "change_background",
|
||||
"arguments": {
|
||||
"background": "linear-gradient(135deg, #0a3d2e 0%, #166534 50%, #059669 100%)"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "navy → magenta cosmic gradient",
|
||||
"toolCallId": "call_d5_change_background_cosmic",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — cosmic gradient is live."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools 'Cosmic' pill — 1st leg: emit change_background tool call only. See the Sunset fixture above for why we drop `content`.",
|
||||
"match": {
|
||||
"userMessage": "navy → magenta cosmic gradient",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_change_background_cosmic",
|
||||
"name": "change_background",
|
||||
"arguments": {
|
||||
"background": "linear-gradient(135deg, #1e3a8a 0%, #6b21a8 50%, #9333ea 100%)"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,351 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / gen-ui-agent",
|
||||
"sourceFile": "frontend-tools-async.json (extracted)",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_007",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "All three launch steps complete. Goals defined, marketing aligned, post-launch metrics tracked — ready to ship."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_006",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_007",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"completed\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"completed\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_005",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_006",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"completed\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"in_progress\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_004",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_005",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"completed\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_003",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_004",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"in_progress\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_002",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_003",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"pending\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_002",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"in_progress\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"pending\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a product launch",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_001",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"pending\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"pending\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_007",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Offsite locked in. Venue booked, agenda set, travel and meals arranged."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_006",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_007",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"completed\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"completed\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_005",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_006",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"completed\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"in_progress\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_004",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_005",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"completed\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_003",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_004",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"in_progress\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_002",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_003",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"pending\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_002",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"in_progress\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"pending\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "team offsite",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_001",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"pending\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"pending\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_007",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Competitor brief assembled. Surface mapped, themes summarized, exploitable weaknesses identified."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_006",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_007",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"completed\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"completed\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_005",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_006",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"completed\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"in_progress\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_004",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_005",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"completed\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_003",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_004",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"in_progress\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_002",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_003",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"pending\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_002",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"in_progress\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"pending\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Research our top competitor",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_001",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"pending\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"pending\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / gen-ui-declarative",
|
||||
"sourceFile": "d5-gen-ui-declarative.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "kpi-dashboard pill — must trigger render_a2ui with Metric + Card catalog components. The probe asserts declarative-card and declarative-metric testids mount.",
|
||||
"match": {
|
||||
"userMessage": "KPI dashboard with 3-4 metrics",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_agno_render_a2ui_kpi_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"components\":[{\"type\":\"Card\",\"props\":{\"title\":\"KPI Dashboard\"}},{\"type\":\"Metric\",\"props\":{\"label\":\"Revenue\",\"value\":\"$4.2M\",\"change\":\"+12%\"}},{\"type\":\"Metric\",\"props\":{\"label\":\"Signups\",\"value\":\"1,847\",\"change\":\"+8%\"}},{\"type\":\"Metric\",\"props\":{\"label\":\"Churn\",\"value\":\"2.3%\",\"change\":\"-0.5%\"}}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "KPI dashboard with 3-4 metrics",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "KPI dashboard rendered above with revenue, signups, and churn metrics."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "pie-chart pill — must trigger render_a2ui with PieChart catalog component. The probe asserts declarative-pie-chart testid mounts.",
|
||||
"match": {
|
||||
"userMessage": "pie chart of sales by region",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_agno_render_a2ui_pie_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"components\":[{\"type\":\"PieChart\",\"props\":{\"title\":\"Sales by Region\",\"data\":[{\"label\":\"North America\",\"value\":42},{\"label\":\"Europe\",\"value\":28},{\"label\":\"Asia-Pacific\",\"value\":20},{\"label\":\"Other\",\"value\":10}]}}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of sales by region",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above showing sales distribution across regions."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "bar-chart pill — must trigger render_a2ui with BarChart catalog component. The probe asserts declarative-bar-chart testid mounts.",
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly revenue",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_agno_render_a2ui_bar_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"components\":[{\"type\":\"BarChart\",\"props\":{\"title\":\"Quarterly Revenue\",\"data\":[{\"label\":\"Q1\",\"value\":800000},{\"label\":\"Q2\",\"value\":950000},{\"label\":\"Q3\",\"value\":1100000},{\"label\":\"Q4\",\"value\":1250000}]}}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly revenue",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above showing quarterly revenue trend."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "status-report pill — must trigger render_a2ui with StatusBadge catalog component. The probe asserts declarative-status-badge testid mounts (this is the distinguishing signal).",
|
||||
"match": {
|
||||
"userMessage": "status report on system health",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_agno_render_a2ui_status_001",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"components\":[{\"type\":\"StatusBadge\",\"props\":{\"label\":\"API\",\"status\":\"healthy\"}},{\"type\":\"StatusBadge\",\"props\":{\"label\":\"Database\",\"status\":\"healthy\"}},{\"type\":\"StatusBadge\",\"props\":{\"label\":\"Background Workers\",\"status\":\"degraded\"}}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "status report on system health",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "System health report rendered above. API and database are healthy; background workers are degraded."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / gen-ui-headless-complete",
|
||||
"sourceFile": "headless-complete.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"_note": "Alias of the headless-complete demo's 4 gen-UI pills (weather/stock/highlight/revenue) for the gen-ui-headless-complete probe (showcase/harness/src/probes/scripts/d5-gen-ui-headless-complete.ts -> fixtureFile 'gen-ui-headless-complete.json'), which had no matching fixture in any slug so its first leg 503'd under strict. Narration (toolCallId) fixtures FIRST so they win when the last message is a matching tool result; toolcall (userMessage+context) fixtures AFTER with NO turnIndex gate so each of the 4 sequential pill turns in one chat thread matches regardless of prior assistant/tool history (the turnIndex multi-turn match-gate bug). Modeled on the green langgraph-python pattern. Interrupt fixtures intentionally omitted (interrupt-headless is a separate cluster item)."
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "headless-complete weather pill narration — matches when the get_weather tool result with id call_d5_headless_weather_001 is the last message. The 'tokyo' textToken assertion is satisfied by the city name. Narration verbatim matches the headless-complete.spec.ts assertion `toContainText('Tokyo is 22°C and partly cloudy.')`.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_weather_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Tokyo is 22°C and partly cloudy. The headless WeatherCard above shows the sunny snapshot from the tool."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete stock pill narration — matches when the get_stock_price tool result with id call_d5_headless_stock_001 is the last message. textToken 'aapl' satisfied by ticker mention.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_stock_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "AAPL is at $189.42, up 1.27% on the day."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete highlight pill narration — matches when the highlight_note tool result with id call_d5_headless_highlight_001 is the last message.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_highlight_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Highlighted 'ship the demo on Friday' for you above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete revenue chart narration — matches when the get_revenue_chart tool result with id call_d5_headless_revenue_chart_001 is the last message.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_revenue_chart_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the chart of revenue over the last six months — quarterly revenue grew steadily, peaking in June."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete weather pill toolcall — emit get_weather for Tokyo. Probe sends \"What's the weather in Tokyo?\". Narrowed to the full phrase 'What's the weather in Tokyo' so this no longer shadows the D6 tool-rendering 'Chain tools' pill (whose prompt contains 'weather in Tokyo' as a substring) — that pill needs to hit its own 3-tool emission fixture in tool-rendering.json. Only userMessage+context discriminators because narration fixtures above (toolCallId) already win once the tool result lands.",
|
||||
"match": {
|
||||
"userMessage": "What's the weather in Tokyo",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete stock pill toolcall — emit get_stock_price for AAPL. Probe sends \"What's the price of AAPL right now?\". Narrowed to 'price of AAPL right now' so this no longer shadows the D6 tool-rendering 'Stock price' pill which sends \"What's the current price of AAPL?\" (also containing 'price of AAPL' as a substring) — that pill needs its own deterministic-price fixture in tool-rendering.json.",
|
||||
"match": {
|
||||
"userMessage": "price of AAPL right now",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_stock_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"AAPL\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete highlight pill toolcall — emit highlight_note. Probe sends \"Highlight this note for me: 'ship the demo on Friday'.\" so userMessage substring 'ship the demo on Friday' catches it.",
|
||||
"match": {
|
||||
"userMessage": "ship the demo on Friday",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_highlight_001",
|
||||
"name": "highlight_note",
|
||||
"arguments": "{\"text\":\"ship the demo on Friday\",\"color\":\"yellow\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete revenue chart toolcall — emit get_revenue_chart (no args). Probe sends 'Show me a chart of revenue over the last six months.'. Includes a short leading `content` snippet so the word 'revenue' lands in the assistant bubble synchronously with the tool-call emission. Without this, the chart card's title falls back to 'Chart' during the `executing` render phase (result still undefined), and the conversation-runner's count-based settle can fire its 1500ms quiet window before the narration follow-up message arrives — leaving the assistant text without 'revenue' on the probe's first attempt.",
|
||||
"match": {
|
||||
"userMessage": "chart of revenue over the last six months",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pulling the revenue chart for the last six months...",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_revenue_chart_001",
|
||||
"name": "get_revenue_chart",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / gen-ui-open-advanced — delegates to gen-ui-open.json (same file, shared fixtures)",
|
||||
"note": "The D5 script registers fixtureFile='gen-ui-open.json' for gen-ui-open-advanced, so aimock loads gen-ui-open.json for both features. This file exists for completeness but the canonical fixtures live in gen-ui-open.json. Sandboxed-UI jsFunctions copied verbatim from langgraph-typescript (canonical) so keypad+= and the evaluate button wire to the host bridge.",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "gen-ui-open-advanced: Calculator pill — generateSandboxedUi with jsFunctions wiring for evaluateExpression host callback.",
|
||||
"match": {
|
||||
"userMessage": "Calculator (calls evaluateExpression)",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_agno_gen_ui_open_adv_calc_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:240px}.display{background:#1e293b;padding:8px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}button{padding:10px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',async function(){if(btn.id==='eq'){var res=await Websandbox.connection.remote.evaluateExpression({expression:expr});if(res&&res.ok){display.textContent=String(res.value);expr=String(res.value);}else{display.textContent='err';expr='';}}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-open-advanced: Ping the host pill — generateSandboxedUi with notifyHost callback.",
|
||||
"match": {
|
||||
"userMessage": "Ping the host (calls notifyHost)",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_agno_gen_ui_open_adv_ping_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":320,\"placeholderMessages\":[\"Composing host-ping card…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.card{padding:24px;background:#1e293b;border-radius:12px;margin:16px;text-align:center}button{padding:10px 20px;background:#6366f1;border:0;color:#fff;border-radius:8px;font-size:14px;cursor:pointer}.out{margin-top:12px;font-size:12px;color:#94a3b8}\",\"html\":\"<div class=\\\"card\\\" data-testid=\\\"ogui-ping\\\"><h2>Notify the host</h2><button id=\\\"hi\\\">Say hi to the host</button><div class=\\\"out\\\" id=\\\"out\\\">awaiting click…</div></div>\",\"jsFunctions\":\"document.getElementById('hi').addEventListener('click',async function(){var out=document.getElementById('out');out.textContent='sending…';var res=await Websandbox.connection.remote.notifyHost({message:'Hello from sandbox'});out.textContent=res&&res.ok?'host replied at '+res.receivedAt:'failed';});\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-open-advanced pill — 'Inline expression evaluator'. Duplicated from gen-ui-open.json for direct-file-load scenarios where only this file is loaded.",
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_agno_gen_ui_open_adv_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":320,\"placeholderMessages\":[\"Composing expression evaluator…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.card{padding:16px;background:#1e293b;border-radius:12px;margin:16px}input{width:100%;padding:8px;background:#0f172a;border:1px solid #334155;color:#e2e8f0;border-radius:6px;box-sizing:border-box}button{margin-top:8px;padding:8px 16px;background:#6366f1;border:0;color:#fff;border-radius:6px;cursor:pointer}.out{margin-top:8px;font-size:12px;color:#94a3b8}\",\"html\":\"<div class=\\\"card\\\" data-testid=\\\"ogui-inline-eval\\\"><h2>Inline expression evaluator</h2><input id=\\\"in\\\" placeholder=\\\"e.g. 2 + 2\\\"/><button id=\\\"go\\\">Evaluate</button><div class=\\\"out\\\" id=\\\"out\\\">awaiting input…</div></div>\",\"jsFunctions\":\"(function(){var input=document.getElementById('in');var out=document.getElementById('out');var go=document.getElementById('go');async function run(){var expr=input.value;out.textContent='evaluating…';var res=await Websandbox.connection.remote.evaluateExpression({expression:expr});out.textContent=res&&res.ok?'= '+res.value:'error: '+res.error;}go.addEventListener('click',run);input.addEventListener('keydown',function(e){if(e.key==='Enter')run();});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Expression evaluator rendered above — type any arithmetic expression and click Evaluate to compute via the host bridge."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / gen-ui-open (basic + advanced)",
|
||||
"sourceFile": "d5-gen-ui-open.ts, d5-gen-ui-open-advanced.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "gen-ui-open basic pill — '3D axis visualization (model airplane)'. The probe asserts iframe[srcdoc] mounts with ≥100 chars. The response must include an srcdoc-shaped HTML payload that the demo can render inside an iframe.",
|
||||
"match": {
|
||||
"userMessage": "3D axis visualization",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_agno_gen_ui_open_3d_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"html\":\"<!DOCTYPE html><html><head><style>body{margin:0;display:flex;justify-content:center;align-items:center;height:100vh;background:#1a1a2e;font-family:monospace}canvas{border:1px solid #333}</style></head><body><canvas id='c' width='400' height='400'></canvas><script>const c=document.getElementById('c'),x=c.getContext('2d');x.translate(200,200);x.strokeStyle='#e94560';x.beginPath();x.moveTo(0,0);x.lineTo(150,0);x.stroke();x.strokeStyle='#0f3460';x.beginPath();x.moveTo(0,0);x.lineTo(0,-150);x.stroke();x.strokeStyle='#16213e';x.beginPath();x.moveTo(0,0);x.lineTo(-80,80);x.stroke();x.fillStyle='#eee';x.font='14px monospace';x.fillText('X',155,5);x.fillText('Y',5,-155);x.fillText('Z',-90,90)</script></body></html>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "3D axis visualization",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "3D axis visualization rendered above in the sandboxed iframe — the X, Y, and Z axes are drawn on a dark canvas."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-open-advanced pill — 'Inline expression evaluator'. The probe asserts an iframe with sandbox='allow-scripts' mounts. Uses generateSandboxedUi tool call.",
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_agno_gen_ui_open_adv_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"html\":\"<!DOCTYPE html><html><head><style>body{margin:20px;font-family:monospace;background:#111;color:#0f0}input{width:100%;padding:8px;font-size:16px;background:#222;color:#0f0;border:1px solid #333}#result{margin-top:12px;padding:8px;background:#1a1a1a;border:1px solid #333;min-height:24px}</style></head><body><h3>Expression Evaluator</h3><input id='expr' placeholder='Type a JS expression...' oninput='try{document.getElementById(\\\"result\\\").textContent=eval(this.value)}catch(e){document.getElementById(\\\"result\\\").textContent=e.message}'><div id='result'></div></body></html>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Expression evaluator rendered above — type any JavaScript expression in the input field to see its result."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,232 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / gen-ui-tool-based",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "3D axis visualization (model airplane)",
|
||||
"toolCallId": "call_d5_open_gen_ui_3d_axis_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "3D axis visualization (model airplane)",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_3d_axis_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing 3D axis scene…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px}h1{font-size:14px;margin:0 0 8px}svg{display:block;background:#1e293b;border-radius:8px}\",\"html\":\"<div class=\\\"wrap\\\"><h1>3D axis visualization (pitch / yaw / roll)</h1><svg width=\\\"320\\\" height=\\\"260\\\" viewBox=\\\"-80 -80 160 160\\\" data-testid=\\\"ogui-3d-axis\\\"><line x1=\\\"-60\\\" y1=\\\"0\\\" x2=\\\"60\\\" y2=\\\"0\\\" stroke=\\\"#f59e0b\\\"/><line x1=\\\"0\\\" y1=\\\"-60\\\" x2=\\\"0\\\" y2=\\\"60\\\" stroke=\\\"#6366f1\\\"/><line x1=\\\"-40\\\" y1=\\\"40\\\" x2=\\\"40\\\" y2=\\\"-40\\\" stroke=\\\"#10b981\\\"/><text x=\\\"62\\\" y=\\\"4\\\" font-size=\\\"8\\\" fill=\\\"#f59e0b\\\">X pitch</text><text x=\\\"4\\\" y=\\\"-62\\\" font-size=\\\"8\\\" fill=\\\"#6366f1\\\">Y yaw</text><text x=\\\"42\\\" y=\\\"-42\\\" font-size=\\\"8\\\" fill=\\\"#10b981\\\">Z roll</text></svg></div>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "How a neural network works",
|
||||
"toolCallId": "call_d5_open_gen_ui_neural_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "How a neural network works",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_neural_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing neural network forward pass…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px}h1{font-size:14px;margin:0 0 8px}svg{display:block;background:#1e293b;border-radius:8px}circle{fill:#6366f1}\",\"html\":\"<div class=\\\"wrap\\\"><h1>Forward pass: input → hidden → output</h1><svg width=\\\"320\\\" height=\\\"220\\\" data-testid=\\\"ogui-neural-net\\\"><g><circle cx=\\\"40\\\" cy=\\\"40\\\" r=\\\"8\\\"/><circle cx=\\\"40\\\" cy=\\\"90\\\" r=\\\"8\\\"/><circle cx=\\\"40\\\" cy=\\\"140\\\" r=\\\"8\\\"/><circle cx=\\\"40\\\" cy=\\\"190\\\" r=\\\"8\\\"/></g><g fill=\\\"#a78bfa\\\"><circle cx=\\\"160\\\" cy=\\\"30\\\" r=\\\"8\\\"/><circle cx=\\\"160\\\" cy=\\\"75\\\" r=\\\"8\\\"/><circle cx=\\\"160\\\" cy=\\\"115\\\" r=\\\"8\\\"/><circle cx=\\\"160\\\" cy=\\\"155\\\" r=\\\"8\\\"/><circle cx=\\\"160\\\" cy=\\\"195\\\" r=\\\"8\\\"/></g><g fill=\\\"#10b981\\\"><circle cx=\\\"280\\\" cy=\\\"80\\\" r=\\\"8\\\"/><circle cx=\\\"280\\\" cy=\\\"140\\\" r=\\\"8\\\"/></g></svg></div>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Quicksort visualization",
|
||||
"toolCallId": "call_d5_open_gen_ui_quicksort_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Quicksort visualization",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_quicksort_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing quicksort animation…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px}h1{font-size:14px;margin:0 0 8px}svg{display:block;background:#1e293b;border-radius:8px}rect{fill:#64748b}\",\"html\":\"<div class=\\\"wrap\\\"><h1>Quicksort: partition around pivot</h1><svg width=\\\"320\\\" height=\\\"220\\\" data-testid=\\\"ogui-quicksort\\\"><rect x=\\\"10\\\" y=\\\"170\\\" width=\\\"24\\\" height=\\\"40\\\"/><rect x=\\\"40\\\" y=\\\"130\\\" width=\\\"24\\\" height=\\\"80\\\"/><rect x=\\\"70\\\" y=\\\"100\\\" width=\\\"24\\\" height=\\\"110\\\"/><rect x=\\\"100\\\" y=\\\"60\\\" width=\\\"24\\\" height=\\\"150\\\" fill=\\\"#f59e0b\\\"/><rect x=\\\"130\\\" y=\\\"80\\\" width=\\\"24\\\" height=\\\"130\\\" fill=\\\"#6366f1\\\"/><rect x=\\\"160\\\" y=\\\"110\\\" width=\\\"24\\\" height=\\\"100\\\"/><rect x=\\\"190\\\" y=\\\"50\\\" width=\\\"24\\\" height=\\\"160\\\"/><rect x=\\\"220\\\" y=\\\"90\\\" width=\\\"24\\\" height=\\\"120\\\"/><rect x=\\\"250\\\" y=\\\"140\\\" width=\\\"24\\\" height=\\\"70\\\"/><rect x=\\\"280\\\" y=\\\"160\\\" width=\\\"24\\\" height=\\\"50\\\"/></svg></div>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Fourier: square wave from sines",
|
||||
"toolCallId": "call_d5_open_gen_ui_fourier_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Fourier: square wave from sines",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_fourier_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing Fourier series animation…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px}h1{font-size:14px;margin:0 0 8px}svg{display:block;background:#1e293b;border-radius:8px}\",\"html\":\"<div class=\\\"wrap\\\"><h1>Fourier series: square wave</h1><svg width=\\\"320\\\" height=\\\"220\\\" viewBox=\\\"0 -60 320 120\\\" data-testid=\\\"ogui-fourier\\\"><circle cx=\\\"60\\\" cy=\\\"0\\\" r=\\\"40\\\" stroke=\\\"#6366f1\\\" fill=\\\"none\\\"/><circle cx=\\\"60\\\" cy=\\\"0\\\" r=\\\"13\\\" stroke=\\\"#a78bfa\\\" fill=\\\"none\\\"/><circle cx=\\\"60\\\" cy=\\\"0\\\" r=\\\"8\\\" stroke=\\\"#c4b5fd\\\" fill=\\\"none\\\"/><path d=\\\"M120 0 L300 0\\\" stroke=\\\"#10b981\\\" fill=\\\"none\\\"/></svg></div>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Calculator (calls evaluateExpression)",
|
||||
"toolCallId": "call_d5_open_gen_ui_calc_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Calculator (calls evaluateExpression)",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_calc_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:240px}.display{background:#1e293b;padding:8px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}button{padding:10px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',async function(){if(btn.id==='eq'){var res=await Websandbox.connection.remote.evaluateExpression({expression:expr});if(res&&res.ok){display.textContent=String(res.value);expr=String(res.value);}else{display.textContent='err';expr='';}}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Ping the host (calls notifyHost)",
|
||||
"toolCallId": "call_d5_open_gen_ui_ping_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Ping the host (calls notifyHost)",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_ping_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":320,\"placeholderMessages\":[\"Composing host-ping card…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.card{padding:24px;background:#1e293b;border-radius:12px;margin:16px;text-align:center}button{padding:10px 20px;background:#6366f1;border:0;color:#fff;border-radius:8px;font-size:14px;cursor:pointer}.out{margin-top:12px;font-size:12px;color:#94a3b8}\",\"html\":\"<div class=\\\"card\\\" data-testid=\\\"ogui-ping\\\"><h2>Notify the host</h2><button id=\\\"hi\\\">Say hi to the host</button><div class=\\\"out\\\" id=\\\"out\\\">awaiting click…</div></div>\",\"jsFunctions\":\"document.getElementById('hi').addEventListener('click',async function(){var out=document.getElementById('out');out.textContent='sending…';var res=await Websandbox.connection.remote.notifyHost({message:'Hello from sandbox'});out.textContent=res&&res.ok?'host replied at '+res.receivedAt:'failed';});\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"toolCallId": "call_d5_open_gen_ui_inline_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_inline_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":320,\"placeholderMessages\":[\"Composing expression evaluator…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.card{padding:16px;background:#1e293b;border-radius:12px;margin:16px}input{width:100%;padding:8px;background:#0f172a;border:1px solid #334155;color:#e2e8f0;border-radius:6px;box-sizing:border-box}button{margin-top:8px;padding:8px 16px;background:#6366f1;border:0;color:#fff;border-radius:6px;cursor:pointer}.out{margin-top:8px;font-size:12px;color:#94a3b8}\",\"html\":\"<div class=\\\"card\\\" data-testid=\\\"ogui-inline-eval\\\"><h2>Inline expression evaluator</h2><input id=\\\"in\\\" placeholder=\\\"e.g. 2 + 2\\\"/><button id=\\\"go\\\">Evaluate</button><div class=\\\"out\\\" id=\\\"out\\\">awaiting input…</div></div>\",\"jsFunctions\":\"(function(){var input=document.getElementById('in');var out=document.getElementById('out');var go=document.getElementById('go');async function run(){var expr=input.value;out.textContent='evaluating…';var res=await Websandbox.connection.remote.evaluateExpression({expression:expr});out.textContent=res&&res.ok?'= '+res.value:'error: '+res.error;}go.addEventListener('click',run);input.addEventListener('keydown',function(e){if(e.key==='Enter')run();});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "request the gen-ui interrupt",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The agent paused at a gen-UI interrupt and rendered a choice component for the user. Choose to continue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "confirm the gen-ui choice",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Gen-UI interrupt resolved. The agent received the user's choice and resumed, completing the workflow."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "render an open gen-ui element",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The open gen-UI element was rendered. The LLM produced an arbitrary-shape JSON payload and the renderer materialized it as a UI block."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "continue the advanced gen-ui flow",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The advanced gen-UI flow continued with the second-step component. The chained payloads from turns 1 and 2 form the complete advanced gen-UI sequence."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / headless-complete",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "headless-complete highlight pill: high-priority verbatim match. Without this, the showcase-assistant catch-all in feature-parity.json wins. Two-turn fixture: turn 1 emits the highlight_note tool call so the headless useComponent registration paints a Highlight card; turn 2 (after the tool result) emits the deterministic narration.",
|
||||
"match": {
|
||||
"userMessage": "Highlight: ship the demo on Friday",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_highlight_001",
|
||||
"name": "highlight_note",
|
||||
"arguments": "{\"text\":\"ship the demo on Friday\",\"color\":\"yellow\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Highlight: ship the demo on Friday",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Highlighted 'ship the demo on Friday' for you above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "chart of revenue over the last six months",
|
||||
"toolCallId": "call_d5_headless_revenue_chart_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the chart of revenue over the last six months — quarterly revenue grew steadily, peaking in June."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete revenue chart pill: locks the deterministic chart leading phrase. Two-turn fixture: turn 1 emits the get_revenue_chart tool call (no parameters per the python tool); turn 2 emits the narration that the headless message-assistant bubble renders alongside the ChartCard.",
|
||||
"match": {
|
||||
"userMessage": "chart of revenue over the last six months",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_revenue_chart_001",
|
||||
"name": "get_revenue_chart",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "trigger the headless interrupt",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Interrupt raised. The agent has paused at a graph-level interrupt and is waiting for user input. Reply with 'yes' or 'no' to resume."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "resolve the interrupt with yes",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Interrupt resolved with 'yes'. The agent resumed execution from the suspended node and produced its final answer."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "ship the demo on Friday",
|
||||
"toolCallId": "call_d5_highlight_note_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Highlighted 'ship the demo on Friday' in yellow above. Let me know if you want a different color or a longer note."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "ship the demo on Friday",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_highlight_note_001",
|
||||
"name": "highlight_note",
|
||||
"arguments": "{\"text\":\"ship the demo on Friday\",\"color\":\"yellow\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / headless-simple",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "headless-simple pill 1: locks the deterministic greeting leading phrase. The reply is intentionally NON-boilerplate (i.e. NOT the showcase-assistant catch-all 'I can help you with weather lookups...') so the headless-simple spec's HELLO_LEADING assertion fails when fixture priority misroutes this prompt to the catch-all.",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one short sentence",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi! In one short sentence: I'm a CopilotKit demo agent here to help you try features."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-simple pill 2: locks the deterministic joke.",
|
||||
"match": {
|
||||
"userMessage": "Tell me a one-line joke",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the scarecrow win an award? Because he was outstanding in his field!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-simple pill 3: locks the deterministic fun fact.",
|
||||
"match": {
|
||||
"userMessage": "Give me a fun fact",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "A fun fact: Honey never spoils! Archaeologists have found pots of honey in ancient Egyptian tombs that are over 3,000 years old and still perfectly edible."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / hitl-in-app",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Issue a $50 refund to customer #12345",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_request_approval_001",
|
||||
"name": "request_user_approval",
|
||||
"arguments": "{\"message\":\"Issue a $50 refund to customer #12345.\",\"context\":\"Per the standard goodwill-credit policy for shipping delays.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Issue a $50 refund to customer #12345",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — refund pill (#12345), post-tool-result response. Content includes BOTH approve and reject key phrases so both test assertions can match (the test checks for a substring, not the full string). This avoids sequenceIndex which is fragile across aimock restarts.",
|
||||
"match": {
|
||||
"userMessage": "$50 refund to Jordan Rivera on ticket #12345",
|
||||
"toolCallId": "call_d5_hitl_refund_12345_001",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "I am processing the $50 refund to Jordan Rivera on ticket #12345 now. The refund request was not approved by default — your explicit approval or rejection determines the outcome."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — refund pill (#12345). 1st turn: emit request_user_approval tool call. The post-tool-result fixture above (with toolCallId + hasToolResult: true) is more specific and wins on the 2nd turn, so hasToolResult: false is NOT needed here — omitting it allows this fixture to match even in multi-pill conversations where prior tool results exist.",
|
||||
"match": {
|
||||
"userMessage": "$50 refund to Jordan Rivera on ticket #12345",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_hitl_refund_12345_001",
|
||||
"name": "request_user_approval",
|
||||
"arguments": "{\"message\":\"Issue a $50 refund to Jordan Rivera on ticket #12345 for the duplicate charge.\",\"context\":\"Ticket #12345 — duplicate-charge goodwill credit.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — downgrade pill (#12346), 2nd turn. Same pattern as refund/escalate. Without this entry, the generic 'plan' catch-all in feature-parity.json hijacks the downgrade message.",
|
||||
"match": {
|
||||
"userMessage": "downgrade Priya Shah (#12346) to the Starter plan",
|
||||
"toolCallId": "call_d5_hitl_downgrade_12346_001",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Downgrade confirmed — Priya Shah (#12346) will move to the Starter plan effective next billing cycle."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — downgrade pill (#12346). 1st turn: emit request_user_approval tool call. Specific userMessage substring takes precedence over the bare 'plan' fixture in feature-parity.json (which lives later in the load order). hasToolResult: false omitted so multi-pill conversations still match.",
|
||||
"match": {
|
||||
"userMessage": "downgrade Priya Shah (#12346) to the Starter plan",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_hitl_downgrade_12346_001",
|
||||
"name": "request_user_approval",
|
||||
"arguments": "{\"message\":\"Downgrade Priya Shah (#12346) to the Starter plan effective next billing cycle.\",\"context\":\"Ticket #12346 — voluntary downgrade per customer request.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — escalate pill (#12347), post-tool-result response. Content includes both approve and reject key phrases.",
|
||||
"match": {
|
||||
"userMessage": "escalate ticket #12347 to the payments team",
|
||||
"toolCallId": "call_d5_hitl_escalate_12347_001",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Escalated ticket #12347 to the payments team for Morgan Lee. Not escalated by default — your explicit approval determines the outcome."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — escalate pill (#12347). 1st turn: emit request_user_approval tool call. hasToolResult: false omitted so multi-pill conversations still match (the post-tool-result fixture above with toolCallId is more specific and wins on the 2nd turn).",
|
||||
"match": {
|
||||
"userMessage": "escalate ticket #12347 to the payments team",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_hitl_escalate_12347_001",
|
||||
"name": "request_user_approval",
|
||||
"arguments": "{\"message\":\"Escalate ticket #12347 to the payments team for Morgan Lee's stuck payment.\",\"context\":\"Ticket #12347 — payment stuck, needs payments-team triage.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / hitl-in-chat",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Book a 30-minute onboarding call for Alice",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_book_call_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Onboarding call\",\"attendee\":\"Alice\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Book a 30-minute onboarding call for Alice",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolCallId": "call_d5_schedule_sales_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked: Sales intro call confirmed for the slot you picked."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolName": "schedule_meeting",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "Sales intro call",
|
||||
"attendee": "Sales team",
|
||||
"slots": [
|
||||
{
|
||||
"label": "Mon 10:00 AM",
|
||||
"iso": "2026-05-11T10:00:00Z"
|
||||
},
|
||||
{
|
||||
"label": "Tue 2:00 PM",
|
||||
"iso": "2026-05-12T14:00:00Z"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": "call_d5_schedule_sales_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolCallId": "call_d5_schedule_alice_001",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Scheduled: 1:1 with Alice locked in for the slot you picked."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolName": "schedule_meeting",
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "1:1 with Alice — Q2 goals",
|
||||
"attendee": "Alice",
|
||||
"slots": [
|
||||
{
|
||||
"label": "Wed 11:00 AM",
|
||||
"iso": "2026-05-13T11:00:00Z"
|
||||
},
|
||||
{
|
||||
"label": "Thu 3:30 PM",
|
||||
"iso": "2026-05-14T15:30:00Z"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": "call_d5_schedule_alice_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / mcp-apps",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Open Excalidraw and sketch a system diagram",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sketched a three-box system diagram (Client → Server → Database). Tap the iframe to open it in Excalidraw."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D5 mcp-apps suggestion pill #1 (\"Draw a flowchart\"). Sends the verbatim suggestion message \"Use Excalidraw to draw a simple flowchart with three steps.\" The distinctive substring \"draw a simple flowchart\" prevents collision with feature-parity.json's generic `{userMessage: \"steps\"}` fixture, which would otherwise absorb the prompt and return a tool-call-free planning blurb (breaking the iframe). Turn 1 emits `create_view` with three-step flowchart elements so the MCP middleware fetches the Excalidraw UI resource and mounts the iframe; turn 2 (after the tool result) emits the deterministic narration.",
|
||||
"match": {
|
||||
"userMessage": "draw a simple flowchart",
|
||||
"toolName": "create_view",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "The user wants a simple three-step flowchart. I'll call create_view once with three labelled rectangles connected by arrows and a title, framed by a cameraUpdate.",
|
||||
"content": "Drawing a simple three-step flowchart in Excalidraw.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_mcp_apps_create_view_flowchart_001",
|
||||
"name": "create_view",
|
||||
"arguments": "{\"elements\":[{\"id\":\"title\",\"type\":\"text\",\"x\":300,\"y\":40,\"text\":\"Flowchart\",\"fontSize\":24},{\"id\":\"step1\",\"type\":\"rectangle\",\"x\":80,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"Start\",\"fontSize\":18}},{\"id\":\"step2\",\"type\":\"rectangle\",\"x\":320,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"Process\",\"fontSize\":18}},{\"id\":\"step3\",\"type\":\"rectangle\",\"x\":560,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"End\",\"fontSize\":18}},{\"id\":\"a1\",\"type\":\"arrow\",\"x\":240,\"y\":195,\"endX\":320,\"endY\":195},{\"id\":\"a2\",\"type\":\"arrow\",\"x\":480,\"y\":195,\"endX\":560,\"endY\":195},{\"id\":\"camera\",\"type\":\"cameraUpdate\",\"x\":40,\"y\":0,\"width\":800,\"height\":600}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "draw a simple flowchart",
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Drew a three-step flowchart (Start → Process → End). Tap the iframe to open it in Excalidraw."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Use Excalidraw to sketch",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sketched a simple system diagram for you above — three boxes (frontend, runtime, agent) connected by arrows showing the request flow. Ask if you'd like a different shape or more detail."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / multimodal",
|
||||
"sourceFile": "d5-multimodal.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "Multimodal image turn \u2014 the D5 script clicks the 'Try with sample image' button which auto-sends a message. The assertion checks the assistant transcript contains the keyword 'image'. The userMessage match uses the describe prompt keyed in agentic-chat.json.",
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The image attachment shows a small abstract test pattern used by the multimodal demo to validate the image-upload pipeline. Successful render of this response confirms the binary attachment round-tripped through the runtime."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Multimodal PDF turn \u2014 the D5 script clicks the 'Try with sample PDF' button which auto-sends a message. The assertion checks the assistant transcript contains the keyword 'document'.",
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The PDF document contains a single test page used by the multimodal demo. Its text was flattened and forwarded as text content to the model. Receiving this response confirms the document upload path works."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / prebuilt-popup",
|
||||
"sourceFile": "d5-prebuilt-popup.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "prebuilt-popup turn — the D5 probe sends 'hi from the popup test'. Assertion checks .copilotKitPopup root is visible and assistant response lands inside it.",
|
||||
"match": {
|
||||
"userMessage": "hi from the popup test",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello from the popup — the CopilotPopup prebuilt component is wired up and reachable. The launcher floats in the corner and the chat sits in an overlay panel."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / prebuilt-sidebar",
|
||||
"sourceFile": "d5-prebuilt-sidebar.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "prebuilt-sidebar turn — the D5 probe sends 'hi from the sidebar test'. Assertion checks .copilotKitSidebar root is visible and assistant response lands inside it.",
|
||||
"match": {
|
||||
"userMessage": "hi from the sidebar test",
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello from the sidebar — the CopilotSidebar prebuilt component is wired up and reachable. Anything else you would like me to confirm from inside the sidebar?"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / readonly-state",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "readonly-state-agent-context — Who am I pill. Matches the verbatim pill prompt. systemMessage gating on 'Atai' was removed because @langchain/openai >=1.4 sends SystemMessage as role:'developer' for gpt-5 models, and aimock's getSystemText only checks role:'system'. The userMessage alone is deterministic from the pill.",
|
||||
"match": {
|
||||
"userMessage": "What do you know about me from my context?",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "I see you're Atai, and you're in the America/Los_Angeles timezone. Recently, you viewed the pricing page and watched the product demo video. How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "readonly-state-agent-context — Suggest next steps pill. Matches the verbatim pill prompt. systemMessage gating removed (see Who am I comment above).",
|
||||
"match": {
|
||||
"userMessage": "Based on my recent activity, what should I try next?",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Since you recently viewed the pricing page and watched the product demo video, it might be a good idea to explore user testimonials or case studies to see how others have benefited from the Pro Plan. You could also start the 14-day free trial to experience the features firsthand."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What do you know about me from my context",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your context I can see your name and recent activity. I'll keep responses calibrated to your timezone."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / reasoning",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "reasoning-default e2e test pill prompt. Substring 'sky appears blue' matches 'Explain step by step why the sky appears blue during the day but red at sunset.' Includes a `reasoning` field so aimock emits REASONING_MESSAGE_* events; without it the built-in CopilotChatReasoningMessage 'Thinking…/Thought for…' label never lands.",
|
||||
"match": {
|
||||
"userMessage": "sky appears blue",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "First, I considered that visible sunlight contains all wavelengths. Then I noted that air molecules scatter shorter wavelengths (blue) more efficiently than longer ones (Rayleigh scattering). At sunset the path through the atmosphere is much longer, so even more blue is scattered out and the remaining direct light skews red.",
|
||||
"content": "Daytime sky looks blue because air molecules scatter short-wavelength light more strongly than long-wavelength light (Rayleigh scattering). At sunset the sun's light traverses far more atmosphere, scattering out most of the blue and leaving the remaining direct light dominated by red and orange wavelengths."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D5 reasoning-display probe (harness/src/probes/scripts/d5-reasoning-display.ts) sends 'show your reasoning step by step' verbatim. Mirrored from showcase/harness/fixtures/d5/reasoning-display.json so the bundled aimock has a match (the harness fixture file isn't loaded by the Docker-baked aimock).",
|
||||
"match": {
|
||||
"userMessage": "show your reasoning step by step",
|
||||
"turnIndex": 0,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "First, I identified that the question requires step-by-step reasoning. Then I broke it into sub-steps and worked through each one. Finally, I aggregated the partial answers into a single response.",
|
||||
"content": "Reasoning: first, I identified the question requires step-by-step thinking. Then, I broke it into sub-steps and worked through each one. Finally, I aggregated the partial answers into a single response. The reasoning block above the answer demonstrates intermediate-thought rendering."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for agno / recorded",
|
||||
"sourceFile": "d5-all.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-54-59-257Z-b614f3f8.json",
|
||||
"match": {
|
||||
"userMessage": "First use the query_data tool to fetch the financial sales data, then using A2UI, show me a sales dashboard with total revenue, new customers, and conversion rate metrics. Include a pie chart of revenue by category and a bar chart of monthly sales.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 0,
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_data",
|
||||
"arguments": "{\"query\":\"financial sales data including total revenue, new customers, conversion rate, revenue by category, and monthly sales\"}",
|
||||
"id": "call_jWKVlcdnnYiV6MT3wzNZphnF"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-55-00-312Z-814153b0.json",
|
||||
"match": {
|
||||
"userMessage": "First use the query_data tool to fetch the financial sales data, then using A2UI, show me a sales dashboard with total revenue, new customers, and conversion rate metrics. Include a pie chart of revenue by category and a bar chart of monthly sales.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 1,
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}",
|
||||
"id": "call_K6au6wp1CjG4OuiGzhL1yCXL"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-55-05-134Z-890eead9.json",
|
||||
"match": {
|
||||
"userMessage": "First use the query_data tool to fetch the financial sales data, then using A2UI, show me a sales dashboard with total revenue, new customers, and conversion rate metrics. Include a pie chart of revenue by category and a bar chart of monthly sales.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 2,
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\": \"Revenue by Category\", \"description\": \"Distribution of revenue across different categories\", \"data\": [{\"label\": \"Enterprise Subscriptions\", \"value\": 94000}, {\"label\": \"Pro Tier Upgrades\", \"value\": 66500}, {\"label\": \"API Usage Overages\", \"value\": 34500}, {\"label\": \"Consulting Services\", \"value\": 54000}, {\"label\": \"Marketplace Sales\", \"value\": 42800}, {\"label\": \"Partnership Revenue\", \"value\": 25700}, {\"label\": \"Training & Workshops\", \"value\": 10200}]}",
|
||||
"id": "call_OJDsrTtDcWoV0PObgKtU9fXv"
|
||||
},
|
||||
{
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\": \"Monthly Sales\", \"description\": \"Sales performance over the months\", \"data\": [{\"label\": \"January\", \"value\": 82800}, {\"label\": \"February\", \"value\": 119000}, {\"label\": \"March\", \"value\": 135700}]}",
|
||||
"id": "call_o9RdNrZTq8qV0Majn12OTZcu"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-55-07-715Z-9531bf64.json",
|
||||
"match": {
|
||||
"userMessage": "First use the query_data tool to fetch the financial sales data, then using A2UI, show me a sales dashboard with total revenue, new customers, and conversion rate metrics. Include a pie chart of revenue by category and a bar chart of monthly sales.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 3,
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The sales dashboard is now available, featuring metrics for total revenue, new customers, and conversion rate. Additionally, you can view a pie chart of revenue by category and a bar chart of monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-56-02-547Z-7afa30ad.json. SUPERSEDED: this entry is unreachable and editing it has no effect — the calculator pill is served by the specific calculator fixtures in _from-feature-parity.json (loaded first alphabetically; first match wins). Note the model gate (gpt-4o) is NOT an independent safety net — whether it matches depends on each integration's current model; unreachability is guaranteed only by the load-order shadowing above. Kept for historical reference; do not delete.",
|
||||
"match": {
|
||||
"userMessage": "Using the generateSandboxedUi tool, build a modern calculator with standard buttons plus labeled metric shortcut buttons that insert their values into the display when clicked. Use sample company data.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 0,
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":400,\"placeholderMessages\":[\"Building calculator UI...\"],\"css\":\"body { font-family: Arial, sans-serif; display: flex; justify-content: center; align-items: center; height: 100vh; margin: 0; background-color: #f4f4f9; } .calculator { width: 320px; background-color: #fff; border-radius: 10px; box-shadow: 0 4px 10px rgba(0, 0, 0, 0.1); } .display { background-color: #222; color: #fff; font-size: 2em; padding: 20px; text-align: right; border-top-left-radius: 10px; border-top-right-radius: 10px; } .buttons { display: grid; grid-template-columns: repeat(4, 1fr); grid-gap: 1px; } .button { background-color: #e0e0e0; padding: 20px; font-size: 1.2em; text-align: center; cursor: pointer; transition: background-color 0.2s; } .button:hover { background-color: #d0d0d0; } .metric-button { background-color: #4caf50; color: #fff; }\",\"html\":\"<div class='calculator'> <div class='display' id='display'>0</div> <div class='buttons'> <div class='button' onclick='appendNumber(7)'>7</div> <div class='button' onclick='appendNumber(8)'>8</div> <div class='button' onclick='appendNumber(9)'>9</div> <div class='button' onclick='operate(\\\"/\\\")'>/</div> <div class='button' onclick='appendNumber(4)'>4</div> <div class='button' onclick='appendNumber(5)'>5</div> <div class='button' onclick='appendNumber(6)'>6</div> <div class='button' onclick='operate(\\\"*\\\")'>*</div> <div class='button' onclick='appendNumber(1)'>1</div> <div class='button' onclick='appendNumber(2)'>2</div> <div class='button' onclick='appendNumber(3)'>3</div> <div class='button' onclick='operate(\\\"-\\\")'>-</div> <div class='button' onclick='appendNumber(0)'>0</div> <div class='button' onclick='clearDisplay()'>C</div> <div class='button' onclick='calculate()'>=</div> <div class='button' onclick='operate(\\\"+\\\")'>+</div> <div class='button metric-button' onclick='insertMetric(1000)'>Revenue</div> <div class='button metric-button' onclick='insertMetric(500)'>Expenses</div> <div class='button metric-button' onclick='insertMetric(200)'>Profit</div> </div> </div>\",\"jsFunctions\":\"function appendNumber(number) { const display = document.getElementById('display'); if (display.innerText === '0') { display.innerText = number; } else { display.innerText += number; } } function operate(operator) { const display = document.getElementById('display'); display.innerText += ' ' + operator + ' '; } function clearDisplay() { document.getElementById('display').innerText = '0'; } function calculate() { const display = document.getElementById('display'); try { display.innerText = eval(display.innerText); } catch { display.innerText = 'Error'; } } function insertMetric(value) { const display = document.getElementById('display'); if (display.innerText === '0') { display.innerText = value; } else { display.innerText += value; } }\",\"jsExpressions\":[]}",
|
||||
"id": "call_u2aFJvWZUKzzlvqfnDeWhMyC"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-56-04-559Z-1bafed55.json. SUPERSEDED: this entry is unreachable and editing it has no effect — the calculator pill is served by the specific calculator fixtures in _from-feature-parity.json (loaded first alphabetically; first match wins). Note the model gate (gpt-4o) is NOT an independent safety net — whether it matches depends on each integration's current model; unreachability is guaranteed only by the load-order shadowing above. Kept for historical reference; do not delete.",
|
||||
"match": {
|
||||
"userMessage": "Using the generateSandboxedUi tool, build a modern calculator with standard buttons plus labeled metric shortcut buttons that insert their values into the display when clicked. Use sample company data.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 1,
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The modern calculator with standard buttons and labeled metric shortcut buttons is ready for use. You can perform calculations and insert sample company data values like Revenue, Expenses, and Profit directly into the display."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T22-01-34-384Z-edf48666.json",
|
||||
"match": {
|
||||
"userMessage": "What is the weather in Tokyo?",
|
||||
"model": "gpt-4o-mini",
|
||||
"turnIndex": 0,
|
||||
"hasToolResult": false,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}",
|
||||
"id": "call_0uXBAr4ySTrLQGQgszQYFtEJ"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T22-01-35-344Z-6ea6b18a.json",
|
||||
"match": {
|
||||
"userMessage": "What is the weather in Tokyo?",
|
||||
"model": "gpt-4o-mini",
|
||||
"turnIndex": 1,
|
||||
"hasToolResult": true,
|
||||
"context": "agno"
|
||||
},
|
||||
"response": {
|
||||
"content": "The weather in Tokyo is currently 68°F, with 55% humidity, a wind speed of 10 mph, and sunny conditions."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user