wehub/copilotkit--copilotkit
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 12:58:18 +08:00
commit 6d5d58c1a9
18293 changed files with 3502153 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/showcases/banking/docs/DESIGN.md
+128
View File
@@ -0,0 +1,128 @@
# Northwind Finance — Design System ("Aurora")
A premium, airy, light-theme fintech aesthetic for the banking demo. This is a
**visual layer**: it restyles the existing app without changing logic, data
flow, REST calls, the CopilotKit provider/runtime, HITL approval flows, the
threads drawer, auth/role gating, or the teachable policy-exception gate.
The system is token-driven. Tokens live in
[`src/app/globals.css`](../src/app/globals.css) as CSS custom properties,
surfaced to Tailwind v4 via `@theme inline` so utilities like `bg-surface`,
`text-ink-muted`, `from-brand-violet`, `ring-brand`, `bg-positive` resolve
everywhere. Both **light** (the primary showcase mode) and **dark** are fully
supported via the class-based `.light` / `.dark` toggle on `<html>`.
## Design language
- **Mood** — premium, airy, modern. Generous whitespace, soft layered shadows,
large radii, glassmorphism-lite surfaces.
- **Canvas** — a soft lavender/lilac page background; content floats above it on
white glass cards.
- **Primary** — a violet→indigo gradient (`#7C5CFC` → `#5B3DF5`), reused on the
primary button, the active nav icon, the credit-card face, progress bars, and
CTAs (`.brand-gradient`).
- **Semantics** — income/positive is emerald green with an up-and-right arrow;
expense/negative is rose/red with a down-and-right arrow.
- **Typography** — Inter (loaded via `next/font/google`, exposed as
`--font-inter` → `--font-sans`). Very large bold balance numbers, small
muted-grey labels, medium-weight violet section headings (`.section-heading`).
- **Shape** — cards ~22px radius (`rounded-2xl` / `--radius`), buttons fully
rounded pills (`rounded-full`), inputs/menus ~12–16px.
## Color tokens
All colors are HSL triplets behind `hsl(var(--token))`, exposed as Tailwind
colors (`brand`, `brand-violet`, `brand-indigo`, `brand-soft`, `surface`,
`surface-muted`, `canvas`, `ink`, `ink-muted`, `positive`, `positive-soft`,
`negative`, `negative-soft`, `hairline`).
| Token | Light | Dark | Role |
| ------------------- | ------------- | ------------- | --------------------------------------- |
| `--canvas` | `255 60% 97%` | `252 30% 7%` | App background (lavender / deep indigo) |
| `--surface` | `0 0% 100%` | `252 24% 11%` | Cards, sidebar, menus |
| `--surface-muted` | `252 40% 98%` | `252 22% 14%` | Row hover, inset blocks |
| `--ink` | `252 30% 14%` | `250 30% 96%` | Primary text / headings |
| `--ink-muted` | `250 12% 46%` | `250 12% 66%` | Secondary / label text |
| `--hairline` | `252 30% 92%` | `252 20% 22%` | Borders / dividers |
| `--brand`/`-violet` | `252 83% 67%` | (shared) | Primary violet |
| `--brand-indigo` | `248 84% 60%` | (shared) | Gradient end / heading color |
| `--brand-soft` | `252 90% 96%` | `252 50% 18%` | Lilac chips, hover wash, avatar bg |
| `--positive` | `152 62% 40%` | `152 56% 50%` | Income / available credit |
| `--positive-soft` | `152 70% 95%` | `152 40% 16%` | Income chip background |
| `--negative` | `349 78% 56%` | `349 80% 64%` | Expense / destructive |
| `--negative-soft` | `349 90% 96%` | `349 40% 18%` | Expense chip background |
## Radius, shadow, font scale
| Token | Value |
| --------------- | ---------------------------------------------- |
| `--radius` (lg) | `1.375rem` (~22px) — card baseline |
| `radius-xl/2xl` | `+6px` / `+12px` — sidebar, hero panels |
| `radius-md/sm` | `-6px` / `-10px` |
| `--shadow-soft` | resting card shadow (low, violet-tinted) |
| `--shadow-lift` | hover / floating panels / menus |
| `--shadow-glow` | violet glow under gradient CTAs |
| `--font-sans` | `var(--font-inter), ui-sans-serif, system-ui…` |
## Helper classes (in `@layer components`)
- `.brand-gradient` — the 135° violet→indigo gradient (buttons, card, CTAs).
- `.brand-text-gradient` — same gradient clipped to text.
- `.glass-surface` — translucent surface + backdrop blur (floating sidebar, cards).
- `.section-heading` — medium-weight violet/indigo section title.
## Layout
- **Floating icon rail** (`src/components/layout.tsx`) — ~72px, white glass,
rounded, soft-shadowed, detached from the screen edge. Brand mark at top; nav
icons (Dashboard / Credit Cards / Team) with a violet **gradient active
state**; theme toggle, user switcher, and a help "?" pinned at the bottom.
Role gating (Team only for admins) and the `useAgentContext` page readable are
unchanged.
- **Dashboard** (`src/app/dashboard/page.tsx`, mirrored by `/cards`) — two
columns on a lavender canvas:
- **Left**: "My Cards" (dashed add-card tile + a vivid gradient credit card,
with a second card peeking behind) and "Recent Transactions" with a
"View All" link, **underline** ALL / INCOME / EXPENSES tabs, a "TODAY"
chip, and transaction rows (circular tinted badge + title + subtitle +
colored amount).
- **Right rail**: a tall rounded panel — **Balance** (large bold) + masked
card number; an Income / Expenses split with colored arrows; a divider;
**Last Payment Details**; a **Statistics** sparkline; and a gradient pill
**New Transaction →** CTA.
## Statistics chart
`src/components/statistics-chart.tsx` is a **hand-rolled inline SVG** area+line
sparkline — no charting dependency. It takes a numeric series (derived from real
transaction data, bucketed by month; falls back to representative seeded points
when there isn't enough data), draws a violet→indigo gradient stroke over a soft
gradient area fill, emphasizes the latest point, and labels the axis with the
final point highlighted in violet. It includes an `aria-label` and an
`sr-only` numeric summary.
## Credit-card visual
`src/components/card-visual.tsx` exports `GradientCreditCard` (the vivid violet
gradient face: EMV chip, masked `•••• •••• •••• last4`, holder, valid-thru, and a
brand mark) plus `VisaWordmark` and the overlapping-circles `MastercardMark`. A
`subtle` variant renders the dimmed card that peeks behind the active one.
## CopilotKit chat popup
The embedded `CopilotPopup` is themed **via CSS only** — no component props
were changed. The v2 chat scopes its shadcn-style tokens to `[data-copilotkit]`;
`globals.css` re-points `--primary` / `--primary-foreground` / `--ring` on that
selector (and its dark variant) to the brand violet, so the send button, focus
rings, and links match. This is additive and degrades gracefully if the SDK's
internals change. All chat wiring (provider, threads, suggestions, HITL) is
untouched.
## Accessibility
- Focus-visible rings (`ring-brand` with offset) on buttons, tabs, inputs, menu
items, and the help/nav controls.
- Semantic `<button>` / `<nav>` / `aria-current="page"` on the active nav item;
`aria-label`s on icon-only controls; `role="img"` + `aria-label` on the chart.
- Color pairings (ink on surface, brand-foreground on gradient, positive/negative
on their soft backgrounds) target legible contrast in both themes.
examples/showcases/banking/docs/teach-mode/LEARNING-TRACK-PLAN.md
+548
View File
@@ -0,0 +1,548 @@
# Learning-Track Plan — make the teach-mode loop actually learn in the banking demo
> **What this is.** A design doc (not an implementation) for closing the loop on the
> banking teach-mode demo: human demonstrates the over-limit unlock → the demonstration is
> recorded on the thread → distilled into `/knowledge` → a fresh agent performs the unlock
> unaided. Plus the three pieces of teachable-demo UX (suggested prompt, inline HITL,
> recording vignette) that make the loop legible on screen.
>
> **Owner:** jerel@copilotkit.ai · **Date:** 2026-06-05 · **Status:** Draft for review
>
> **Repos in play (read both):**
>
> - **Banking demo (canonical, OSS):** `CopilotKit/examples/showcases/banking` — Next.js App
> Router, CopilotKit v2 hooks, `workspace:*` packages (react-core 1.59.2, **no recording
> hook**). This is where the gate/unlock/framing/UX live and verify today.
> - **Intelligence repo (the backend + the real hook):** `cpk-intelligence-banking` —
> Nx monorepo with `apps/app-api`, `apps/realtime-gateway`, `apps/sl-worker`, and
> `demos/{e-commerce,banking,…}`. Its root `package.json` pins
> `@copilotkit/react-core@e103a19` (the build that **exports** the recording hook).
> - **Cookbook contract:** `./README.md` (the 5-role teachable loop) + the recording seam at
> `../../src/lib/record-user-action.ts` (a no-op shim today) + `./verify-teachable-gate.sh`
> (the backend-independent REST proof).
---
## 0. TL;DR
The **left half** of the teachable loop (gate → symptom → agent framing → human unlock → a
`recordUserAction(...)` call site) is built and verifiable in the banking demo today. The
**right half** (recording actually streams → distill → `/knowledge` → fresh agent learns)
is gated on two blockers:
- **Blocker 2a — the recording hook.** `useRecordUserActionInCurrentThread` does not exist
in the OSS `@copilotkit/react-core/v2` build the banking demo currently resolves
(`workspace:*` → 1.59.2). It exists at CopilotKit commit `e103a19`. Clearing it = pin that
build + a one-line import swap of the `record-user-action.ts` shim.
- **Blocker 2b — the Intelligence backend.** The runtime route is already env-gated on
`INTELLIGENCE_API_URL` / `INTELLIGENCE_GATEWAY_WS_URL` / `INTELLIGENCE_API_KEY`. Clearing
it = stand up the Intelligence stack (`app-api` + `realtime-gateway` + `sl-worker` →
`/knowledge`) locally or hosted, and point those three env vars at it.
The three teachable-demo UX pieces — **suggested prompt**, **inline HITL tool-call card**,
**pulsating recording vignette** — can be built TODAY against the no-op shim and a local
`recording` flag, because none of them needs the backend to render. They form the
buildable-now track (FOR-148). The live learn/distill/retrieve loop is the blocked track
(FOR-146/147/149). FOR-145 is the fresh-agent verification harness.
Recommended sequencing: **UX shell now (FOR-148)** in parallel with **standing up the
backend (FOR-147)**; then **pin the hook + swap the import (FOR-146)**; then the
**fresh-agent proof (FOR-145/149)** ties it together.
---
## 1. The loop, end to end — concretely, against THIS demo
The banking entities are: `transaction` (the gated write is _approve_), `expense-policy`
(the limit that blocks it), `policy-exception` (the unlock record), and a
`policy-exception-code` catalogue (justifying vs decoy vs invalid).
| Step | What happens | THIS demo's concrete entities + file |
| ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **1. Agent A tries the obvious write** | A fresh agent is asked to approve an over-limit transaction. It calls the approve write. | The agent prompt in `src/app/api/copilotkit/[[...slug]]/route.ts` (`bankingAgent`) lists the tools but withholds the unlock recipe. The approve path is the `showAndApproveTransactions` HITL in `src/app/page.tsx` → `changeTransactionStatus` (`src/app/actions.ts`) → `PUT /api/v1/transactions/[id]`. |
| **2. Gate fails, symptom-only** | The PUT returns **422 `OVER_POLICY_LIMIT`**, message `"<policy> policy limit exceeded"`. It names the _problem_ (over limit), never the _fix_ (policy exception). | `src/app/api/v1/transactions/[id]/route.ts` (the `patch.status === "approved" && !isWithinPolicyLimit && !hasApprovedException` branch). Rules in `src/lib/store.ts`: `isWithinPolicyLimit` / `hasApprovedException` / `canApprove`. Seed: `t-1` (Google Ads, −5000, Marketing limit 5000/spent 500) ⇒ over limit. |
| **3. Agent stops (framing holds)** | Per the **ACTION DISCIPLINE** clause, the agent does not improvise. It reports the failure and asks the human how to proceed. It does NOT fire a distractor (`sendSpendAlert` / `requestCardReplacement` / `flagForReview`). | Prompt + distractor tools in the runtime route and `src/app/page.tsx` (the three `useFrontendTool` no-op distractors). This is the **control**: pre-learning, a correctly-framed agent cannot pass. |
| **4. Human demonstrates the unlock** | A human opens a policy exception under a **justifying** code (e.g. `EXC-BOARD-APPROVED`), finalizes it (auto-approves + links `activeExceptionId`), then re-approves — now **201**. | Today: `PolicyExceptionModal` (`src/components/policy-exception-modal.tsx`) opened from the over-limit row in `src/components/transactions-list.tsx`. Catalogue: `src/app/api/v1/policy-exception-codes.ts` (`JUSTIFYING_EXCEPTION_CODES` = BOARD-APPROVED / CONTRACTUAL-COMMITMENT / EMERGENCY-SPEND; decoys = WILL-REIMBURSE / ONE-TIME). REST: `exceptions/route.ts` + `exceptions/[id]/finalize/route.ts`. **This plan moves that flow inline into the chat — see §3.2.** |
| **5. Each mutation is recorded on the thread** | After `open()` and after `finalize()`, the UI calls `recordUserAction({title, description, previousData, newData, metadata})`. `previousData` carries the gated flags (`approvePermitted: false`); `newData` the unlocked effect (flipped flags + the linking exception id); `metadata` the `transactionId`. | Two existing calls in `policy-exception-modal.tsx` (`policy_exception.opened`, `policy_exception.finalized`) + two in `transactions-list.tsx` (`transaction.approved`, `transaction.denied`). The import is the no-op shim `@/lib/record-user-action` **today** — see Blocker 2a. |
| **6. Events stream to the gateway** | With the real hook + Intelligence runtime, every AG-UI event of the run (including the recorded user actions) streams over the Phoenix WebSocket to the Intelligence gateway, scoped to the current user + thread. | The env-gated `CopilotKitIntelligence({apiUrl,wsUrl,apiKey})` branch of `createRuntime()` in the runtime route; `identifyUser` maps `properties.userRole` → a stable `northwind-<role>` id so threads + knowledge are scoped consistently. Reference: `cpk-intelligence-banking/demos/e-commerce/bff/.../main.ts`. |
| **7. Distilled into `/knowledge`** | The `sl-worker` sweeps the recorded actions and an LLM writer distills a reusable procedure: _"to approve an over-policy-limit transaction, open a policy exception under a justifying code (board-approved / contractual / emergency), finalize it, then approve."_ It lands in `/knowledge` (shared per org+project). | `apps/sl-worker` in the Intelligence repo (gated on `SL_ENABLED=true`); writes `cpki.knowledge_base_files` exposed to agents as `/knowledge`. |
| **8. Agent B (fresh) learns + succeeds** | In a NEW thread with no memory of the human, the agent is asked the same over-limit approval. It greps `/knowledge` (via the `copilotkit_knowledge_base_shell` tool), discovers the procedure, files a _justifying_ exception, finalizes it, approves → **201** — no human help, nothing added to the prompt. | Same `bankingAgent` prompt (still recipe-free) reading `/knowledge`. This is the **proof of learning** — see §5. |
The contrast in step 5 (`previousData` gated flags vs `newData` unlocked flags) is the signal
the distiller turns into the procedure — which is why the flag names must stay stable across
`open → finalize`. This invariant is already honored in both call sites; do not break it.
---
## 2. The two blockers + how to clear them, in order
### 2a. The recording hook (FOR-146) — pin the build, swap one import
**Current state (verified).**
- Banking demo `package.json` pins CopilotKit packages as `workspace:*`; the installed
`@copilotkit/react-core` is **1.59.2**, whose `v2` hooks index exports
`useFrontendTool` / `useHumanInTheLoop` / `useAgent` / `useThreads` / `useComponent` /
`useConfigureSuggestions` — but **not** `useRecordUserActionInCurrentThread`
(`grep` of `node_modules/@copilotkit/react-core/dist/v2/` returns nothing).
- So all four call sites import the **no-op shim** at `src/lib/record-user-action.ts`. The
shim returns a `recordUserAction` that only `console.debug`s in dev and resolves — it
records nothing. The call-site bodies are already byte-for-byte what the real hook expects.
- The Intelligence repo's root `package.json` pins the hook-bearing build:
`"@copilotkit/react-core": "https://pkg.pr.new/CopilotKit/CopilotKit/@copilotkit/react-core@e103a19"`
(and the matching `core` / `runtime` / `shared` / `sdk-js` at `e103a19`). The e-commerce
demo there imports `useRecordUserActionInCurrentThread` directly from
`@copilotkit/react-core/v2` and it resolves — proving `e103a19` exports it.
**The unblock, exactly.**
1. **Land/pin a react-core build that exports the hook.** Two options:
- **(A) Pin the published pkg.pr.new build** (matches the Intelligence repo). In the
banking demo `package.json`, replace the four `workspace:*` CopilotKit entries with the
`e103a19` pins (at minimum `@copilotkit/react-core`, plus `@copilotkit/core`,
`@copilotkit/runtime`, `@copilotkit/shared` to keep the runtime route's
`CopilotKitIntelligence` / `BuiltInAgent` imports on the same line). Re-install.
_Cost:_ the demo leaves the OSS monorepo `workspace:*` graph; lockfile churn. Best when
the demo is being **vendored into the Intelligence repo** (where these pins already
exist — see §6).
- **(B) Land the recording hook into the OSS `react-core/v2` build** the demo's
`workspace:*` already resolves, then bump. _Cost:_ a real OSS change; out of scope for
this demo plan but the cleaner long-term home. Treat as a CopilotKit-core ticket.
2. **The one-line import swap** at each of the **four** call sites. Change ONLY the import;
every `recordUserAction({...})` body and the `UserActionRecord` type stay identical:
```ts
// before — no-op shim (banking today):
import { useRecordUserActionInCurrentThread } from "@/lib/record-user-action";
// after — real hook (e-commerce already does this):
import { useRecordUserActionInCurrentThread } from "@copilotkit/react-core/v2";
```
Call sites to edit: `src/components/policy-exception-modal.tsx` (line 10),
`src/components/transactions-list.tsx` (line 15) — plus the two inline-HITL components this
plan introduces in §3.2 (which inherit the same import). **Recommended variant:** instead
of editing imports, turn the shim into a **re-export** so zero call sites change:
```ts
// src/lib/record-user-action.ts (after the hook ships):
export { useRecordUserActionInCurrentThread } from "@copilotkit/react-core/v2";
export type { UserActionRecord } from "@copilotkit/react-core/v2"; // if the type is exported there
```
This is the smallest possible diff and keeps the "copy this file verbatim" cookbook story
intact.
**Definition of done for 2a:** the real hook resolves; `recordUserAction(...)` calls stream
to the runtime instead of `console.debug`; no call-site body changed.
### 2b. The Intelligence backend (FOR-147) — stand up record→distill→`/knowledge`
**Current state (verified).** The runtime route already env-gates the backend:
```ts
const intelligenceEnabled = Boolean(
INTELLIGENCE_API_URL && INTELLIGENCE_GATEWAY_WS_URL && INTELLIGENCE_API_KEY,
);
// enabled → new CopilotRuntime({ agents:{default:bankingAgent}, intelligence, identifyUser })
// missing → new CopilotRuntime({ agents:{default:bankingAgent}, runner: new InMemoryAgentRunner() })
```
So no route code changes to _turn on_ the backend — it is purely a deploy + env exercise.
What each var points at:
| Env var | Points at | Local-dev value | Notes |
| ----------------------------- | -------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `INTELLIGENCE_API_URL` | `apps/app-api` HTTP — the `/knowledge` + threads + user_actions store | `http://localhost:7050` (`APP_API_PORT` default in `scripts/local-dev.sh`) | The durable backend the gateway writes to and `/knowledge` is read from. |
| `INTELLIGENCE_GATEWAY_WS_URL` | `apps/realtime-gateway` Phoenix WebSocket — where AG-UI run events (incl. recorded actions) stream | `ws://localhost:7053` (`REALTIME_GATEWAY_PORT` default) | The live ingestion seam. |
| `INTELLIGENCE_API_KEY` | the org/project key (scopes threads + `/knowledge`) | the seeded `cpk_…` key for `casa-de-erlang` (e-commerce uses `cpk_sPRVSEED_seed0privat0longtoken00`) | Must belong to an org whose `cpki.users` includes the demo identities `identifyUser` mints (`northwind-<role>`), or those users must be seeded. |
| `COPILOTKIT_LICENSE_TOKEN` | optional, read automatically by the runtime | — | Only if the build requires it. |
| `SL_ENABLED` | gate on the **`sl-worker`** distillation sweep (Intelligence side) | `true` | Without it the worker won't distill — recording streams but `/knowledge` never fills. |
**Standing it up — two paths.**
- **Local-dev (recommended for building the proof).** The Intelligence repo's
`scripts/local-dev.sh` already boots the whole stack with `pnpm nx serve`:
`app-api` (`:7050`), `realtime-gateway` (`:7053`), the per-demo BFFs, and — when
`SL_ENABLED=true` — the `sl-worker` on a heartbeat. It **already registers a `banking`
demo (demo 6: BFF `:7071`, web `:7072`)** alongside e-commerce. So the realistic path to a
live banking loop is to run the demo **inside the Intelligence repo** (where the `e103a19`
pins and the SL services already exist), not to point the standalone Next.js app at a
hand-rolled stack. See §6 for that recommendation. If you keep the standalone Next.js app,
set the three env vars to the local-dev URLs/key above and run `scripts/local-dev.sh` in the
Intelligence repo to provide the backend.
- **Hosted.** Point the three vars at a deployed Intelligence environment (app-api URL,
gateway WSS URL, a real `cpk_…` key) — e.g. a Railway/staging deploy of the Intelligence
apps. Same route code; only env differs. Ensure `SL_ENABLED=true` on that environment's
`sl-worker` and that the key's org has the demo users seeded.
**Definition of done for 2b:** with 2a also done, a recorded human unlock results in a
distilled procedure appearing in `/knowledge` (verifiable by grepping `/knowledge` via the
agent or inspecting `cpki.knowledge_base_files`).
**Ordering.** 2a and 2b are independent to _build_ but both required for the live loop. Do
2b's standup in parallel with the UX shell; 2a is the final flip. The cleanest single move is
§6 (vendor into the Intelligence repo), which clears 2a and 2b together because the pins and
services already live there.
---
## 3. The teachable-demo UX (first-class — explicit feedback)
These three make the loop legible. **All three are buildable today** against the no-op shim
and a local `recording` flag (§4). Map below is to real files in
`CopilotKit/examples/showcases/banking`.
### 3.1 Suggested prompt — a starter pill that kicks off the teachable scenario
**Goal.** The first thing a viewer sees offers a one-click path straight into the over-limit
gate, so the "agent fails → human teaches → agent learns" arc starts without anyone having to
know the domain.
**Where it plugs in.** `src/app/wrapper.tsx` already registers welcome-screen pills via
`useConfigureSuggestions` (the v2 way — there is no `suggestions` prop on the v2 chat
component). The `BankingSuggestions()` component calls it with
`available: "before-first-message"` and three pills today.
**The change.** Add the teachable pill as the **first** suggestion so it leads. Exact copy:
```ts
useConfigureSuggestions({
available: "before-first-message",
suggestions: [
{
title: "Approve the $5,000 Marketing transaction",
message:
"Approve the $5,000 Google Ads transaction on the Marketing policy.",
},
{ title: "View transactions", message: "Show me my recent transactions" },
{ title: "Add a card", message: "Add a new credit card" },
{
title: "Assign a policy",
message: "Assign a spending policy to one of my cards",
},
],
});
```
- The `message` deliberately matches seed `t-1` (Google Ads, −$5,000, Marketing policy, limit
$5,000 / spent $500), so approving it hits `OVER_POLICY_LIMIT` — the gate — every time.
- Keep `title` human and benign ("Approve the $5,000 Marketing transaction"); it must **not**
hint at the exception path (same symptom-only spirit as the gate).
- Pre-learning the agent will fail this correctly (control). Post-learning it will succeed.
The same pill demonstrates both halves of the arc.
**Touch-points:** `src/app/wrapper.tsx` (`BankingSuggestions`). No other file.
### 3.2 Inline HITL — approve/deny + file-exception rendered IN the chat
**Goal.** The human's whole demonstration — see the over-limit symptom, approve/deny, and
_file a policy exception_ — happens **inline in the chat as a tool-call card**, not in a
separate page modal. That's what makes the recorded demonstration feel like "the agent
watched me do it right here."
**What exists today.**
- The approve/deny inline card is **already** an inline HITL: `showAndApproveTransactions`
(`src/app/page.tsx`, ~line 405) renders `<TransactionsList showApprovalInterface>` inside
the chat via `useHumanInTheLoop`'s `render`. That list shows the **over-limit symptom**
("Over policy limit" badge) and a **"File policy exception"** button — but that button
currently opens a **separate page modal** (`PolicyExceptionModal` mounted at the bottom of
`transactions-list.tsx`, a shadcn `Dialog`). That modal is the one piece that breaks the
"inline" story.
- The standalone approve/deny buttons (`src/components/approval-buttons.tsx`) are a shared
primitive reused across every HITL card.
**The design.**
1. **New inline component `PolicyExceptionInline`** (`src/components/policy-exception-inline.tsx`),
modeled on `PolicyExceptionModal` but rendered as a chat card (no `Dialog` chrome — the
same rounded `bg-surface` card the other HITL renders use). It shows:
- the **symptom** ("This transaction is over its Marketing policy limit"),
- the **code picker** (the existing `POLICY_EXCEPTION_CODES` select — labels for humans,
codes persisted),
- **File exception** (calls `openPolicyException` → `finalizePolicyException`, the same
REST callers threaded from `useCreditCards` in `actions.ts`),
- and on success a confirmation + the approve affordance.
It carries the **same two `recordUserAction` calls** (`policy_exception.opened` →
`policy_exception.finalized`) verbatim from the modal — the recording payloads do not
change.
2. **A new inline HITL tool `fileAndApproveOverLimit`** (a `useHumanInTheLoop` in
`src/app/page.tsx`) whose `render` mounts `PolicyExceptionInline`. Description stays
**neutral** (does not name the exception path or which codes justify — preserves the
learning invariant), e.g. _"Resolve a blocked over-limit approval. Requires human
approval."_ This becomes the inline surface the human uses to teach, and later the surface
the learned agent's `openPolicyException` / `finalizePolicyException` calls render into.
3. **Reuse vs replace.**
- **Reuse** `approval-buttons.tsx` unchanged for approve/deny.
- **Reuse** the existing `openPolicyException` / `finalizePolicyException` HITL tools
(`src/app/page.tsx`, ~lines 492 / 549) — they already render inline approve cards; the
learned agent drives the unlock through these. `PolicyExceptionInline` is the
_human-initiated_ twin.
- **Replace** the page-modal entrypoint: drop the `setExceptionTxnId` → `<PolicyExceptionModal>`
branch in `transactions-list.tsx` in favor of rendering `PolicyExceptionInline` within the
chat card. Keep `policy-exception-modal.tsx` only if a non-chat entry is still wanted;
otherwise retire it (its recording payloads move into the inline component).
**Touch-points:** new `src/components/policy-exception-inline.tsx`; new HITL tool +
`render` in `src/app/page.tsx`; edits to `src/components/transactions-list.tsx` (swap the
modal mount for the inline card); reuse `src/components/approval-buttons.tsx`. Reference shape:
`cpk-intelligence-banking/demos/e-commerce/.../incident-create-modal.tsx` (same
open→record→finalize→record pattern).
### 3.3 Pulsating "recording" vignette — a violet edge glow while recording
**Goal.** While the agent is recording the human's demonstrated actions, a soft pulsating
violet glow/vignette hugs the **canvas edges**, signaling "the agent is watching and recording
this for future reference." It turns off when recording ends.
**Trigger / state.** Introduce a tiny **recording context** (`src/components/recording-context.tsx`)
exposing `isRecording` + `beginRecording()` / `endRecording()` (or a ref-counted
`withRecording()` wrapper). Wire it around the **`recordUserAction` calls**: each call site
calls `beginRecording()` immediately before firing the record(s) and `endRecording()` when the
demonstration step settles. Concretely:
- In `PolicyExceptionInline` (§3.2): `beginRecording()` at the start of `handleSubmit`, and
`endRecording()` after the second (`finalized`) record resolves (or in `finally`).
- In `transactions-list.tsx` approve/deny: wrap the `recordUserAction(...)` call the same way.
- Use a small **debounce/min-duration** (e.g. keep it on ≥1200ms) so a fast fire-and-forget
record still produces a visible pulse, and **ref-count** so overlapping records (open +
finalize) don't flicker it off between steps.
> This flag is **independent of the backend**: it reflects "the UI is emitting a record right
> now," which is true even against the no-op shim. So the vignette is fully demoable today
> (FOR-148) and stays correct once the real hook streams (FOR-146).
**Visual treatment.**
- An **edge vignette**: a full-viewport overlay with `box-shadow: inset 0 0 0 …` /
`radial-gradient` mask so color concentrates at the **edges** and fades to transparent in the
center (content stays unobscured).
- **On-brand violet:** use the existing tokens — `--brand-violet` (`hsl(252 83% 67%)`) /
`--brand-indigo` (`hsl(248 84% 60%)`) and the `--shadow-glow` feel
(`0 12px 30px hsl(252 83% 60% / 0.35)`). Keep alpha low (~0.25–0.4) so it reads as a glow,
not a wash.
- **Pulse animation:** a 2–3s `@keyframes` easing opacity/blur between two low values
(e.g. 0.25 ↔ 0.45), `ease-in-out`, infinite while recording. Fade in/out over ~200ms on
enter/leave so it doesn't snap.
- **Non-blocking:** `position: fixed; inset: 0; pointer-events: none; z-index` above content
but below modals/toasts. It must never intercept clicks.
- **Reduced motion:** under `@media (prefers-reduced-motion: reduce)`, drop the pulse — show a
**static** low-opacity violet edge glow instead (still communicates "recording", no
animation).
**Where it mounts.** A top-level overlay so it frames the whole canvas. Mount
`<RecordingVignette />` inside `CopilotKitWrapper` in `src/app/wrapper.tsx` (a sibling of
`LayoutComponent` / `ChatPanel`, both inside the provider tree so it can read the recording
context). The `RecordingProvider` wraps the same subtree. Add the keyframes + `.recording-vignette`
styles to `src/app/globals.css` (which already owns the brand tokens and `.brand-gradient`).
**How it turns off.** When `endRecording()` drops the ref-count to zero (after the min-duration
elapses), `isRecording` flips false; the overlay fades out over ~200ms and the animation stops.
**Touch-points:** new `src/components/recording-context.tsx`; new `src/components/recording-vignette.tsx`;
keyframes/classes in `src/app/globals.css`; mount + provider in `src/app/wrapper.tsx`; `begin/endRecording`
wrapping in `policy-exception-inline.tsx` and `transactions-list.tsx`.
---
## 4. Buildable NOW vs blocked — the explicit split
| Piece | Ticket | Needs the real hook? | Needs the Intelligence backend? | Buildable today? |
| ------------------------------------------------- | ------- | ---------------------------------------------------------------------------------- | -------------------------------------------------------------- | ------------------------------------------------------------- |
| Suggested prompt (§3.1) | FOR-148 | No | No | **Yes** — pure `useConfigureSuggestions` copy. |
| Inline HITL card (§3.2) | FOR-148 | No (renders the demonstration UI; recording payloads already present via the shim) | No | **Yes** — renders + drives REST unlock; records via the shim. |
| Recording vignette (§3.3) | FOR-148 | No (reads a local `recording` flag set around the record calls) | No | **Yes** — flag is true even against the no-op shim. |
| Recording actually streams (role #3 beyond no-op) | FOR-146 | **Yes** (pin `e103a19` + import swap) | Indirectly (events have somewhere to go) | No — blocked on 2a. |
| Distill → `/knowledge` (role #5) | FOR-147 | — | **Yes** (`app-api` + gateway + `sl-worker`, `SL_ENABLED=true`) | No — blocked on 2b. |
| Fresh-agent learns + succeeds | FOR-149 | Yes | Yes | No — needs 2a + 2b. |
| Fresh-agent verification harness | FOR-145 | The script half works today (§5); the learning half needs 2a+2b | Partial | The REST proof: **yes**. The learning proof: blocked. |
**Why the UX is safe to build first.** The three UX pieces only depend on (a) the v2 chat +
HITL APIs the demo already uses, and (b) a local `recording` boolean. The no-op shim already
keeps the `recordUserAction` call sites real and stable, so wrapping them in `begin/endRecording`
and rendering inline cards is wiring that does not change when the real hook lands — at that
point the same calls simply also stream. **No rework.**
### Recommended sequencing
1. **FOR-148 — UX shell (now, parallelizable).** Suggested prompt → inline HITL card →
recording vignette. Verifiable visually + via `verify-teachable-gate.sh` (the REST contract
is unaffected). Ship this regardless of backend timing — it makes the demo legible today.
2. **FOR-147 — backend standup (now, parallel to FOR-148).** Bring up `app-api` + gateway +
`sl-worker` (local-dev or hosted) and confirm the three env vars + `SL_ENABLED`. No demo
code changes (the route is already gated).
3. **FOR-146 — pin the hook + swap the import (after 147 so streaming has a destination).**
Pin `e103a19` (or re-export shim) → recorded actions now stream.
4. **FOR-149 / FOR-145 — fresh-agent proof.** With 146+147 live, run the learning proof (§5)
and codify it in a harness.
**Strong recommendation (§6):** do FOR-146 + FOR-147 by **vendoring the demo into the
Intelligence repo**, where both the `e103a19` pins and the SL services already exist — that
collapses 2a+2b into "run the existing local-dev with the banking demo wired like e-commerce."
---
## 5. Verification strategy — proving learning, not scripting
### 5.1 Backend-independent REST proof (works TODAY) — roles #1 + #2 — FOR-145 (lower half)
`verify-teachable-gate.sh` already drives the real REST routes against a running banking dev
server and asserts the full gate→unlock contract. Run it (point `BASE_URL` at the served port):
```bash
BASE_URL=http://localhost:3000 ./verify-teachable-gate.sh # next dev defaults to :3000
```
It asserts, in order:
- **A. GATE** — `PUT /api/v1/transactions/t-1 {"status":"approved"}` → **422
`OVER_POLICY_LIMIT`**, and the body does **not** mention the exception/unlock path
(symptom-only invariant).
- **B. UNLOCK** — open `EXC-BOARD-APPROVED` on `t-1` → **201** → finalize → **200 approved** →
re-approve `t-1` → **201** (gate lifted by a justifying code).
- **C. DECOY** — `EXC-WILL-REIMBURSE` on `t-3` files + finalizes (**201/200**) but the approve
stays **422** (decoy does not justify).
- **D. CATALOGUE** — an invalid code → **422 `INVALID_EXCEPTION_CODE`**, body does **not**
enumerate the catalogue (non-enumeration invariant).
This proves the gate is real and the unlock is discriminating — i.e. there is genuinely
_something to learn_ — without any Intelligence backend. It is the control that the demo isn't
faked. Re-run from a fresh server to reseed (in-memory store).
### 5.2 The fresh-agent proof (activates after 2a + 2b) — roles #3 + #5 — FOR-149
This is the proof the loop **learned**, not that REST works. Requires the real hook (2a) and
the env-gated `CopilotKitIntelligence` backend with `SL_ENABLED=true` (2b).
1. **Baseline (control).** Fresh thread, ask: _"Approve the $5,000 Google Ads transaction on
the Marketing policy."_ With the recipe-free prompt + ACTION DISCIPLINE intact, the agent
hits the gate, has no procedure, and **reports the failure** instead of firing a distractor.
_This failure is the control — record it._
2. **Human teaches.** Open the inline policy-exception card (§3.2), pick a **justifying** code,
file + finalize. Each step fires `recordUserAction(...)` on the current thread — now a real
stream (2a), and the vignette (§3.3) confirms recording is live on screen.
3. **Distill.** The `sl-worker` (2b, `SL_ENABLED=true`) distills the recorded actions into a
reusable procedure in `/knowledge`. _Spot-check:_ grep `/knowledge` (via the agent or
`cpki.knowledge_base_files`) and confirm the over-limit/policy-exception procedure exists.
4. **Fresh agent succeeds unaided.** A **new** thread (and ideally a different seeded user, to
prove cross-thread/cross-user transfer), same approval request. The agent greps `/knowledge`,
files a _justifying_ exception, finalizes, approves → **201** — **no human help, nothing
added to the prompt.**
**Pass criteria:** step 1 fails, step 4 succeeds, and the **only** thing that changed between
them is the distilled `/knowledge`. That delta is the learning. Codify as the FOR-145 harness:
the REST proof (§5.1) gates "is there something to learn," the fresh-agent run gates "did it
learn it."
**Anti-cheat checks (keep the proof honest):**
- The agent prompt at step 4 is **byte-identical** to step 1 (recipe still withheld — diff the
runtime route prompt).
- A run that files a **decoy** code must still fail (the agent must learn _which_ codes justify,
not just "file an exception").
- Distractor tools must remain harmless no-ops (a "success" from `sendSpendAlert` must not be
mistaken for clearing the gate).
---
## 6. Recommended path: vendor into the Intelligence repo (clears 2a + 2b together)
The standalone Next.js banking demo can _render_ the full UX today, but the **live learning
loop's natural home is the Intelligence repo**, because:
- it already pins `@copilotkit/react-core@e103a19` (the hook-bearing build) — **2a is free
there**;
- it already runs `app-api` + `realtime-gateway` + `sl-worker` via `scripts/local-dev.sh`, and
already registers a **`banking` demo (demo 6)** beside e-commerce — **2b is free there**;
- the e-commerce demo there is a working reference for the exact wiring: `BuiltInAgent` +
`CopilotKitIntelligence` in the BFF, `identifyUser` from a user header, domain tools in the
browser via `useFrontendTool`, and the **real** `useRecordUserActionInCurrentThread` import.
This aligns with the existing vendor plan
(`docs/superpowers/plans/2026-06-02-vendor-canonical-saas-demo-sl-threads.md`): port the
banking React surface (pages, the inline HITL card, the vignette, suggestions) into
`demos/banking/react`, keep the prompt + Intelligence wiring in `demos/banking/bff`, and the
two blockers dissolve into "run the local-dev that's already there." The standalone Next.js
app remains the OSS-verifiable artifact for roles #1/#2/#4 + the UX shell; the Intelligence
copy is where roles #3/#5 actually run.
If vendoring is deferred, the standalone demo can still close the loop by (A) pinning `e103a19`
in its own `package.json` and (B) pointing its three env vars at an Intelligence backend
(local-dev in the Intelligence repo, or hosted) — but that hand-wires what the vendor path
gives for free.
---
## 7. File / component touch-point summary
**Edit (UX shell, FOR-148 — buildable now):**
- `src/app/wrapper.tsx` — add the teachable suggestion pill (§3.1); mount `RecordingProvider`
- `<RecordingVignette />` (§3.3).
- `src/app/page.tsx` — add the `fileAndApproveOverLimit` inline HITL tool whose `render` mounts
`PolicyExceptionInline` (§3.2).
- `src/components/transactions-list.tsx` — swap the page-modal entry for the inline card; wrap
the approve/deny `recordUserAction` calls in `begin/endRecording` (§3.2, §3.3).
- `src/app/globals.css` — `.recording-vignette` + `@keyframes` + reduced-motion variant (§3.3).
**Create (UX shell, FOR-148):**
- `src/components/policy-exception-inline.tsx` — inline version of the file-exception flow
(carries the two existing recording payloads verbatim).
- `src/components/recording-context.tsx` — `isRecording` + `begin/endRecording` (ref-counted,
min-duration).
- `src/components/recording-vignette.tsx` — the edge-glow overlay.
**Edit (unblock the loop):**
- `package.json` — pin `@copilotkit/react-core` (+ core/runtime/shared) to `e103a19` (FOR-146,
option A) — or do this via vendoring (§6).
- `src/lib/record-user-action.ts` — turn into a re-export of the real hook (FOR-146) so no call
site changes.
- **Env only (no code):** set `INTELLIGENCE_API_URL` / `INTELLIGENCE_GATEWAY_WS_URL` /
`INTELLIGENCE_API_KEY` (+ backend `SL_ENABLED=true`) (FOR-147). The runtime route
(`src/app/api/copilotkit/[[...slug]]/route.ts`) already branches on these.
**Reuse unchanged:** `src/components/approval-buttons.tsx`; the existing `openPolicyException`
/ `finalizePolicyException` HITL tools in `page.tsx`; the gate route; the catalogue; `store.ts`;
`verify-teachable-gate.sh`.
**Reference (read-only):** `cpk-intelligence-banking/demos/e-commerce/{bff/.../main.ts,
react/.../order-actions-bar.tsx, react/.../incident-create-modal.tsx}` and
`scripts/local-dev.sh`.
---
## 8. Risks / open questions
- **Hook parity at `e103a19`.** Confirm the published `react-core@e103a19` `v2` index exports
`useRecordUserActionInCurrentThread` _and_ the `UserActionRecord` type (e-commerce imports the
hook; verify the type export before relying on it in the re-export shim — otherwise keep the
local type).
- **Pin drift vs `workspace:*`.** Pinning the demo to `e103a19` takes it off the OSS monorepo
graph; the runtime route imports `CopilotKitIntelligence` / `BuiltInAgent` from
`@copilotkit/runtime/v2`, so pin runtime to the same commit to avoid a split-brain build.
Vendoring (§6) sidesteps this.
- **Identity scoping.** `identifyUser` mints `northwind-<role>`; the `INTELLIGENCE_API_KEY`'s
org must have those users seeded (e-commerce seeds four users in `casa-de-erlang`). Mismatch
= threads/knowledge land under an unexpected scope and the fresh-agent retrieval misses.
- **Writer non-determinism.** The LLM distiller may phrase `/knowledge` differently run to run;
keep a deterministic fallback note for scripted live demos, and assert on _behavior_ (the 201) not on the knowledge text.
- **Vignette over modals.** Ensure the overlay's `z-index` sits above page content but **below**
HITL cards/toasts, and `pointer-events: none` everywhere, so it never blocks the approve
buttons the human needs during recording.
- **Inline modal retirement.** Decide whether to fully retire `policy-exception-modal.tsx` or
keep it as a non-chat entry; if retired, ensure its recording payloads are preserved exactly
in `policy-exception-inline.tsx` (the distiller depends on the stable flag names).
---
## 9. Ticket map (suggested)
| Ticket | Scope |
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| **FOR-145** | Verification harness: REST gate proof (works today) + fresh-agent learning proof (activates post-146/147). |
| **FOR-146** | Recording hook unblock: pin `e103a19` (or re-export shim) + one-line import swap. |
| **FOR-147** | Intelligence backend standup: `app-api` + gateway + `sl-worker` (`SL_ENABLED=true`); wire the three env vars (local-dev or hosted). |
| **FOR-148** | Teachable-demo UX shell (buildable now): suggested prompt + inline HITL card + recording vignette. |
| **FOR-149** | Close the loop: with 146+147 live, demonstrate fresh-agent success unaided and capture it in the harness. |
examples/showcases/banking/docs/teach-mode/README.md
+405
View File
@@ -0,0 +1,405 @@
# teach-mode cookbook
A reusable recipe for building **self-learning, teachable** CopilotKit demos.
"Teach mode" is the loop where an agent **fails a task it was never told how to
do**, a human **demonstrates** the workaround in the UI, that demonstration is
**recorded → distilled → written to `/knowledge`**, and a **fresh agent then
succeeds unaided**. The agent didn't have the recipe prompt-stuffed in; it
_learned_ it from watching a person.
This loop is already implemented **identically** in two demos — only the domain
entities differ. This cookbook documents the contract they share so a **third
demo is a copy-and-reskin**, not a redesign.
| Demo | Path | Entities |
| -------------------------- | ------------------------------------------- | ----------------------------------------------------- |
| **Banking** (canonical) | `examples/showcases/banking` | `transaction` / `expense-policy` / `policy-exception` |
| **E-commerce** (reference) | `cpk-intelligence-banking/demos/e-commerce` | `order` / `refund` / `incident-report` |
> Paths in this doc are repo-relative to `CopilotKit/` for banking, and to
> `cpk-intelligence-banking/` for e-commerce.
---
## (a) What teach-mode is — the teachable loop
The whole demo turns on one asymmetry: **the agent is given the goal and the
tools, but NOT the procedure.** A gate blocks the obvious write with a
symptom-only error. A human knows the unlock and performs it in the UI. That
human action is captured and distilled into knowledge. A later agent reads the
knowledge and clears the same gate on its own.
```
┌─────────────────────────────────────────────┐
│ Agent A (knows the goal + tools, NOT the │
│ procedure) tries the obvious write │
└───────────────────────┬─────────────────────┘
│
▼
role #1 GATE ──► write FAILS with a SYMPTOM-ONLY error ─────────────┐
("<policy> policy limit exceeded" — 422) │
names the PROBLEM, never the FIX │
▼
role #4 AGENT FRAMING: prompt withholds the recipe + ships DISTRACTOR │
tools + ACTION DISCIPLINE ──► agent CANNOT bluff its way past; │
it stops and reports. │
▼
role #2 UNLOCK: a HUMAN performs the multi-step workaround in the UI │
file a record under a JUSTIFYING code → finalize → link to entity │
(DECOY codes file but don't justify; INVALID codes are rejected) │
│
┌─────────────────────────────────────────┘
▼
role #3 RECORDING: each human UI mutation is captured on the CURRENT
thread via useRecordUserActionInCurrentThread()
recordUserAction({ title, description, previousData, newData, metadata })
previousData = the gated flags · newData = the unlocked effect
│
▼
role #5 KNOWLEDGE BACKEND: writer agent DISTILLS the recorded actions
────► /knowledge (a reusable "to clear this gate, do X" procedure)
│
▼
┌─────────────────────────────────────────────┐
│ Agent B (FRESH, no memory of A) retrieves │
│ /knowledge and clears the SAME gate UNAIDED │ ◄── proof of LEARNING
└─────────────────────────────────────────────┘
```
The left half (gate → symptom → framing → human unlock → recording call) works
and is **verifiable today** with no Intelligence backend. The right half
(distill → `/knowledge` → fresh agent) activates when the self-learning
react-core + Intelligence runtime are wired — see role #5 and the honest
backend-block note in **(e)**.
> **Banking's narrated dashboard variant (PR #5266).** On top of this contract,
> the banking demo drives the unlock as an _agent-orchestrated, narrated_ loop.
> When asked to approve an over-limit charge it has no saved procedure for, the
> agent declines ("I don't have a saved way to approve an over-limit charge yet")
> and offers to record (`offerWorkflowRecording`) — no approval card is shown.
> The officer demonstrates the unlock on the real **/dashboard → Transactions →
> Pending approval** view (file a justifying exception, then approve) while a
> waiting card (`awaitDashboardDemonstration`) holds the chat; the agent then
> summarizes and saves the procedure (`saveLearnedWorkflow`) and, on a later
> request, applies it itself to a _different_ over-limit charge
> (`openPolicyException` → `finalizePolicyException` → `approveTransaction`).
> Because the demonstration happens on a different route, these teach/recall HITL
> tools are registered **globally** in `src/components/copilot-context.tsx` (not
> in a page component) so they survive navigation — a route-scoped registration
> unmounts mid-run and the followUp never fires. Same-session recall works by
> echoing the saved procedure back into the thread; the cross-thread `/knowledge`
> proof still requires the backend (role #5).
>
> The waiting card ("Recording your workflow") stays **non-directional** — it
> never lists the steps ("go ahead and do it yourself now and I'll watch and
> learn"), since the point is the agent doesn't yet know how. The card embeds a
> live **recorder feed** (`RecordingSteps` in `src/components/recording-feed.tsx`,
> fed by `logStep` from the nav / tab / file-exception / approve call sites) that
> narrates each officer action as it happens ("Opened Dashboard" → "Filed the
> policy exception" → "Approved the charge"). It renders INSIDE the chat card (a
> child component subscribed to the recording context, so it updates live without
> a stale-closure dep), reading consistently with the other cards rather than as
> a floating overlay. `saveLearnedWorkflow`'s tool result is
> **directive** so the model renders the Save card instead of asking "should I
> save this?" in prose (the failure that otherwise leaves the user nothing to
> click). After saving, the agent treats the demonstrated charge as already
> cleared and waits, rather than re-running the fresh procedure on it.
---
## (b) The 5-role contract (with load-bearing invariants)
State each role demo-agnostically. The **invariant** is the part you must not
break when reskinning — it's what makes the demo _prove learning_ rather than
merely _script a workflow_.
### 1. GATE — a write that fails with a SYMPTOM-ONLY error
A normal-looking write (approve, refund, …) is blocked when a domain rule isn't
satisfied. The rejection **names the problem, never the fix**.
> **Invariant.** The error is symptom-only. It may say _"\<policy\> policy limit
> exceeded"_; it must NEVER mention the policy-exception path (or whatever the
> unlock is). Leaking the recipe in the error lets the agent derive it in one
> round-trip and defeats the demo. The gate must also be _liftable_ — it passes
> once the unlock is in place (`isWithinLimit(x) || hasApprovedException(x)`).
### 2. UNLOCK — a discriminating multi-step procedure that lifts the gate
A human (and, post-learning, the agent) lifts the gate by **filing a record
under a JUSTIFYING code → finalizing it → linking it** to the entity. The
catalogue mixes justifying codes with **decoys**, and unknown codes are
**rejected without enumeration**.
> **Invariant.** The procedure is _discriminating_: only JUSTIFYING codes lift
> the gate; DECOY codes file successfully (recorded for history) but do NOT
> justify; INVALID codes are rejected _without listing the valid ones_. The
> agent is **never told which codes justify** — it must learn that from observed
> human flows. (If any code worked, or the catalogue were leaked, there'd be
> nothing to learn.)
### 3. RECORDING surface — human UI mutations captured on the current thread
Every human mutation that advances the unlock is recorded on the current thread
via `useRecordUserActionInCurrentThread()`, called as
`recordUserAction({ title, description, previousData, newData, metadata }).catch(...)`.
> **Invariant.** The record shape is fixed and identical across demos:
> `previousData` carries the **gated capability flags** (e.g.
> `approvePermitted: false`), `newData` the **unlocked effect** (flipped flags +
> linking ids), `metadata` the **domain ids**. `title` is a machine-ish dotted
> event name (e.g. `policy_exception.opened`); `description` is one human
> sentence. The contrast between `previousData` and `newData` is the signal the
> distiller learns from — keep flag names stable across the open→finalize steps.
### 4. AGENT FRAMING — withhold the recipe, ship distractors, enforce discipline
The system prompt lists the unlock's tools but **never the procedure**, and
ships **plausible distractor tools** that look helpful but don't lift the gate.
An **ACTION DISCIPLINE** clause forbids improvising a substitute.
> **Invariant.** A successful unlock must prove **learning, not
> prompt-stuffing**. So: (a) the prompt withholds the unlock recipe; (b) it
> ships distractors (banking: `sendSpendAlert`, `requestCardReplacement`,
> `flagForReview`) so "called a plausible tool" ≠ "cleared the gate"; (c) ACTION
> DISCIPLINE makes the agent stop and report on failure rather than guess. Before
> learning, the correctly-framed agent _cannot_ pass.
### 5. KNOWLEDGE BACKEND — record → distill → `/knowledge` → fresh agent learns
Recorded actions are distilled into `/knowledge`; a fresh agent retrieves it and
succeeds unaided. The runtime is **env-gated**: OSS `InMemoryAgentRunner` by
default, `CopilotKitIntelligence` when configured.
> **Invariant.** The backend is a **swappable seam**, and roles #1–#2 are proven
> _without_ it. **Honest current block:** the OSS `@copilotkit/react-core/v2`
> build does **not** yet export the recording hook, so the recording surface
> (role #3) is a **no-op shim** today — gate/unlock/framing all work and verify,
> but the distill→`/knowledge`→fresh-agent leg is deferred until the
> self-learning react-core ships. The hook exists at CopilotKit commit
> `e103a19` (pinned by the Intelligence repo); adoption is then a **one-line
> import swap** (call sites don't change). See **(e)**.
---
## (c) Worked-example mapping table
Each role → the banking file → the e-commerce file → what you swap for a new demo.
| Role | Banking (`examples/showcases/banking`) | E-commerce (`cpk-intelligence-banking/demos/e-commerce`) | What you swap for a new demo |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **#1 GATE** | `src/app/api/v1/transactions/[id]/route.ts` — PUT returns **422 `OVER_POLICY_LIMIT`** when `status==="approved" && !isWithinPolicyLimit && !hasApprovedException`. Rule fns in `src/lib/store.ts`: `isWithinPolicyLimit` / `hasApprovedException` / `canApprove`. | `react/src/app/data/store.ts` — `processRefund` / `initiateReturn` throw **`REFUND_NOT_PERMITTED` / `RETURN_NOT_PERMITTED`** when `!isWithinRefundWindow && !hasApprovedActiveIncident`. | The gated write + its symptom-only error code. Pick your domain's "blocked action" (publish, ship, escalate…) and the rule that blocks it. |
| **#2 UNLOCK** | Catalogue `src/app/api/v1/policy-exception-codes.ts` (`POLICY_EXCEPTION_CODES`, `JUSTIFYING_EXCEPTION_CODES`, `isValidExceptionCode`, `isJustifying`). REST `src/app/api/v1/exceptions/route.ts` (open, POST) + `src/app/api/v1/exceptions/[id]/finalize/route.ts` (finalize, POST). Store `openPolicyException` / `finalizePolicyException`. | Catalogue `react/src/app/data/incident-codes.ts` (`INCIDENT_CODES`, `REFUND_JUSTIFYING_CODES`, `isValidIncidentCode`). Store `openIncidentReport` / `finalizeIncidentReport`. | The record entity + its code catalogue. Keep 3 justifying + N decoys; keep open→finalize→link; keep the catalogue check that rejects unknown codes **without enumerating**. |
| **#3 RECORDING** | `src/lib/record-user-action.ts` (**no-op shim**) → consumed in `src/components/policy-exception-modal.tsx` (two `recordUserAction` calls: `policy_exception.opened` then `.finalized`). | `@copilotkit/react-core/v2` (**real hook import**) → consumed in `react/src/app/components/incident-create-modal.tsx` (`incident_report.opened` / `.finalized`) and `order-actions-bar.tsx` (`order.refunded` / `order.return_initiated`). | Nothing in the seam itself — copy `record-user-action.ts` verbatim. Swap only the **payload values** (`title`/flags/`metadata`) for your domain. |
| **#4 AGENT FRAMING** | `src/app/api/copilotkit/[[...slug]]/route.ts` — `BuiltInAgent` prompt withholds the unlock recipe; ships distractors `sendSpendAlert` / `requestCardReplacement` / `flagForReview`; has the **ACTION DISCIPLINE** clause. | Same role in the e-commerce runtime route (refund/return tools listed; distractors present; recipe withheld). | The prompt's tool list, your 3 distractor tools, and the ACTION DISCIPLINE clause (reuse the wording — it's domain-neutral). |
| **#5 KNOWLEDGE BACKEND** | Same route — env-gated `CopilotKitIntelligence` (OSS `InMemoryAgentRunner` default) keyed on `INTELLIGENCE_API_URL` / `INTELLIGENCE_GATEWAY_WS_URL` / `INTELLIGENCE_API_KEY`; `identifyUser` scopes threads by role. | Equivalent env-gated Intelligence runtime in the e-commerce app. | Nothing structural — reuse the env-gated `createRuntime()` pattern verbatim; only `agents: { default: <yourAgent> }` changes. |
---
## (d) Adoption checklist — add teach-mode to a new demo
Eight concrete steps. Assumes a CopilotKit demo with an in-memory store and a v2
runtime route already scaffolded.
1. **Pick the gated write + symptom.** Choose the domain action to block
(approve / refund / publish / ship …) and the rule that blocks it. Add the
rule fns to your store (mirror `isWithinPolicyLimit` / `hasApprovedException`
/ `canApprove`). Make the write's route return a **422 with a symptom-only
error code** (mirror `OVER_POLICY_LIMIT` in
`transactions/[id]/route.ts`). **Do not name the unlock in the error.**
2. **Author the code catalogue (role #2).** Create a `*-codes.ts` (mirror
`policy-exception-codes.ts`): a `CODES` array (`{ code, label }`, label for
humans only), a `JUSTIFYING_CODES` set (keep ~3), `isValid*Code`, and
`isJustifying`. Include **decoy** codes that are valid-but-not-justifying.
3. **Add the unlock record + REST.** Add the record entity to your store with
`open*` (validates code via `isValid*Code`, throws on unknown) and
`finalize*` (auto-approves and links `active*Id` to the gated entity, which is
what `hasApprovedException` checks). Expose them over REST (mirror
`exceptions/route.ts` + `exceptions/[id]/finalize/route.ts`) — or as store
calls if your demo is client-side like e-commerce.
4. **Copy the recording seam (role #3).** Copy
[`record-user-action.ts`](../../src/lib/record-user-action.ts) into your `src/lib/`
**verbatim**. It is domain-neutral; do not edit it.
5. **Wire the human UI to record (role #3).** In the modal/bar where the human
performs the unlock, call `useRecordUserActionInCurrentThread()` and emit a
record after each successful mutation. Follow the field convention exactly:
`previousData` = gated flags (`{ approvePermitted: false }`), `newData` =
unlocked effect (flipped flags + linking ids), `metadata` = domain ids,
`title` = dotted event name, `description` = one sentence. Always
`.catch(...)` (fire-and-forget). Mirror `policy-exception-modal.tsx`.
6. **Frame the agent (role #4).** In your runtime route's prompt: list the
unlock's tools but **not the procedure**; add **3 distractor tools** that look
plausible but don't lift the gate; paste the **ACTION DISCIPLINE** clause
(reuse banking's wording). Implement the distractors as harmless no-ops/logs.
7. **Wire the env-gated backend (role #5).** Reuse banking's `createRuntime()`:
build `CopilotKitIntelligence` when `INTELLIGENCE_API_URL` /
`INTELLIGENCE_GATEWAY_WS_URL` / `INTELLIGENCE_API_KEY` are all set, else fall
back to `InMemoryAgentRunner`. Keep `identifyUser` to scope threads by role.
8. **Verify (role #1+#2 today; #5 when the backend lands).** Adapt
[`verify-teachable-gate.sh`](./verify-teachable-gate.sh) to your entity ids
and codes and run it against your dev server. It must show: gate blocks (422)
→ justifying unlock succeeds → decoy stays blocked → invalid code rejected
without leaking the catalogue. Add the fresh-agent learning proof (**(f)**)
once the Intelligence backend is configured.
---
## (e) The RECORDING SEAM contract
The canonical primitive lives in
[`record-user-action.ts`](../../src/lib/record-user-action.ts) — copy it once, never edit
it.
### The `UserActionRecord` shape
```ts
export type UserActionRecord = {
title: string; // machine-ish dotted event name
description: string; // one human sentence
previousData?: unknown; // GATED state (flags that were false)
newData?: unknown; // UNLOCKED effect (flipped flags + ids)
metadata?: Record<string, unknown>; // domain ids ("which")
};
export const useRecordUserActionInCurrentThread =
() =>
(record: UserActionRecord): Promise<void> => {
/* … */
};
```
Call-site convention, verbatim from `policy-exception-modal.tsx` (the e-commerce
modal is identical bar the entity names):
```ts
const recordUserAction = useRecordUserActionInCurrentThread();
// ...after open() succeeds...
recordUserAction({
title: "policy_exception.opened",
description: "Opened a policy exception from the transactions view.",
previousData: { transactionActiveExceptionId: null, approvePermitted: false },
newData: { exceptionId, exceptionStatus: "draft", exceptionCode: code },
metadata: { transactionId: props.transactionId },
}).catch(console.error);
// ...then after finalize() succeeds, a second record flips the flags to the unlocked state.
```
### The no-op shim (current state)
Today the shim **records nothing** — it only `console.debug`s in dev — because
the OSS `@copilotkit/react-core/v2` build does **not** export
`useRecordUserActionInCurrentThread`. Its hooks index exports only
`useFrontendTool` / `useHumanInTheLoop` / `useAgent` / `useThreads` / etc. The
shim exists purely to keep the call sites real and stable.
### The one-line swap to the real hook
When a react-core build exporting the hook is one you can depend on, change
**only the import** at each call site:
```ts
// before — no-op shim (banking today):
import { useRecordUserActionInCurrentThread } from "@/lib/record-user-action";
// after — real hook (e-commerce already does this):
import { useRecordUserActionInCurrentThread } from "@copilotkit/react-core/v2";
```
The `UserActionRecord` type and every call body stay byte-for-byte identical.
(Alternatively, make `record-user-action.ts` re-export the real hook so not even
imports change.)
### Honest backend-block note
- **Works today, no backend:** roles #1 (gate), #2 (unlock + decoy + catalogue),
#4 (framing). Provable via `verify-teachable-gate.sh`.
- **Deferred until the self-learning react-core + Intelligence runtime are
wired:** the recording actually streaming (role #3 beyond the no-op) and the
distill → `/knowledge` → fresh-agent-learns leg (role #5).
- **Known landing point:** the recording hook exists at CopilotKit commit
`e103a19`, which the Intelligence repo pins. The banking demo points its
import at the shim; the e-commerce demo already imports the hook from
`@copilotkit/react-core/v2` — that single import line is the entire difference
between "backend pending" and "backend wired".
---
## (f) Verification recipe
### Backend-independent proof (works TODAY) — roles #1 + #2
Run the bundled script against a running banking dev server. It drives the real
REST routes and asserts the full gate→unlock contract.
```bash
# default base URL is http://localhost:3939 (next dev defaults to :3000 —
# point BASE_URL at whatever port you actually serve)
./verify-teachable-gate.sh
BASE_URL=http://localhost:3000 ./verify-teachable-gate.sh
```
What it asserts (each step commented in the script with the role it exercises):
- **A. GATE (#1)** — `PUT /api/v1/transactions/t-1 {"status":"approved"}` →
**422 `OVER_POLICY_LIMIT`**, and the body does **not** mention the
exception/unlock path (symptom-only invariant).
- **B. UNLOCK (#2)** — `POST /api/v1/exceptions {transactionId:"t-1",
code:"EXC-BOARD-APPROVED"}` → **201** → `POST
/api/v1/exceptions/{id}/finalize` → **200 approved** → re-`PUT` approve `t-1`
→ **201** (gate lifted).
- **C. DECOY (#2)** — same flow on `t-3` with `EXC-WILL-REIMBURSE` files +
finalizes (**201/200**) but the approve stays **422 `OVER_POLICY_LIMIT`**.
- **D. CATALOGUE (#2)** — `POST /api/v1/exceptions {code:"EXC-…NOT-REAL"}` →
**422 `INVALID_EXCEPTION_CODE`**, and the body does **not** enumerate any real
catalogue codes (non-enumeration invariant).
> The store is in-memory and seeded from `src/data/seed.json`. Each scenario uses
> a different seeded over-limit transaction (`t-1`, `t-3`, `t-2`), so one run
> needs no reset. To re-run from scratch, **restart the dev server** to reseed.
Minimal manual equivalent of the gate→unlock payoff:
```bash
BASE=http://localhost:3939/api/v1
# A. gate blocks
curl -s -X PUT "$BASE/transactions/t-1" -H 'content-type: application/json' \
-d '{"status":"approved"}' # -> 422 OVER_POLICY_LIMIT
# B. unlock
EXC=$(curl -s -X POST "$BASE/exceptions" -H 'content-type: application/json' \
-d '{"transactionId":"t-1","code":"EXC-BOARD-APPROVED"}' | jq -r .id)
curl -s -X POST "$BASE/exceptions/$EXC/finalize" # -> 200 approved
curl -s -X PUT "$BASE/transactions/t-1" -H 'content-type: application/json' \
-d '{"status":"approved"}' # -> 201 (now allowed)
```
### Fresh-agent learning proof (activates once the backend lands) — roles #3 + #5
This is the proof that the loop _learned_, not that the REST works. It requires
the recording hook (real, not the shim) and the env-gated `CopilotKitIntelligence`
runtime configured (`INTELLIGENCE_API_URL`, `INTELLIGENCE_GATEWAY_WS_URL`,
`INTELLIGENCE_API_KEY`).
1. **Baseline (no knowledge).** In a fresh thread, ask the agent to approve an
over-limit transaction. With role #4 framing intact it **fails correctly**:
it hits the gate, has no procedure, and (per ACTION DISCIPLINE) reports the
failure instead of firing a distractor. _This failure is the control._
2. **Human teaches.** A human opens the policy-exception modal and performs the
unlock (justifying code → finalize). Each step fires `recordUserAction(...)`
on the current thread (now a real stream, not a no-op).
3. **Distill.** The Intelligence writer agent distills those recorded actions
into a reusable procedure in `/knowledge`.
4. **Fresh agent succeeds unaided.** In a **new** thread (no memory of the
human's session), ask the same over-limit approval. The agent retrieves
`/knowledge`, files a _justifying_ exception, finalizes it, and the approval
now returns **201** — with **no human help and nothing added to the prompt**.
Pass criteria: step 1 fails, step 4 succeeds, and the only thing that changed
between them is the distilled `/knowledge`. That delta _is_ the learning.
examples/showcases/banking/docs/teach-mode/verify-teachable-gate.sh
Executable
+218
View File
@@ -0,0 +1,218 @@
#!/usr/bin/env bash
#
# verify-teachable-gate.sh — proves the teach-mode GATE → UNLOCK contract end to
# end against the BANKING demo's real REST routes. This is the BACKEND-INDEPENDENT
# proof: it exercises roles #1 (GATE), #2 (UNLOCK + DECOY + catalogue check) of
# the 5-role contract entirely over HTTP, with NO Intelligence stack required.
# (Roles #3 RECORDING / #4 AGENT FRAMING / #5 KNOWLEDGE BACKEND are proven
# separately — see the README "Verification" section for the fresh-agent learning
# proof that activates once the backend lands.)
#
# It demonstrates, in order:
# A. GATE — approving an over-policy-limit transaction is blocked (422
# OVER_POLICY_LIMIT, symptom-only: it names the problem, never
# the fix).
# B. UNLOCK — filing a JUSTIFYING policy exception (open → finalize → link)
# lifts the gate; the same approval now succeeds.
# C. DECOY — a NON-justifying catalogue code files fine but does NOT lift
# the gate; approval stays blocked (422).
# D. CATALOGUE — an INVALID code is rejected (422 INVALID_EXCEPTION_CODE)
# WITHOUT the response enumerating the valid catalogue.
#
# USAGE
# ./verify-teachable-gate.sh # against http://localhost:3939
# BASE_URL=http://localhost:3000 ./verify-teachable-gate.sh
# ./verify-teachable-gate.sh http://localhost:3000 # positional override
#
# PREREQUISITES
# - The banking demo running locally (its dev server, e.g. `pnpm dev`). NOTE:
# `next dev` defaults to :3000; this script defaults to :3939 to match the
# project convention — point BASE_URL at whatever port you actually serve.
# - `curl` and `jq` on PATH.
# - A FRESH server process (or one not yet mutated by a prior run). The store is
# in-memory and seeded from `src/data/seed.json`; restart the dev server to
# reset. Each scenario below uses a DIFFERENT seeded transaction so a single
# run does not need a reset between scenarios.
#
# SEED FACTS THIS SCRIPT RELIES ON (src/data/seed.json):
# - t-1 "Google Ads" amount -5000 policy Marketing (limit 5000, spent 500)
# → approve needs 500+5000=5500 > 5000 ⇒ OVER LIMIT (scenario A/B)
# - t-3 "Microsoft 365" amount -10000 policy Executive (limit 10000, spent 1000)
# → approve needs 1000+10000=11000 > 10000 ⇒ OVER LIMIT (scenario C)
# - t-2 "AWS" amount -15000 policy Engineering(limit 15000, spent 1500)
# → approve needs 1500+15000=16500 > 15000 ⇒ OVER LIMIT (scenario D)
#
# JUSTIFYING vs DECOY codes (src/app/api/v1/policy-exception-codes.ts):
# JUSTIFYING (lift the gate): EXC-BOARD-APPROVED, EXC-CONTRACTUAL-COMMITMENT,
# EXC-EMERGENCY-SPEND
# DECOY (filed for history, do NOT lift the gate): EXC-WILL-REIMBURSE, EXC-ONE-TIME
# INVALID (not in the catalogue at all): anything else, e.g. EXC-MADE-UP
#
set -euo pipefail
# --- Config ----------------------------------------------------------------
BASE_URL="${1:-${BASE_URL:-http://localhost:3939}}"
API="${BASE_URL%/}/api/v1"
# Seeded over-limit transactions (one per scenario so they don't interfere).
TXN_GATE="t-1" # scenarios A + B
TXN_DECOY="t-3" # scenario C
TXN_INVALID="t-2" # scenario D
JUSTIFYING_CODE="EXC-BOARD-APPROVED" # lifts the gate (role #2 UNLOCK)
DECOY_CODE="EXC-WILL-REIMBURSE" # valid catalogue code, does NOT justify
INVALID_CODE="EXC-DEFINITELY-NOT-REAL" # not in the catalogue at all
# --- Pretty helpers --------------------------------------------------------
have() { command -v "$1" >/dev/null 2>&1; }
if ! have curl; then echo "ERROR: curl not found on PATH" >&2; exit 1; fi
if ! have jq; then echo "ERROR: jq not found on PATH (used to parse JSON)" >&2; exit 1; fi
PASS=0
FAIL=0
section() { printf '\n\033[1m== %s ==\033[0m\n' "$1"; }
ok() { printf ' \033[32mPASS\033[0m %s\n' "$1"; PASS=$((PASS+1)); }
bad() { printf ' \033[31mFAIL\033[0m %s\n' "$1"; FAIL=$((FAIL+1)); }
# status_of METHOD URL [JSON_BODY] -> prints HTTP status, stashes body in $BODY
BODY=""
status_of() {
local method="$1" url="$2" data="${3:-}"
local resp code
if [[ -n "$data" ]]; then
resp="$(curl -sS -o /tmp/tm_body.$$ -w '%{http_code}' \
-X "$method" "$url" \
-H 'content-type: application/json' \
-d "$data")"
else
resp="$(curl -sS -o /tmp/tm_body.$$ -w '%{http_code}' -X "$method" "$url")"
fi
code="$resp"
BODY="$(cat /tmp/tm_body.$$ 2>/dev/null || true)"
rm -f /tmp/tm_body.$$
printf '%s' "$code"
}
approve() { status_of PUT "$API/transactions/$1" '{"status":"approved"}'; }
open_exception() { status_of POST "$API/exceptions" "{\"transactionId\":\"$1\",\"code\":\"$2\"}"; }
finalize() { status_of POST "$API/exceptions/$1/finalize"; }
printf 'teach-mode gate→unlock verification\n'
printf 'BASE_URL = %s\n' "$BASE_URL"
# ===========================================================================
# Scenario A — GATE (role #1): approving an over-limit txn is blocked, 422,
# symptom-only. The error names the over-limit POLICY but NEVER the
# policy-exception path that would lift it.
# Route: PUT /api/v1/transactions/[id] (src/app/api/v1/transactions/[id]/route.ts)
# ===========================================================================
section "A. GATE — over-limit approval is blocked (role #1)"
code="$(approve "$TXN_GATE")"
if [[ "$code" == "422" && "$(jq -r '.error' <<<"$BODY")" == "OVER_POLICY_LIMIT" ]]; then
ok "approve $TXN_GATE → 422 OVER_POLICY_LIMIT"
else
bad "expected 422 OVER_POLICY_LIMIT, got HTTP $code body=$BODY"
fi
# Symptom-only invariant: the rejection must NOT leak the unlock recipe.
if grep -qiE 'exception|EXC-|finalize|override|knowledge' <<<"$BODY"; then
bad "GATE error leaked the fix (mentions the exception/unlock path): $BODY"
else
ok "GATE error is symptom-only (no mention of the exception/unlock recipe)"
fi
# ===========================================================================
# Scenario B — UNLOCK (role #2): file a JUSTIFYING exception, then approve.
# open → POST /api/v1/exceptions (src/app/api/v1/exceptions/route.ts)
# final → POST /api/v1/exceptions/[id]/finalize
# (src/app/api/v1/exceptions/[id]/finalize/route.ts)
# The finalize step auto-approves the exception AND links it to the
# transaction's activeExceptionId — which is what lifts the gate, but ONLY
# because the code is justifying (store.hasApprovedException + isJustifying).
# ===========================================================================
section "B. UNLOCK — justifying exception lifts the gate (role #2)"
code="$(open_exception "$TXN_GATE" "$JUSTIFYING_CODE")"
if [[ "$code" == "201" ]]; then
EXC_ID="$(jq -r '.id' <<<"$BODY")"
ok "open exception ($JUSTIFYING_CODE) on $TXN_GATE → 201 id=$EXC_ID"
else
bad "expected 201 opening exception, got HTTP $code body=$BODY"; EXC_ID=""
fi
if [[ -n "${EXC_ID:-}" ]]; then
code="$(finalize "$EXC_ID")"
if [[ "$code" == "200" && "$(jq -r '.status' <<<"$BODY")" == "approved" ]]; then
ok "finalize $EXC_ID → 200 status=approved (linked to $TXN_GATE)"
else
bad "expected 200 approved finalizing, got HTTP $code body=$BODY"
fi
fi
# The payoff: the SAME approval that was blocked in A now succeeds.
code="$(approve "$TXN_GATE")"
if [[ "$code" == "201" ]]; then
ok "approve $TXN_GATE again → 201 (gate lifted by justifying exception)"
else
bad "expected 201 after justifying unlock, got HTTP $code body=$BODY"
fi
# ===========================================================================
# Scenario C — DECOY (role #2 invariant): a NON-justifying catalogue code is a
# valid code (files + finalizes fine, recorded for history) but does NOT lift
# the gate. Approval must stay 422. This is what forces the agent to LEARN
# WHICH codes justify — it can file every code and still fail.
# ===========================================================================
section "C. DECOY — non-justifying code files but does NOT unlock (role #2)"
code="$(open_exception "$TXN_DECOY" "$DECOY_CODE")"
if [[ "$code" == "201" ]]; then
DECOY_ID="$(jq -r '.id' <<<"$BODY")"
ok "open exception ($DECOY_CODE) on $TXN_DECOY → 201 id=$DECOY_ID (accepted: valid code)"
else
bad "expected 201 opening decoy exception, got HTTP $code body=$BODY"; DECOY_ID=""
fi
if [[ -n "${DECOY_ID:-}" ]]; then
code="$(finalize "$DECOY_ID")"
if [[ "$code" == "200" ]]; then
ok "finalize $DECOY_ID → 200 (filed for history)"
else
bad "expected 200 finalizing decoy, got HTTP $code body=$BODY"
fi
fi
code="$(approve "$TXN_DECOY")"
if [[ "$code" == "422" && "$(jq -r '.error' <<<"$BODY")" == "OVER_POLICY_LIMIT" ]]; then
ok "approve $TXN_DECOY → 422 OVER_POLICY_LIMIT (decoy did NOT lift the gate)"
else
bad "expected 422 OVER_POLICY_LIMIT (decoy must not unlock), got HTTP $code body=$BODY"
fi
# ===========================================================================
# Scenario D — CATALOGUE CHECK (role #2 invariant): an INVALID code is rejected
# (422 INVALID_EXCEPTION_CODE) WITHOUT enumerating the valid catalogue. Leaking
# the list would itself be a hint; the agent must discover the catalogue via
# /knowledge. Rejection happens at open time, so nothing to finalize/approve.
# ===========================================================================
section "D. CATALOGUE — invalid code rejected without leaking the list (role #2)"
code="$(open_exception "$TXN_INVALID" "$INVALID_CODE")"
if [[ "$code" == "422" && "$(jq -r '.error' <<<"$BODY")" == "INVALID_EXCEPTION_CODE" ]]; then
ok "open exception ($INVALID_CODE) → 422 INVALID_EXCEPTION_CODE"
else
bad "expected 422 INVALID_EXCEPTION_CODE, got HTTP $code body=$BODY"
fi
# Non-enumeration invariant: the rejection must not list any real catalogue codes.
if grep -qE 'EXC-BOARD-APPROVED|EXC-CONTRACTUAL-COMMITMENT|EXC-EMERGENCY-SPEND|EXC-WILL-REIMBURSE|EXC-ONE-TIME' <<<"$BODY"; then
bad "INVALID_EXCEPTION_CODE response leaked catalogue codes: $BODY"
else
ok "rejection does not enumerate the catalogue (no valid codes leaked)"
fi
# --- Summary ---------------------------------------------------------------
section "Summary"
printf ' passed: %d failed: %d\n' "$PASS" "$FAIL"
if [[ "$FAIL" -gt 0 ]]; then
printf '\n\033[31mGATE→UNLOCK contract NOT satisfied.\033[0m If failures are 404s or\n'
printf 'unexpected 201s on scenario A/C, you are likely re-running against a server\n'
printf 'whose store was already mutated — restart the dev server to reseed.\n'
exit 1
fi
printf '\n\033[32mGATE→UNLOCK contract verified (roles #1 and #2).\033[0m\n'