chore: import upstream snapshot with attribution
This commit is contained in:
@@ -0,0 +1,52 @@
|
||||
# QA: Agent Config Object — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
- `ANTHROPIC_API_KEY` is set on the deployment
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/agent-config`
|
||||
- [ ] Verify the config card renders with three select inputs:
|
||||
`agent-config-tone-select`, `agent-config-expertise-select`,
|
||||
`agent-config-length-select`
|
||||
- [ ] Verify the chat surface is visible below the config card
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Tone
|
||||
|
||||
- [ ] Set tone to `professional`, send "Tell me about Paris"
|
||||
- [ ] Verify Claude's reply avoids emoji and exclamation marks
|
||||
- [ ] Change tone to `enthusiastic`, send the same prompt
|
||||
- [ ] Verify the next reply is noticeably more upbeat
|
||||
|
||||
#### Expertise
|
||||
|
||||
- [ ] Set expertise to `beginner`, ask "Explain quantum entanglement"
|
||||
- [ ] Verify the reply defines jargon and uses an analogy
|
||||
- [ ] Switch to `expert` and ask again
|
||||
- [ ] Verify the reply uses precise terminology and skips basics
|
||||
|
||||
#### Response length
|
||||
|
||||
- [ ] Set responseLength to `concise` and ask anything open-ended
|
||||
- [ ] Verify the reply is at most 3 sentences
|
||||
- [ ] Switch to `detailed`
|
||||
- [ ] Verify the reply spans multiple paragraphs
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Send a message without changing any config first — verify the
|
||||
default (professional / intermediate / concise) behaviour holds
|
||||
- [ ] No console errors during normal usage
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Config changes take effect on the NEXT message (not retroactively).
|
||||
- `forwardedProps` reaches the backend via
|
||||
`forwarded_props.config.configurable.properties`.
|
||||
@@ -0,0 +1,62 @@
|
||||
# QA: Agentic Chat — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to the agentic-chat demo page
|
||||
- [ ] Verify the chat interface loads with a text input placeholder "Type a message"
|
||||
- [ ] Verify the background container (`data-testid="background-container"`) is visible
|
||||
- [ ] Verify the default background color is the theme default (rgb(250, 250, 249))
|
||||
- [ ] Send a basic message (e.g. "Hello")
|
||||
- [ ] Verify the agent responds with a text message
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Suggestions
|
||||
|
||||
- [ ] Verify "Change background" suggestion button is visible
|
||||
- [ ] Verify "Generate sonnet" suggestion button is visible
|
||||
- [ ] Click the "Change background" suggestion
|
||||
- [ ] Verify the suggestion either populates the input or sends the message
|
||||
|
||||
#### Background Change (useFrontendTool)
|
||||
|
||||
- [ ] Ask "Change the background to a sunset gradient"
|
||||
- [ ] Verify the background container style changes from the default
|
||||
- [ ] Verify the change_background tool returns a success status
|
||||
|
||||
#### Weather Render Tool (useRenderTool)
|
||||
|
||||
- [ ] Type "What's the weather in Tokyo?"
|
||||
- [ ] Verify loading state shows "Loading weather..." (`data-testid="weather-info-loading"`)
|
||||
- [ ] Verify WeatherCard renders (`data-testid="weather-info"`) with:
|
||||
- [ ] City name displayed
|
||||
- [ ] Temperature in degrees C
|
||||
- [ ] Humidity percentage
|
||||
- [ ] Wind speed in mph
|
||||
- [ ] Conditions text
|
||||
|
||||
#### Agent Context
|
||||
|
||||
- [ ] Verify the agent knows the user's name is "Bob" (provided via useAgentContext)
|
||||
- [ ] Ask "What is my name?" and verify the agent responds with "Bob"
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Send an empty message (should be handled gracefully)
|
||||
- [ ] Verify no console errors during normal usage
|
||||
- [ ] Send a very long message and verify no UI breakage
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Chat loads within 3 seconds
|
||||
- Agent responds within 10 seconds
|
||||
- Background changes are instant after tool execution
|
||||
- Weather card renders with all data fields populated
|
||||
- No UI errors or broken layouts
|
||||
@@ -0,0 +1,48 @@
|
||||
# QA: Authentication — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
- `ANTHROPIC_API_KEY` is set on the deployment
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality (happy path — start signed in)
|
||||
|
||||
- [ ] Navigate to `/demos/auth`
|
||||
- [ ] Verify the auth banner renders with
|
||||
`data-authenticated="true"`
|
||||
- [ ] Verify the status reads "✓ Signed in as demo user"
|
||||
- [ ] Verify the "Sign out" button is visible
|
||||
- [ ] Send "Hello" and verify Claude responds
|
||||
|
||||
### 2. Sign-out path
|
||||
|
||||
- [ ] Click "Sign out"
|
||||
- [ ] Verify the banner flips to amber with
|
||||
`data-authenticated="false"`
|
||||
- [ ] Verify the button now reads "Sign in"
|
||||
- [ ] Attempt to send a message
|
||||
- [ ] Verify the error banner (`data-testid="auth-demo-error"`)
|
||||
renders with "401 Unauthorized" text
|
||||
- [ ] Verify no white-screen and no uncaught React errors (the
|
||||
ChatErrorBoundary catches any render-time crash from chat
|
||||
internals in the unauth state and renders
|
||||
`data-testid="auth-demo-chat-boundary"`)
|
||||
|
||||
### 3. Sign-in recovery
|
||||
|
||||
- [ ] Click "Sign in"
|
||||
- [ ] Verify the banner flips back to green
|
||||
- [ ] Verify the error banner clears
|
||||
- [ ] Send a message and verify Claude replies again
|
||||
|
||||
### 4. Error Handling
|
||||
|
||||
- [ ] No console errors during normal usage.
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Runtime rejects un-bearer'd requests with HTTP 401.
|
||||
- Page never white-screens, even during auth transitions.
|
||||
@@ -0,0 +1,49 @@
|
||||
# QA: BYOC hashbrown — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
- `ANTHROPIC_API_KEY` is set on the deployment
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/byoc-hashbrown`
|
||||
- [ ] Verify the header "BYOC: Hashbrown" renders
|
||||
- [ ] Verify the description paragraph mentions `@hashbrownai/react`
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Q4 Sales Summary (mixed catalog)
|
||||
|
||||
- [ ] Send "Show me a Q4 sales summary" (or click a suggestion)
|
||||
- [ ] Verify a `data-testid="metric-card"` renders with a formatted value
|
||||
- [ ] Verify a `data-testid="pie-chart"` renders with at least three
|
||||
legend rows
|
||||
- [ ] Verify a `data-testid="bar-chart"` renders with at least three
|
||||
columns
|
||||
- [ ] Verify at least one Markdown heading renders inline
|
||||
|
||||
#### Deal card
|
||||
|
||||
- [ ] Ask "Show me a sample deal in the negotiation stage"
|
||||
- [ ] Verify a `data-testid="hashbrown-deal-card"` renders with a stage
|
||||
badge
|
||||
|
||||
### 3. Streaming behaviour
|
||||
|
||||
- [ ] Observe components progressively appear as Claude streams the
|
||||
JSON envelope — no full-refresh flash at the end of streaming
|
||||
|
||||
### 4. Error Handling
|
||||
|
||||
- [ ] No console errors during normal usage.
|
||||
- [ ] No hashbrown schema-validation errors logged.
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Chat loads within 3 seconds
|
||||
- Agent responds within 15 seconds
|
||||
- Backend emits the JSON envelope (`{ui: [...]}`), NEVER XML
|
||||
@@ -0,0 +1,59 @@
|
||||
# QA: BYOC json-render — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
- `ANTHROPIC_API_KEY` is set on the deployment
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/byoc-json-render`
|
||||
- [ ] Verify the chat surface loads inside the centered 4xl container
|
||||
- [ ] Verify the three suggestion pills are visible:
|
||||
"Sales dashboard", "Revenue by category", "Expense trend"
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Sales dashboard (MetricCard + BarChart)
|
||||
|
||||
- [ ] Click the "Sales dashboard" suggestion pill
|
||||
- [ ] Verify a `data-testid="metric-card"` element renders with a label
|
||||
and a dollar-formatted value
|
||||
- [ ] Verify a `data-testid="bar-chart"` element renders inside the same
|
||||
`data-testid="json-render-root"` wrapper
|
||||
|
||||
#### Revenue by category (PieChart)
|
||||
|
||||
- [ ] Click "Revenue by category"
|
||||
- [ ] Verify a `data-testid="pie-chart"` element renders with at least
|
||||
three legend rows
|
||||
|
||||
#### Expense trend (BarChart)
|
||||
|
||||
- [ ] Click "Expense trend"
|
||||
- [ ] Verify `data-testid="bar-chart"` renders with three months of data
|
||||
|
||||
### 3. Streaming behaviour
|
||||
|
||||
- [ ] Observe the raw JSON streaming into the chat bubble briefly while
|
||||
the model emits the spec
|
||||
- [ ] Verify the catalog components swap in cleanly once the JSON
|
||||
becomes valid — no flicker, no duplicate render
|
||||
|
||||
### 4. Error Handling
|
||||
|
||||
- [ ] Ask a free-form question that has nothing to do with dashboards
|
||||
(e.g. "What is 2+2?"). The agent should still reply with a JSON
|
||||
spec — it may emit a single MetricCard — and the page must NOT
|
||||
white-screen.
|
||||
- [ ] No console errors during normal usage.
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Chat loads within 3 seconds
|
||||
- Agent responds within 15 seconds (Claude opus)
|
||||
- Components render from the json-render catalog wrapped in a single
|
||||
`<JSONUIProvider>` (no missing-provider crashes)
|
||||
@@ -0,0 +1,39 @@
|
||||
# QA: Chat Customization (CSS) — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/chat-customization-css`
|
||||
- [ ] Verify the chat loads inside the `.chat-css-demo-scope` wrapper
|
||||
- [ ] Send a basic message
|
||||
- [ ] Verify the agent responds
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Theme Application
|
||||
|
||||
- [ ] Verify user message bubbles render with the hot-pink gradient
|
||||
- [ ] Verify assistant message bubbles render with the amber background and monospace font
|
||||
- [ ] Verify the input area shows a dashed pink border
|
||||
- [ ] Verify the input placeholder color is pink italic
|
||||
|
||||
#### Variable Overrides
|
||||
|
||||
- [ ] Inspect the `.chat-css-demo-scope` element and verify CopilotKit CSS variables are overridden (e.g. `--copilot-kit-primary-color: #ff006e`)
|
||||
- [ ] Verify the customization does not leak outside the scoped container
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Verify no console errors during normal usage
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Page loads within 3 seconds
|
||||
- Theme is applied only inside `.chat-css-demo-scope`
|
||||
- Agent responds within 10 seconds
|
||||
@@ -0,0 +1,44 @@
|
||||
# QA: Chat Slots — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/chat-slots`
|
||||
- [ ] Verify the custom welcome screen (`data-testid="custom-welcome-screen"`) is visible
|
||||
- [ ] Verify the welcome card shows "Welcome to the Slots demo"
|
||||
- [ ] Verify the suggestion pills ("Write a sonnet", "Tell me a joke") are visible
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Welcome Slot
|
||||
|
||||
- [ ] Confirm the gradient welcome card wraps the default input + suggestions
|
||||
- [ ] Verify the "Custom Slot" pill is visible above the headline
|
||||
|
||||
#### Disclaimer Slot
|
||||
|
||||
- [ ] After sending a message, verify the custom disclaimer (`data-testid="custom-disclaimer"`) is visible
|
||||
- [ ] Verify the disclaimer text contains "Custom disclaimer injected via"
|
||||
|
||||
#### Assistant Message Slot
|
||||
|
||||
- [ ] Send a basic message
|
||||
- [ ] Wait for the agent response
|
||||
- [ ] Verify the assistant message is wrapped by the custom slot (`data-testid="custom-assistant-message"`)
|
||||
- [ ] Verify the "slot" badge is visible on the assistant message
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Verify no console errors during normal usage
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Page loads within 3 seconds
|
||||
- All three slot overrides (welcome, disclaimer, assistantMessage) render as custom components
|
||||
- Agent responds within 10 seconds
|
||||
@@ -0,0 +1,22 @@
|
||||
# QA: CLI Start Command — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- None (manifest-only demo — no hosted route)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Manifest
|
||||
|
||||
- [ ] Verify the `cli-start` entry exists in `manifest.yaml` under `demos:`
|
||||
- [ ] Verify the `command` field reads `npx copilotkit@latest init --framework claude-sdk-python`
|
||||
- [ ] Verify `cli-start` is listed in the `features:` array
|
||||
|
||||
### 2. Expected Behavior
|
||||
|
||||
- [ ] Running the command in a fresh directory scaffolds the Claude Agent SDK (Python) starter
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Manifest entry validates against the showcase schema
|
||||
- Command matches the canonical framework slug
|
||||
@@ -0,0 +1,33 @@
|
||||
# QA: Frontend Tools (Async) — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy
|
||||
- ANTHROPIC_API_KEY is set
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/frontend-tools-async`
|
||||
- [ ] Verify the chat renders
|
||||
- [ ] Click the "Find project-planning notes" suggestion
|
||||
- [ ] Verify a NotesCard appears briefly in loading state ("Querying local notes DB...")
|
||||
- [ ] After ~500ms, verify the card shows matching notes (Q2 kickoff, migrate auth to passkeys, retrospective notes, career planning)
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
- [ ] Search for "auth" — verify the "migrate auth to passkeys" note is returned
|
||||
- [ ] Ask "Do I have notes tagged reading?" — verify the Book recommendations note appears
|
||||
- [ ] Search for "nonsense-keyword-no-match" — verify "No notes matched." empty state
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Verify no console errors during search
|
||||
- [ ] Verify the loading state transitions to complete within a few seconds
|
||||
|
||||
## Expected Results
|
||||
|
||||
- NotesCard with `data-testid="notes-card"` displays matching notes
|
||||
- Each note rendered with `data-testid="note-<id>"` and tags
|
||||
@@ -0,0 +1,32 @@
|
||||
# QA: Frontend Tools — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
- ANTHROPIC_API_KEY is set
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/frontend-tools`
|
||||
- [ ] Verify a chat surface renders in the centered card
|
||||
- [ ] Send a message: "Change the background to a blue-to-purple gradient."
|
||||
- [ ] Verify the agent calls the `change_background` frontend tool
|
||||
- [ ] Verify the page background updates to the requested gradient
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
- [ ] Click the "Sunset theme" suggestion — background changes to a sunset gradient
|
||||
- [ ] Ask "Change the background back to default." — background resets
|
||||
- [ ] Verify no errors in the console
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Verify the chat remains usable after multiple background changes
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Background element with `data-testid="background-container"` reflects the CSS background value returned by the tool handler
|
||||
- Agent calls the tool and produces a short confirmation message
|
||||
@@ -0,0 +1,62 @@
|
||||
# QA: Agentic Generative UI — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to the gen-ui-agent demo page
|
||||
- [ ] Verify the chat interface loads in a centered full-height layout
|
||||
- [ ] Verify the chat input placeholder "Type a message" is visible
|
||||
- [ ] Send a basic message
|
||||
- [ ] Verify the agent responds
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Suggestions
|
||||
|
||||
- [ ] Verify "Simple plan" suggestion button is visible (plan to go to Mars in 5 steps)
|
||||
- [ ] Verify "Complex plan" suggestion button is visible (plan to make pizza in 10 steps)
|
||||
|
||||
#### Task Progress Tracker (useAgent with state streaming)
|
||||
|
||||
- [ ] Click "Simple plan" suggestion or type "Build a plan to go to Mars in 5 steps"
|
||||
- [ ] Verify the TaskProgress component renders (`data-testid="task-progress"`)
|
||||
- [ ] Verify the progress bar appears with a gradient fill
|
||||
- [ ] Verify step items appear with descriptions (`data-testid="task-step-text"`)
|
||||
- [ ] Verify the "N/N Complete" counter updates as steps complete
|
||||
- [ ] Verify completed steps show:
|
||||
- Green background gradient
|
||||
- Check icon
|
||||
- Green text color
|
||||
- [ ] Verify the current pending step shows:
|
||||
- Blue/purple background gradient
|
||||
- Spinner icon with "Processing..." text
|
||||
- Pulsing animation
|
||||
- [ ] Verify future pending steps show:
|
||||
- Gray background
|
||||
- Clock icon
|
||||
- Muted text color
|
||||
|
||||
#### Complex Plan
|
||||
|
||||
- [ ] Type "Plan to make pizza in 10 steps"
|
||||
- [ ] Verify 10 steps appear in the progress tracker
|
||||
- [ ] Verify progress bar width increases as steps complete
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Send an empty message (should be handled gracefully)
|
||||
- [ ] Verify no console errors during normal usage
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Chat loads within 3 seconds
|
||||
- Agent responds within 10 seconds
|
||||
- Task progress tracker shows live step completion
|
||||
- Progress bar animates smoothly
|
||||
- No UI errors or broken layouts
|
||||
@@ -0,0 +1,59 @@
|
||||
# QA: Tool-Based Generative UI — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to the gen-ui-tool-based demo page
|
||||
- [ ] Verify the CopilotSidebar opens by default with title "Haiku Generator"
|
||||
- [ ] Verify a placeholder haiku card is displayed on the main area
|
||||
- [ ] Send a basic message via the sidebar
|
||||
- [ ] Verify the agent responds
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Suggestions
|
||||
|
||||
- [ ] Verify "Nature Haiku" suggestion button is visible
|
||||
- [ ] Verify "Ocean Haiku" suggestion button is visible
|
||||
- [ ] Verify "Spring Haiku" suggestion button is visible
|
||||
|
||||
#### Haiku Generation (useFrontendTool)
|
||||
|
||||
- [ ] Click the "Nature Haiku" suggestion or type "Write me a haiku about nature"
|
||||
- [ ] Verify a HaikuCard renders (`data-testid="haiku-card"`) with:
|
||||
- [ ] Three Japanese text lines (`data-testid="haiku-japanese-line"`)
|
||||
- [ ] Three English translation lines (`data-testid="haiku-english-line"`)
|
||||
- [ ] A background gradient style applied to the card
|
||||
- [ ] Verify the Japanese text contains actual Japanese characters (not Latin)
|
||||
- [ ] Verify the English lines are a readable English translation
|
||||
|
||||
#### Image Display
|
||||
|
||||
- [ ] After haiku generation, verify an image renders (`data-testid="haiku-image"`) if the agent provides an image_name
|
||||
- [ ] Verify the image src points to /images/ with a valid filename from the predefined list
|
||||
|
||||
#### Multiple Haikus
|
||||
|
||||
- [ ] Generate a second haiku (e.g. "Ocean Haiku")
|
||||
- [ ] Verify the new haiku card appears at the top
|
||||
- [ ] Verify the previous haiku card is still visible below
|
||||
- [ ] Verify the initial placeholder haiku is removed
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Send an empty message (should be handled gracefully)
|
||||
- [ ] Verify no console errors during normal usage
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Sidebar loads within 3 seconds
|
||||
- Agent responds and generates haiku within 10 seconds
|
||||
- Haiku cards display with Japanese and English text
|
||||
- Generated haikus stack with newest on top
|
||||
- No UI errors or broken layouts
|
||||
@@ -0,0 +1,33 @@
|
||||
# QA: Headless Chat (Complete) — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy
|
||||
- ANTHROPIC_API_KEY is set
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/headless-complete`
|
||||
- [ ] Verify the custom chat surface renders (header, scrollable list, input bar)
|
||||
- [ ] Verify NO `<CopilotChat />` imports are visible (all chrome is custom)
|
||||
- [ ] Send "Hello" — verify the user bubble + assistant bubble render
|
||||
|
||||
### 2. Generative UI Weave
|
||||
|
||||
- [ ] Click the "Weather in Tokyo" suggestion — verify the WeatherCard renders inline
|
||||
- [ ] Click the "Highlight a note" suggestion — verify the HighlightNote component renders
|
||||
- [ ] Try a message that triggers other tools — verify the catch-all tool card renders
|
||||
|
||||
### 3. Lifecycle
|
||||
|
||||
- [ ] During agent response, verify the Send button swaps to Stop
|
||||
- [ ] Click Stop — verify the agent cancels cleanly
|
||||
- [ ] Verify no console errors after stop
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Messages area with `data-testid="headless-complete-messages"` scrolls automatically
|
||||
- Input disabled while running; Send button disabled when empty
|
||||
@@ -0,0 +1,41 @@
|
||||
# QA: Headless Chat (Simple) — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/headless-simple`
|
||||
- [ ] Verify the heading "Headless Chat (Simple)" is visible
|
||||
- [ ] Verify the placeholder text "No messages yet. Say hi!" is displayed
|
||||
- [ ] Type a message into the textarea
|
||||
- [ ] Click Send (or press Enter)
|
||||
- [ ] Verify the user bubble appears immediately on the right
|
||||
- [ ] Verify the agent responds with a text bubble on the left
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### useAgent / useComponent
|
||||
|
||||
- [ ] Ask "Show me a card about cats"
|
||||
- [ ] Verify the ShowCard component renders with a title and body
|
||||
|
||||
#### Running state
|
||||
|
||||
- [ ] While the agent is replying, verify "Agent is thinking..." appears below the message list
|
||||
- [ ] Verify the Send button is disabled while the agent is running
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Verify sending an empty message does nothing
|
||||
- [ ] Verify no console errors during normal usage
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Page loads within 3 seconds
|
||||
- Agent responds within 10 seconds
|
||||
- ShowCard renders when the agent calls `show_card`
|
||||
@@ -0,0 +1,38 @@
|
||||
# QA: In-App HITL — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
- ANTHROPIC_API_KEY is set
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/hitl-in-app`
|
||||
- [ ] Verify the Support Inbox renders three tickets (#12345, #12346, #12347)
|
||||
- [ ] Verify the chat surface renders on the right
|
||||
|
||||
### 2. Approval Flow
|
||||
|
||||
- [ ] Click the "Approve refund for #12345" suggestion
|
||||
- [ ] Verify a modal dialog appears with the action summary
|
||||
- [ ] Click "Approve" — verify the dialog closes and the agent confirms
|
||||
|
||||
### 3. Rejection Flow
|
||||
|
||||
- [ ] Click the "Downgrade plan for #12346" suggestion
|
||||
- [ ] Verify the modal appears
|
||||
- [ ] Add a note "Customer needs to stay on current plan"
|
||||
- [ ] Click "Reject" — verify the agent acknowledges the rejection
|
||||
|
||||
### 4. Error Handling
|
||||
|
||||
- [ ] Verify no console errors during the approval/rejection flows
|
||||
- [ ] Verify the modal blocks interaction with the chat until resolved
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Modal uses `fixed inset-0` overlay rather than a portal (to avoid @types/react-dom)
|
||||
- Agent accurately reflects the approve/reject outcome
|
||||
@@ -0,0 +1,57 @@
|
||||
# QA: Human in the Loop — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to the HITL demo page (`/demos/hitl`) > _Note: the URL path `/demos/hitl` is intentionally shorter than the feature id `hitl-in-chat`._
|
||||
- [ ] Verify the chat interface loads in a centered max-w-4xl container
|
||||
- [ ] Verify the chat input placeholder "Type a message" is visible
|
||||
- [ ] Send a basic message
|
||||
- [ ] Verify the agent responds
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Suggestions
|
||||
|
||||
- [ ] Verify "Simple plan" suggestion button is visible
|
||||
- [ ] Verify "Complex plan" suggestion button is visible
|
||||
- [ ] Click the "Simple plan" suggestion
|
||||
- [ ] Verify it triggers a message about planning a trip to Mars in 5 steps
|
||||
|
||||
#### Step Selection and Approval
|
||||
|
||||
The HITL demo renders a single StepSelector card regardless of whether
|
||||
the underlying flow uses an interrupt hook or a frontend HITL tool. The
|
||||
framework-specific hook name is an implementation detail covered in each
|
||||
package's demo source; QA below validates the user-visible card behavior.
|
||||
|
||||
- [ ] Send "Plan a trip to Mars in 5 steps"
|
||||
- [ ] Verify the StepSelector card appears (`data-testid="select-steps"`)
|
||||
- [ ] Verify step items are displayed with checkboxes (`data-testid="step-item"`)
|
||||
- [ ] Verify step text is visible (`data-testid="step-text"`)
|
||||
- [ ] Verify the selected count display shows "N/N selected"
|
||||
- [ ] Toggle a step checkbox off and verify the count decreases
|
||||
- [ ] Toggle it back on and verify the count increases
|
||||
- [ ] Click "Perform Steps (N)" / "Confirm (N)" button
|
||||
- [ ] Verify the agent continues processing after confirmation
|
||||
- [ ] In a new conversation, trigger the same flow and click "Reject" (where present)
|
||||
- [ ] Verify the card reflects the decision (Accepted / Rejected) and buttons disable
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Send an empty message (should be handled gracefully)
|
||||
- [ ] Verify no console errors during normal usage
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Chat loads within 3 seconds
|
||||
- Agent responds within 10 seconds
|
||||
- Step selector renders with toggleable checkboxes
|
||||
- Accept/Reject flow completes without errors
|
||||
- No UI errors or broken layouts
|
||||
@@ -0,0 +1,57 @@
|
||||
# QA: Multimodal Attachments — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
- `ANTHROPIC_API_KEY` is set on the deployment
|
||||
- `pypdf` is installed in the backend environment (listed in
|
||||
`requirements.txt`)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/multimodal`
|
||||
- [ ] Verify the "Multimodal attachments" header renders
|
||||
- [ ] Verify the bundled-samples row shows "Try with sample image" and
|
||||
"Try with sample PDF" buttons
|
||||
- [ ] Verify the paperclip button is visible on the chat composer
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Sample image
|
||||
|
||||
- [ ] Click "Try with sample image" — the button should flip to
|
||||
"Loading…" briefly
|
||||
- [ ] Verify the CopilotChat attachment strip shows a pending
|
||||
`sample.png` chip
|
||||
- [ ] Type "What's in this image?" and send
|
||||
- [ ] Verify Claude's reply references concrete visual details (e.g.
|
||||
colors, objects) from the bundled PNG
|
||||
|
||||
#### Sample PDF
|
||||
|
||||
- [ ] Click "Try with sample PDF"
|
||||
- [ ] Verify the `sample.pdf` chip appears
|
||||
- [ ] Type "Summarise this document" and send
|
||||
- [ ] Verify Claude replies with a short summary of the PDF's actual
|
||||
content (not a generic "I cannot read PDFs" reply)
|
||||
|
||||
### 3. LFS-pointer guard
|
||||
|
||||
- [ ] If the deployment accidentally ships a Git LFS pointer instead of
|
||||
the real binary, clicking a sample button should surface
|
||||
`data-testid="multimodal-sample-error"` with an actionable message
|
||||
|
||||
### 4. Error Handling
|
||||
|
||||
- [ ] No console errors during normal usage.
|
||||
- [ ] Upload a file over 10 MB and verify the attachment is rejected
|
||||
with a visible toast.
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Image replies reference real visual details (not generic).
|
||||
- PDF replies reference document-specific content.
|
||||
- No silent dropping of attachments.
|
||||
@@ -0,0 +1,38 @@
|
||||
# QA: Open-Ended Generative UI (Advanced) — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy
|
||||
- ANTHROPIC_API_KEY is set
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/open-gen-ui-advanced`
|
||||
- [ ] Click the "Calculator (calls evaluateExpression)" suggestion
|
||||
- [ ] Verify an iframe renders with a calculator UI (no <form>, only <button type='button'>)
|
||||
|
||||
### 2. Sandbox-Function Round-Trip
|
||||
|
||||
- [ ] In the calculator, enter "12 \* (3 + 4.5)" and press "="
|
||||
- [ ] Verify the display updates with the host-evaluated result
|
||||
- [ ] Open devtools console — verify `[open-gen-ui/advanced] evaluateExpression` log appears
|
||||
|
||||
### 3. notifyHost
|
||||
|
||||
- [ ] Try the "Ping the host (calls notifyHost)" suggestion
|
||||
- [ ] Click the button in the rendered card
|
||||
- [ ] Verify `[open-gen-ui/advanced] notifyHost:` log in the host console
|
||||
- [ ] Verify the card updates with a confirmation showing `receivedAt`
|
||||
|
||||
### 4. Error Handling
|
||||
|
||||
- [ ] Verify no CORS/CSP errors from the sandbox
|
||||
- [ ] Verify evaluateExpression rejects non-arithmetic input safely
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Sandbox -> host calls via Websandbox.connection.remote work end-to-end
|
||||
- Handler return values are visible inside the iframe UI
|
||||
@@ -0,0 +1,33 @@
|
||||
# QA: Open-Ended Generative UI — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy
|
||||
- ANTHROPIC_API_KEY is set
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/open-gen-ui`
|
||||
- [ ] Verify the chat renders with minimal suggestions visible
|
||||
- [ ] Click the "3D axis visualization (model airplane)" suggestion
|
||||
- [ ] Verify a sandboxed iframe appears with the generated visualization
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
- [ ] Verify the iframe contains SVG-based content (not div-stacks)
|
||||
- [ ] Try the "Quicksort visualization" suggestion — verify an animated bar sort renders
|
||||
- [ ] Verify the visualization is self-running (no user interaction needed)
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Verify no CORS/CSP errors in the console
|
||||
- [ ] Verify the sandbox iframe renders within the chat turn
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Agent calls `generateSandboxedUi` once per turn
|
||||
- The runtime's OpenGenerativeUIMiddleware converts the call into activity events
|
||||
- The built-in renderer mounts the agent-authored HTML inside a sandbox
|
||||
@@ -0,0 +1,41 @@
|
||||
# QA: Pre-Built Popup — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/prebuilt-popup`
|
||||
- [ ] Verify the main content heading "Popup demo — look for the floating launcher" is visible
|
||||
- [ ] Verify `<CopilotPopup />` opens by default (popup overlay is visible)
|
||||
- [ ] Verify the chat input placeholder reads "Ask the popup anything..."
|
||||
- [ ] Send a basic message (e.g. "Say hi from the popup!")
|
||||
- [ ] Verify the agent responds in the popup
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Suggestions
|
||||
|
||||
- [ ] Verify the "Say hi" suggestion button is visible inside the popup
|
||||
- [ ] Click the suggestion and verify it sends the message
|
||||
|
||||
#### Popup Toggle
|
||||
|
||||
- [ ] Close the popup via its toggle/launcher
|
||||
- [ ] Verify the floating launcher bubble remains on the page
|
||||
- [ ] Reopen the popup and verify the conversation is preserved
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Verify no console errors during normal usage
|
||||
- [ ] Verify closing/opening the popup does not break the chat state
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Page loads within 3 seconds
|
||||
- Popup opens with floating launcher
|
||||
- Agent responds within 10 seconds
|
||||
@@ -0,0 +1,40 @@
|
||||
# QA: Pre-Built Sidebar — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/prebuilt-sidebar`
|
||||
- [ ] Verify the main content heading "Sidebar demo — click the launcher" is visible
|
||||
- [ ] Verify `<CopilotSidebar />` opens by default (sidebar is visible)
|
||||
- [ ] Verify the sidebar has a chat input
|
||||
- [ ] Send a basic message (e.g. "Say hi")
|
||||
- [ ] Verify the agent responds in the sidebar
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Suggestions
|
||||
|
||||
- [ ] Verify the "Say hi" suggestion button is visible inside the sidebar
|
||||
- [ ] Click the suggestion and verify it sends the message
|
||||
|
||||
#### Sidebar Toggle
|
||||
|
||||
- [ ] Close the sidebar via its toggle/launcher
|
||||
- [ ] Reopen the sidebar and verify the conversation is preserved
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Verify no console errors during normal usage
|
||||
- [ ] Verify closing/opening the sidebar does not break the chat state
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Page loads within 3 seconds
|
||||
- Sidebar renders with launcher
|
||||
- Agent responds within 10 seconds
|
||||
@@ -0,0 +1,33 @@
|
||||
# QA: Readonly State (Agent Context) — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy
|
||||
- ANTHROPIC_API_KEY is set
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/readonly-state-agent-context`
|
||||
- [ ] Verify the Agent Context card renders with Name, Timezone, Recent Activity controls
|
||||
- [ ] Verify the published context JSON pre is visible
|
||||
|
||||
### 2. Context Wiring
|
||||
|
||||
- [ ] Edit the Name input to "Dana"
|
||||
- [ ] Click the "Who am I?" suggestion
|
||||
- [ ] Verify the agent greets "Dana" in its reply
|
||||
- [ ] Change the timezone to "Asia/Tokyo"
|
||||
- [ ] Click "Plan my morning" — verify the agent references Tokyo time
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Verify no console errors on context updates
|
||||
- [ ] Verify the chat remains usable after multiple context changes
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Agent references the current Name, Timezone, and Recent Activity
|
||||
- The context card cannot be modified by the agent (agent has no write tools)
|
||||
@@ -0,0 +1,61 @@
|
||||
# QA: Shared State (Read + Write) — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible at `/demos/shared-state-read-write` on the dashboard host
|
||||
- Agent backend is healthy (`/api/health`); `ANTHROPIC_API_KEY` is set on Railway; the FastAPI backend exposes `POST /shared-state-read-write`
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/shared-state-read-write`; verify the page renders within 3s with the left sidebar (preferences + notes cards) and the right-side `CopilotChat` pane
|
||||
- [ ] Verify `data-testid="preferences-card"` is visible with heading "Your preferences"
|
||||
- [ ] Verify `data-testid="notes-card"` is visible with heading "Agent notes" and empty-state `data-testid="notes-empty"` reading "No notes yet. Ask the agent to remember something."
|
||||
- [ ] Verify the chat input placeholder is "Chat with the agent..."
|
||||
- [ ] Verify all 3 suggestion pills are visible with verbatim titles: "Greet me", "Remember something", "Plan a weekend"
|
||||
- [ ] Send "Hello" and verify an assistant text response appears within 10s
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### UI Writes -> Agent Reads (preferences via `agent.setState`)
|
||||
|
||||
- [ ] Type "Atai" into `data-testid="pref-name"`; verify `data-testid="pref-state-json"` updates to include `"name": "Atai"`
|
||||
- [ ] Change `data-testid="pref-tone"` to `formal`; verify the JSON preview reflects `"tone": "formal"`
|
||||
- [ ] Change `data-testid="pref-language"` to `Spanish`; verify the JSON preview reflects `"language": "Spanish"`
|
||||
- [ ] Click the `Cooking` and `Travel` interest pills; verify both show the selected style (border `#BEC2FF`, bg `#BEC2FF1A`) and the JSON preview's `interests` array contains both entries
|
||||
- [ ] Send "What do you know about me?"; verify within 15s the assistant reply references the name "Atai", a formal tone, Spanish, and the Cooking/Travel interests (the Python backend's `build_preferences_block` injects these into the system prompt each turn)
|
||||
- [ ] Click the "Plan a weekend" suggestion; verify the reply is tailored to the selected interests
|
||||
|
||||
#### Agent Writes -> UI Reads (notes via `set_notes` tool)
|
||||
|
||||
- [ ] Click the "Remember something" suggestion (sends "Remember that I prefer morning meetings and that I don't eat dairy.")
|
||||
- [ ] Within 20s verify `data-testid="notes-list"` appears in the notes card and contains at least 2 `data-testid="note-item"` entries mentioning "morning meetings" and "dairy"
|
||||
- [ ] Verify `data-testid="notes-empty"` is no longer rendered
|
||||
- [ ] Send "Also remember I live in Berlin."; verify within 20s the notes list grows (previous notes preserved, new note added) — confirms the agent passes the FULL updated list per `set_notes` contract
|
||||
|
||||
#### UI Writes Back to Agent-Authored Slice (clear notes)
|
||||
|
||||
- [ ] With notes present, verify `data-testid="notes-clear-button"` is visible
|
||||
- [ ] Click the Clear button; verify the notes list disappears and `data-testid="notes-empty"` re-renders
|
||||
- [ ] Ask "What do you remember about me?"; verify the agent no longer cites the cleared notes (state was written back by the UI via `agent.setState({ notes: [] })`)
|
||||
|
||||
#### Multi-Turn State Persistence
|
||||
|
||||
- [ ] Change tone to `playful` and add the `Music` interest; send "Write me a one-line haiku greeting."; verify the reply is playful and references music
|
||||
- [ ] Send a follow-up "Do it again in French."; verify the reply remains playful, switches to French, and still acknowledges the music interest — confirms preferences persist across turns
|
||||
- [ ] Reload the page; verify preferences reset to defaults (`tone: casual`, `language: English`, empty interests, empty name) and notes reset to empty (state is per-session)
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Attempt to send an empty message; verify it is a no-op (no user bubble, no assistant response)
|
||||
- [ ] Deselect all interests and clear the name; send "Who am I?"; verify the agent answers without crashing (the preferences block is skipped when nothing recognised is set)
|
||||
- [ ] Verify DevTools -> Console shows no uncaught errors during any flow above
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Page loads within 3 seconds; assistant text response within 10 seconds for plain chat
|
||||
- Preferences writes are reflected in `pref-state-json` synchronously on change
|
||||
- Agent-authored notes appear in `notes-card` within 20 seconds of a "remember" prompt, and the full prior list is preserved on subsequent `set_notes` calls
|
||||
- Clear button round-trips UI -> agent state and the agent loses access to the cleared notes on the next turn
|
||||
- No UI layout breaks, no uncaught console errors
|
||||
@@ -0,0 +1,75 @@
|
||||
# QA: Shared State (Reading) — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to the shared-state-read demo page
|
||||
- [ ] Verify the recipe card form loads (`data-testid="recipe-card"`)
|
||||
- [ ] Verify the CopilotSidebar opens by default with title "AI Recipe Assistant"
|
||||
- [ ] Send a message via the sidebar
|
||||
- [ ] Verify the agent responds
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Initial Recipe State
|
||||
|
||||
- [ ] Verify the recipe title input shows "Make Your Recipe"
|
||||
- [ ] Verify the cooking time dropdown defaults to "45 min"
|
||||
- [ ] Verify the skill level dropdown defaults to "Intermediate"
|
||||
- [ ] Verify the default ingredients are displayed:
|
||||
- [ ] Carrots (3 large, grated) with carrot emoji
|
||||
- [ ] All-Purpose Flour (2 cups) with wheat emoji
|
||||
- [ ] Verify the default instruction is displayed: "Preheat oven to 350 F"
|
||||
|
||||
#### Suggestions
|
||||
|
||||
- [ ] Verify "Create Italian recipe" suggestion is visible
|
||||
- [ ] Verify "Make it healthier" suggestion is visible
|
||||
- [ ] Verify "Suggest variations" suggestion is visible
|
||||
|
||||
#### Recipe Editing (Local State)
|
||||
|
||||
- [ ] Edit the recipe title and verify it updates
|
||||
- [ ] Change the skill level dropdown and verify it updates
|
||||
- [ ] Change the cooking time dropdown and verify it updates
|
||||
- [ ] Toggle a dietary preference checkbox (e.g. "Vegetarian") and verify it's checked
|
||||
- [ ] Click "+ Add Ingredient" (`data-testid="add-ingredient-button"`) and verify a new empty row appears
|
||||
- [ ] Edit an ingredient name and amount
|
||||
- [ ] Remove an ingredient by clicking the "x" button
|
||||
- [ ] Click "+ Add Step" and verify a new instruction row appears
|
||||
- [ ] Edit an instruction and verify it saves
|
||||
- [ ] Remove an instruction by clicking the "x" button
|
||||
|
||||
#### AI-Powered Recipe Updates (useAgent with shared state)
|
||||
|
||||
- [ ] Click "Create Italian recipe" suggestion
|
||||
- [ ] Verify the agent updates the recipe title, ingredients, and instructions
|
||||
- [ ] Verify the ping indicator appears on changed sections
|
||||
- [ ] Verify the "Improve with AI" button (`data-testid="improve-button"`) changes to "Please Wait..." while loading
|
||||
- [ ] Click "Improve with AI" and verify the recipe is enhanced
|
||||
|
||||
#### Agent Reads Frontend State
|
||||
|
||||
- [ ] Edit the recipe (change title, add ingredients)
|
||||
- [ ] Ask the agent "What recipe am I making?"
|
||||
- [ ] Verify the agent's response references the current recipe state
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Send an empty message (should be handled gracefully)
|
||||
- [ ] Verify no console errors during normal usage
|
||||
- [ ] Verify the "Improve with AI" button is disabled while loading
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Recipe card and sidebar load within 3 seconds
|
||||
- Agent responds within 10 seconds
|
||||
- Recipe state syncs bidirectionally between UI and agent
|
||||
- Ping indicators highlight changed sections
|
||||
- No UI errors or broken layouts
|
||||
@@ -0,0 +1,41 @@
|
||||
# QA: State Streaming — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to the shared-state-streaming demo page
|
||||
- [ ] Verify the chat interface loads with title "State Streaming"
|
||||
- [ ] Verify the chat input placeholder "Type a message..." is visible
|
||||
- [ ] Send a basic message (e.g. "Hello! What can you do?")
|
||||
- [ ] Verify the agent responds
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Suggestions
|
||||
|
||||
- [ ] Verify "Get started" suggestion button is visible
|
||||
|
||||
#### Note: Stub Demo
|
||||
|
||||
> **Status: Stub** — This demo is currently a stub (TODO: implement)
|
||||
|
||||
- [ ] Verify the basic CopilotChat loads and accepts messages
|
||||
- [ ] Verify the agent responds to messages
|
||||
- [ ] No custom UI components are expected beyond the chat interface
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Send an empty message (should be handled gracefully)
|
||||
- [ ] Verify no console errors during normal usage
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Chat loads within 3 seconds
|
||||
- Agent responds within 10 seconds
|
||||
- No UI errors or broken layouts
|
||||
@@ -0,0 +1,41 @@
|
||||
# QA: Shared State (Writing) — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to the shared-state-write demo page
|
||||
- [ ] Verify the chat interface loads with title "Shared State (Writing)"
|
||||
- [ ] Verify the chat input placeholder "Type a message..." is visible
|
||||
- [ ] Send a basic message (e.g. "Hello! What can you do?")
|
||||
- [ ] Verify the agent responds
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Suggestions
|
||||
|
||||
- [ ] Verify "Get started" suggestion button is visible
|
||||
|
||||
#### Note: Stub Demo
|
||||
|
||||
> **Status: Stub** — This demo is currently a stub (TODO: implement)
|
||||
|
||||
- [ ] Verify the basic CopilotChat loads and accepts messages
|
||||
- [ ] Verify the agent responds to messages
|
||||
- [ ] No custom UI components are expected beyond the chat interface
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Send an empty message (should be handled gracefully)
|
||||
- [ ] Verify no console errors during normal usage
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Chat loads within 3 seconds
|
||||
- Agent responds within 10 seconds
|
||||
- No UI errors or broken layouts
|
||||
@@ -0,0 +1,60 @@
|
||||
# QA: Sub-Agents — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible at `/demos/subagents` on the dashboard host
|
||||
- Agent backend is healthy (`/api/health`); `ANTHROPIC_API_KEY` is set on Railway; the FastAPI backend exposes `POST /subagents`
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/subagents`; verify the page renders within 3s with a left-side delegation log and a right-side `CopilotChat` pane
|
||||
- [ ] Verify `data-testid="delegation-log"` is visible with heading "Sub-agent delegations"
|
||||
- [ ] Verify `data-testid="delegation-count"` reads "0 calls" on first load
|
||||
- [ ] Verify the empty-state placeholder reads "Ask the supervisor to complete a task. Every sub-agent it calls will appear here."
|
||||
- [ ] Verify the chat input placeholder is "Give the supervisor a task..."
|
||||
- [ ] Verify all 3 suggestion pills are visible with verbatim titles: "Write a blog post", "Explain a topic", "Summarize a topic"
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Multi-stage Delegation Flow
|
||||
|
||||
- [ ] Click the "Write a blog post" suggestion (cold-exposure training prompt)
|
||||
- [ ] Within 5s verify `data-testid="supervisor-running"` appears with the "Supervisor running" pulse indicator
|
||||
- [ ] Within 30s verify at least one `data-testid="delegation-entry"` appears with badge "Research" and `data-testid="delegation-status"` initially reading `running`
|
||||
- [ ] Verify the entry's status flips to `completed` once the sub-agent returns and the `result` text is visible inside the entry's white inner panel
|
||||
- [ ] Within 60s total verify additional delegation entries appear in order: Research -> Writing -> Critique (3 entries total in most cases)
|
||||
- [ ] Verify `data-testid="delegation-count"` updates to match the number of entries (e.g. "3 calls")
|
||||
- [ ] Verify the supervisor's final chat reply includes a brief summary of the produced deliverable
|
||||
|
||||
#### Delegation Entry Layout
|
||||
|
||||
- [ ] Each `data-testid="delegation-entry"` shows: a `#N` index, a sub-agent badge with the correct emoji (🔎 Research / ✍️ Writing / 🧐 Critique), a status chip, the task text after "Task:", and the sub-agent's result rendered with whitespace preserved
|
||||
- [ ] Hover the supervisor running chip while a delegation is in flight — verify the pulse animation is present (no static-only state)
|
||||
|
||||
#### Sub-Agent Independence
|
||||
|
||||
- [ ] Click "Explain a topic" (LLM tool calling prompt) and wait for completion
|
||||
- [ ] Verify the writing entry's `result` is a single polished paragraph (the writing sub-agent's signature)
|
||||
- [ ] Verify the research entry's `result` is a bulleted list of 3-5 facts (the research sub-agent's signature)
|
||||
- [ ] Verify the critique entry's `result` contains 2-3 actionable critiques
|
||||
|
||||
#### Supervisor State Reset Across Tasks
|
||||
|
||||
- [ ] After the first run completes, click "Summarize a topic" (reusable rockets)
|
||||
- [ ] Verify NEW delegation entries are appended to the existing list (count keeps growing) — confirms `state["delegations"]` accumulates across turns within the same thread
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Send an empty message; verify it is a no-op
|
||||
- [ ] If a sub-agent call fails (e.g. due to upstream rate limit), verify the failing entry is rendered with status `failed` and a result line starting with "sub-agent call failed:" — confirms the fail-loud path
|
||||
- [ ] Verify DevTools -> Console shows no uncaught errors during any flow above
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Page loads within 3 seconds
|
||||
- First delegation entry appears within 30 seconds of submitting a non-trivial task
|
||||
- Each delegation entry transitions from `running` -> `completed` (or `failed`) and the count badge stays in sync with `state["delegations"].length`
|
||||
- Supervisor's final chat reply summarises the work and arrives within 90 seconds of submission
|
||||
- No UI layout breaks, no uncaught console errors
|
||||
@@ -0,0 +1,30 @@
|
||||
# QA: Tool Rendering (Custom Catch-all) — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy
|
||||
- ANTHROPIC_API_KEY is set
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/tool-rendering-custom-catchall`
|
||||
- [ ] Verify the chat renders
|
||||
- [ ] Click "Weather in SF" — verify the agent calls `get_weather`
|
||||
- [ ] Verify the BRANDED `CustomCatchallRenderer` card appears (not the default)
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
- [ ] Verify `data-testid="custom-catchall-card"` renders with the tool name
|
||||
- [ ] Verify status pill transitions: streaming → running → done
|
||||
- [ ] Try "Find flights" — verify the same branded card renders
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Verify no console errors
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Every tool call renders via the custom `CustomCatchallRenderer`, including tool name, arguments pre, and result pre
|
||||
@@ -0,0 +1,29 @@
|
||||
# QA: Tool Rendering (Default Catch-all) — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy
|
||||
- ANTHROPIC_API_KEY is set
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/tool-rendering-default-catchall`
|
||||
- [ ] Verify the chat renders
|
||||
- [ ] Click "Weather in SF" — verify the agent calls `get_weather`
|
||||
- [ ] Verify the BUILT-IN `DefaultToolCallRenderer` card appears inline
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
- [ ] Verify the tool-call card shows the tool name, status pill (Running → Done), and Arguments/Result sections
|
||||
- [ ] Try "Find flights" — verify multiple tool calls render
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Verify no console errors during tool calls
|
||||
|
||||
## Expected Results
|
||||
|
||||
- CopilotKit's package-provided default tool card renders every tool call (no custom renderers)
|
||||
@@ -0,0 +1,61 @@
|
||||
# QA: Tool Rendering — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to the tool-rendering demo page
|
||||
- [ ] Verify the chat interface loads in a centered full-height layout
|
||||
- [ ] Verify the chat input placeholder "Type a message" is visible
|
||||
- [ ] Send a basic message
|
||||
- [ ] Verify the agent responds
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Suggestions
|
||||
|
||||
- [ ] Verify "Weather in San Francisco" suggestion button is visible
|
||||
- [ ] Verify "Weather in New York" suggestion button is visible
|
||||
- [ ] Verify "Weather in Tokyo" suggestion button is visible
|
||||
- [ ] Click a weather suggestion and verify it populates the input or sends the message
|
||||
|
||||
#### Weather Card Rendering (useRenderTool)
|
||||
|
||||
- [ ] Type "What's the weather in San Francisco?"
|
||||
- [ ] Verify loading state shows "Retrieving weather..." with a spinner
|
||||
- [ ] Verify the WeatherCard renders (`data-testid="weather-card"`) with:
|
||||
- [ ] City name displayed (`data-testid="weather-city"`)
|
||||
- [ ] Temperature in both Celsius and Fahrenheit
|
||||
- [ ] Humidity percentage (`data-testid="weather-humidity"`)
|
||||
- [ ] Wind speed in mph (`data-testid="weather-wind"`)
|
||||
- [ ] Feels-like temperature (`data-testid="weather-feels-like"`)
|
||||
- [ ] Conditions text with appropriate weather icon (sun/rain/cloud)
|
||||
- [ ] Verify the card background color matches the weather condition theme:
|
||||
- Clear/Sunny: #667eea (blue-purple)
|
||||
- Rain/Storm: #4A5568 (dark gray)
|
||||
- Cloudy: #718096 (medium gray)
|
||||
- Snow: #63B3ED (light blue)
|
||||
|
||||
#### Multiple Weather Queries
|
||||
|
||||
- [ ] Ask about weather in a second city
|
||||
- [ ] Verify a second WeatherCard renders without breaking the first
|
||||
- [ ] Verify each card shows the correct city name
|
||||
|
||||
### 3. Error Handling
|
||||
|
||||
- [ ] Send an empty message (should be handled gracefully)
|
||||
- [ ] Verify no console errors during normal usage
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Chat loads within 3 seconds
|
||||
- Agent responds within 10 seconds
|
||||
- Weather cards render with all data fields populated
|
||||
- Weather icon and theme color match the conditions
|
||||
- No UI errors or broken layouts
|
||||
@@ -0,0 +1,53 @@
|
||||
# QA: Voice input — Claude Agent SDK (Python)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Demo is deployed and accessible
|
||||
- Agent backend is healthy (check /api/health)
|
||||
- `ANTHROPIC_API_KEY` is set on the deployment (for chat replies)
|
||||
- `OPENAI_API_KEY` is set on the Next.js runtime (for Whisper
|
||||
transcription). If it is not, the mic button still renders but
|
||||
clicking it (or clicking "Play sample") will return a clean 401.
|
||||
|
||||
## Test Steps
|
||||
|
||||
### 1. Basic Functionality
|
||||
|
||||
- [ ] Navigate to `/demos/voice`
|
||||
- [ ] Verify the "Voice input" header renders
|
||||
- [ ] Verify the mic button is visible on the chat composer
|
||||
- [ ] Verify the "Play sample" button renders next to a caption
|
||||
matching `SAMPLE_LABEL` (e.g. "What is the weather in Tokyo?")
|
||||
|
||||
### 2. Feature-Specific Checks
|
||||
|
||||
#### Play sample
|
||||
|
||||
- [ ] Click "Play sample"
|
||||
- [ ] Verify the button flips to "Transcribing…" briefly
|
||||
- [ ] Verify the composer's textarea is populated with the transcribed
|
||||
text
|
||||
- [ ] Press send and verify Claude replies to the transcribed prompt
|
||||
|
||||
#### Mic button (manual)
|
||||
|
||||
- [ ] Click the mic button, allow microphone access, speak a short
|
||||
sentence, click again
|
||||
- [ ] Verify the textarea is populated with the transcription
|
||||
- [ ] Press send and verify Claude replies
|
||||
|
||||
### 3. Misconfigured deployment
|
||||
|
||||
- [ ] With `OPENAI_API_KEY` unset, click "Play sample"
|
||||
- [ ] Verify the error slot renders "Error — see console"
|
||||
- [ ] Verify the network response is HTTP 401, not 500/503
|
||||
|
||||
### 4. Error Handling
|
||||
|
||||
- [ ] No console errors during normal usage (other than the expected
|
||||
401 when `OPENAI_API_KEY` is missing).
|
||||
|
||||
## Expected Results
|
||||
|
||||
- Mic button always visible (transcription service is always mounted).
|
||||
- Missing `OPENAI_API_KEY` manifests as 401, not silent failure.
|
||||
Reference in New Issue
Block a user