chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,284 @@
|
||||
# CopilotKit Architecture Guide
|
||||
|
||||
CopilotKit lets you add AI agents to your app. You write hooks (React/Angular) or use the core API (vanilla JS), CopilotKit handles the rest — connecting your UI to any AI agent framework.
|
||||
|
||||
---
|
||||
|
||||
## The 30-Second Version
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph Your App
|
||||
A[React / Angular / Vanilla JS]
|
||||
end
|
||||
|
||||
subgraph Your Server
|
||||
B[CopilotKit Runtime]
|
||||
end
|
||||
|
||||
subgraph Any Agent Framework
|
||||
C[LangGraph / CrewAI / Mastra / Custom]
|
||||
end
|
||||
|
||||
A -->|HTTP POST| B
|
||||
B -->|AG-UI Events| C
|
||||
C -->|AG-UI Events| B
|
||||
B -->|SSE Stream| A
|
||||
```
|
||||
|
||||
That's it. Your app talks to a runtime on your server. The runtime talks to an AI agent. They communicate using **AG-UI** — an event-based protocol (think: "text is streaming", "agent wants to call a tool", "state changed").
|
||||
|
||||
---
|
||||
|
||||
## The Three Layers
|
||||
|
||||
### Layer 1: Frontend (your app)
|
||||
|
||||
You use hooks/services to wire up your app — registering tools agents can call, providing context, and getting agent instances.
|
||||
|
||||
### Layer 2: Runtime (your server)
|
||||
|
||||
A few lines create the backend that receives requests from the frontend, runs agents, and streams events back.
|
||||
|
||||
### Layer 3: Agent (any framework)
|
||||
|
||||
The agent is anything that speaks AG-UI protocol. CopilotKit has integrations for 13+ frameworks, or you build your own.
|
||||
|
||||
---
|
||||
|
||||
## How a Message Flows Through the System
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User
|
||||
participant App as Your App
|
||||
participant Core as CopilotKitCore
|
||||
participant Runtime as CopilotRuntime
|
||||
participant Agent as AI Agent
|
||||
|
||||
Note over App: Setup (on mount)
|
||||
App->>Core: Provider creates Core
|
||||
Core->>Runtime: GET /info (fetch agent list)
|
||||
Runtime-->>Core: [{ name, description }]
|
||||
App->>Core: Hooks register tools + context
|
||||
|
||||
Note over User: User sends message
|
||||
User->>App: Types message, hits send
|
||||
App->>Core: Gets agent instance
|
||||
Core->>Runtime: POST /agent/{id}/run
|
||||
Runtime->>Agent: AgentRunner.run()
|
||||
|
||||
Note over Agent: Events stream back
|
||||
Agent-->>Runtime: TEXT_MESSAGE_START
|
||||
Agent-->>Runtime: TEXT_MESSAGE_CONTENT (streaming)
|
||||
Agent-->>Runtime: TEXT_MESSAGE_END
|
||||
Runtime-->>Core: SSE event stream
|
||||
Core-->>App: Subscribers fire, UI re-renders
|
||||
App-->>User: Chat shows streaming response
|
||||
|
||||
Note over Agent: Tool call (optional)
|
||||
Agent-->>Runtime: TOOL_CALL_START + ARGS
|
||||
Runtime-->>Core: SSE events
|
||||
Core->>Core: Execute frontend tool
|
||||
Core-->>Runtime: TOOL_CALL_RESULT
|
||||
Runtime->>Agent: Agent continues
|
||||
Agent-->>Runtime: RUN_FINISHED
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Guides
|
||||
|
||||
| Guide | What you'll learn |
|
||||
| ------------------------------------------ | ------------------------------------------------------- |
|
||||
| [React Setup](setup-react.md) | Provider, hooks, chat UI — full React integration |
|
||||
| [Angular Setup](setup-angular.md) | DI tokens, services, signals — full Angular integration |
|
||||
| [Vanilla JS Setup](setup-vanilla.md) | CopilotKitCore API without any framework |
|
||||
| [Runtime / Backend](setup-runtime.md) | Express/Hono endpoints, agents, runners, middleware |
|
||||
| [Multi-Agent Patterns](multi-agent.md) | Multiple agents, routing, agent-specific tools |
|
||||
| [Pluggable Architecture](plugin-points.md) | Every optional extension point with diagrams |
|
||||
|
||||
---
|
||||
|
||||
## Package Dependency Map
|
||||
|
||||
```mermaid
|
||||
graph BT
|
||||
subgraph AG-UI Protocol
|
||||
core["@ag-ui/core<br/><i>Types + Event schemas</i>"]
|
||||
client["@ag-ui/client<br/><i>AbstractAgent, HttpAgent, Middleware</i>"]
|
||||
encoder["@ag-ui/encoder<br/><i>SSE / Binary / Protobuf encoding</i>"]
|
||||
client --> core
|
||||
encoder --> core
|
||||
end
|
||||
|
||||
subgraph CopilotKit Packages
|
||||
shared["@copilotkit/shared<br/><i>Utils, types, constants</i>"]
|
||||
core["@copilotkit/core<br/><i>CopilotKitCore orchestrator</i>"]
|
||||
reactcore["@copilotkit/react-core<br/><i>Provider + hooks</i>"]
|
||||
reactui["@copilotkit/react-ui<br/><i>Chat, Popup, Sidebar</i>"]
|
||||
reacttextarea["@copilotkit/react-textarea<br/><i>AI text editing</i>"]
|
||||
gql["@copilotkit/runtime-client-gql<br/><i>urql GraphQL client</i>"]
|
||||
runtime["@copilotkit/runtime<br/><i>Express/Hono server + AgentRunner + Built-in agent</i>"]
|
||||
|
||||
core --> shared
|
||||
reactcore --> core
|
||||
reactcore --> gql
|
||||
reactui --> reactcore
|
||||
reacttextarea --> reactcore
|
||||
runtime --> shared
|
||||
reactcore -.-> client
|
||||
gql --> shared
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## AG-UI Protocol at a Glance
|
||||
|
||||
AG-UI is the communication contract between agents and UIs. Everything is an **event** streamed over SSE.
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Lifecycle
|
||||
RS[RUN_STARTED] --> SS[STEP_STARTED]
|
||||
SF[STEP_FINISHED] --> RF[RUN_FINISHED]
|
||||
end
|
||||
|
||||
subgraph Text
|
||||
TMS[TEXT_MESSAGE_START] --> TMC[TEXT_MESSAGE_CONTENT]
|
||||
TMC --> TME[TEXT_MESSAGE_END]
|
||||
end
|
||||
|
||||
subgraph Tools
|
||||
TCS[TOOL_CALL_START] --> TCA[TOOL_CALL_ARGS]
|
||||
TCA --> TCE[TOOL_CALL_END]
|
||||
TCE --> TCR[TOOL_CALL_RESULT]
|
||||
end
|
||||
|
||||
subgraph State
|
||||
SNP[STATE_SNAPSHOT]
|
||||
SD[STATE_DELTA]
|
||||
end
|
||||
|
||||
SS --> TMS
|
||||
TME --> TCS
|
||||
TCR --> SF
|
||||
```
|
||||
|
||||
| Package | Role | Key exports |
|
||||
| ---------------- | ---------------------------------------- | ----------------------------------------------------------------- |
|
||||
| `@ag-ui/core` | The contract — event types + data shapes | `EventType` enum, Zod schemas, `RunAgentInput`, `Message`, `Tool` |
|
||||
| `@ag-ui/client` | Client-side agent abstraction | `AbstractAgent`, `HttpAgent`, `Middleware`, re-exports core |
|
||||
| `@ag-ui/encoder` | Serializes events for transport | `EventEncoder` (SSE, binary, protobuf) |
|
||||
| `@ag-ui/proto` | Protobuf binary transport | `encode()`, `decode()` |
|
||||
|
||||
13+ framework integrations at `ag-ui/integrations/`: LangGraph, CrewAI, Mastra, Vercel AI SDK, Agno, AWS Strands, LlamaIndex, and more.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
**"I want to..."** — here's where to look:
|
||||
|
||||
### Setup & Configuration
|
||||
|
||||
| Goal | Package | Key file / API |
|
||||
| -------------------------------- | ------------------------ | -------------------------------------------- |
|
||||
| Set up a React app | `@copilotkit/react-core` | `<CopilotKit runtimeUrl="...">` provider |
|
||||
| Set up an Angular app | `@copilotkit/angular` | `provideCopilotKit({ runtimeUrl })` DI token |
|
||||
| Set up vanilla JS | `@copilotkit/core` | `new CopilotKitCore({ runtimeUrl })` |
|
||||
| Set up the backend (Express) | `@copilotkit/runtime` | `createCopilotEndpointExpress({ runtime })` |
|
||||
| Set up the backend (Hono) | `@copilotkit/runtime` | `createCopilotEndpointHono({ runtime })` |
|
||||
| Configure authentication headers | Provider / Core config | `headers: { Authorization: "Bearer ..." }` |
|
||||
| Forward cookies to runtime | Provider / Core config | `credentials: "include"` |
|
||||
|
||||
### Agent Communication
|
||||
|
||||
| Goal | Package | Key file / API |
|
||||
| ------------------------------- | ------------------------ | ---------------------------------------------- |
|
||||
| Get an agent instance (React) | `@copilotkit/react-core` | `useAgent({ agentId })` |
|
||||
| Get an agent instance (Angular) | `@copilotkit/angular` | `AgentStore` with signals |
|
||||
| Get an agent instance (vanilla) | `@copilotkit/core` | `copilotkit.getAgent(id)` |
|
||||
| Run an agent | Core / hooks | `copilotkit.runAgent({ agent })` |
|
||||
| Use multiple agents | Runtime config | `agents: { research: agent1, coding: agent2 }` |
|
||||
| Agent-specific tools | `useFrontendTool` | `{ name, agentId: "specific-agent", handler }` |
|
||||
| Shared context for all agents | `useAgentContext` | `useAgentContext("desc", value)` |
|
||||
|
||||
### Tools & Interactivity
|
||||
|
||||
| Goal | Package | Key file / API |
|
||||
| ------------------------------- | ------------------------ | ------------------------------------------------ |
|
||||
| Register a tool agents can call | `react-core` or `react` | `useFrontendTool({ name, parameters, handler })` |
|
||||
| Give agents context data | `react-core` or `react` | `useCopilotReadable()` / `useAgentContext()` |
|
||||
| Share state with an agent (V1) | `@copilotkit/react-core` | `useCoAgent({ name, initialState })` |
|
||||
| Custom UI for tool execution | Provider or hook | `renderToolCalls` / `useRenderToolCall()` |
|
||||
| Require human approval | Provider or hook | `humanInTheLoop` / `useHumanInTheLoop()` |
|
||||
| Auto-generate suggestions | Hook | `useConfigureSuggestions({ instructions })` |
|
||||
| Inject system instructions (V1) | `@copilotkit/react-core` | `useCopilotAdditionalInstructions()` |
|
||||
|
||||
### UI Components
|
||||
|
||||
| Goal | Package | Component |
|
||||
| ---------------------- | ---------------------------- | ------------------- |
|
||||
| Full chat interface | `@copilotkit/react-ui` | `<CopilotChat>` |
|
||||
| Floating popup chat | `@copilotkit/react-ui` | `<CopilotPopup>` |
|
||||
| Side panel chat | `@copilotkit/react-ui` | `<CopilotSidebar>` |
|
||||
| Inline panel chat | `@copilotkit/react-ui` | `<CopilotPanel>` |
|
||||
| AI text autocompletion | `@copilotkit/react-textarea` | `<CopilotTextarea>` |
|
||||
|
||||
### Backend & Runtime
|
||||
|
||||
| Goal | Package | Key file / API |
|
||||
| ---------------------------- | --------------------------- | ---------------------------------------------------- |
|
||||
| Custom agent runner | `@copilotkit/runtime` | Extend `AgentRunner` abstract class |
|
||||
| Persistent agent state | `@copilotkit/sqlite-runner` | `SQLiteAgentRunner` |
|
||||
| Request/response middleware | `CopilotRuntime` options | `beforeRequestMiddleware` / `afterRequestMiddleware` |
|
||||
| Audio transcription | `CopilotRuntime` options | `transcriptionService` |
|
||||
| Voice (speech-to-text / TTS) | `@copilotkit/voice` | Voice services |
|
||||
| Build a custom agent | `@copilotkit/sdk-js` | LangGraph / LangChain helpers |
|
||||
|
||||
### Debugging & Internals
|
||||
|
||||
| Goal | Package | Key file / API |
|
||||
| -------------------------------- | --------------------------------- | -------------------------------------------------------------- |
|
||||
| Understand event types | `@ag-ui/core` | `src/events.ts` — `EventType` enum |
|
||||
| Understand the agent abstraction | `@ag-ui/client` | `src/agent/agent.ts` — `AbstractAgent` |
|
||||
| See how an integration works | `ag-ui/integrations/{framework}/` | Each extends `AbstractAgent` |
|
||||
| Understand the core orchestrator | `@copilotkit/core` | `src/core/core.ts` — `CopilotKitCore` |
|
||||
| Debug agent interactions | `@copilotkit/web-inspector` | Lit web component, enabled via `showDevConsole` |
|
||||
| Subscribe to lifecycle events | Core API | `copilotkit.subscribe({ onError, onToolExecutionStart, ... })` |
|
||||
|
||||
---
|
||||
|
||||
## Monorepo Structure
|
||||
|
||||
```
|
||||
cpk/
|
||||
├── ag-ui/ # AG-UI Protocol (open standard)
|
||||
│ ├── sdks/typescript/packages/
|
||||
│ │ ├── core/ # @ag-ui/core — types + events
|
||||
│ │ ├── client/ # @ag-ui/client — AbstractAgent, HttpAgent
|
||||
│ │ ├── encoder/ # @ag-ui/encoder — SSE/binary encoding
|
||||
│ │ └── proto/ # @ag-ui/proto — protobuf
|
||||
│ └── integrations/ # 13+ framework adapters
|
||||
│ ├── langgraph/
|
||||
│ ├── crewai/
|
||||
│ ├── mastra/
|
||||
│ └── ...
|
||||
│
|
||||
└── CopilotKit/ # CopilotKit Product
|
||||
└── packages/ # All packages flat under @copilotkit/ scope
|
||||
├── shared/ # @copilotkit/shared — utils, types, constants
|
||||
├── core/ # @copilotkit/core — CopilotKitCore orchestrator
|
||||
├── react-core/ # @copilotkit/react-core — provider + hooks
|
||||
├── react-ui/ # @copilotkit/react-ui — chat components
|
||||
├── react-textarea/ # @copilotkit/react-textarea — AI text editing
|
||||
├── runtime/ # @copilotkit/runtime — Express/Hono server + AgentRunner + Built-in agent
|
||||
├── runtime-client-gql/ # @copilotkit/runtime-client-gql — urql GraphQL client
|
||||
├── angular/ # @copilotkit/angular — Angular integration
|
||||
├── voice/ # @copilotkit/voice — voice support
|
||||
├── web-inspector/ # @copilotkit/web-inspector — debug console
|
||||
├── sqlite-runner/ # @copilotkit/sqlite-runner — persistent AgentRunner
|
||||
└── sdk-js/ # @copilotkit/sdk-js — LangGraph/LangChain helpers
|
||||
```
|
||||
@@ -0,0 +1,450 @@
|
||||
# Multi-Agent Patterns Guide
|
||||
|
||||
This guide shows how to use multiple agents in CopilotKit — from basic routing to agent-specific tools and shared context.
|
||||
|
||||
---
|
||||
|
||||
## How Multi-Agent Routing Works
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant React as React App
|
||||
participant Core as CopilotKitCore
|
||||
participant Runtime as CopilotRuntime
|
||||
participant Research as Research Agent
|
||||
participant Coding as Coding Agent
|
||||
|
||||
Note over React: On mount
|
||||
Core->>Runtime: GET /info
|
||||
Runtime-->>Core: agents: [{ id: "research" }, { id: "coding" }]
|
||||
Core->>Core: Create ProxiedAgent for each
|
||||
|
||||
Note over React: User picks "research"
|
||||
React->>Core: useAgent({ agentId: "research" })
|
||||
Core-->>React: ProxiedAgent(research)
|
||||
|
||||
Note over React: User sends message
|
||||
React->>Core: runAgent({ agent: researchAgent })
|
||||
Core->>Runtime: POST /agent/research/run
|
||||
Runtime->>Research: runner.run()
|
||||
Research-->>React: SSE events
|
||||
|
||||
Note over React: User switches to "coding"
|
||||
React->>Core: useAgent({ agentId: "coding" })
|
||||
Core-->>React: ProxiedAgent(coding)
|
||||
React->>Core: runAgent({ agent: codingAgent })
|
||||
Core->>Runtime: POST /agent/coding/run
|
||||
Runtime->>Coding: runner.run()
|
||||
Coding-->>React: SSE events
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Backend: Register Multiple Agents
|
||||
|
||||
```typescript
|
||||
import { CopilotRuntime } from "@copilotkit/runtime";
|
||||
import { createCopilotEndpointExpress } from "@copilotkit/runtime/express";
|
||||
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: {
|
||||
// Each key is the agent ID
|
||||
default: generalAgent, // Fallback agent
|
||||
research: researchAgent, // Specialist for research
|
||||
coding: codingAgent, // Specialist for code
|
||||
writing: writingAgent, // Specialist for content
|
||||
},
|
||||
});
|
||||
|
||||
app.use("/api/copilotkit", createCopilotEndpointExpress({ runtime }));
|
||||
```
|
||||
|
||||
The runtime exposes each agent at its own endpoint:
|
||||
|
||||
| Agent ID | Run Endpoint |
|
||||
| ---------- | -------------------------- |
|
||||
| `default` | `POST /agent/default/run` |
|
||||
| `research` | `POST /agent/research/run` |
|
||||
| `coding` | `POST /agent/coding/run` |
|
||||
| `writing` | `POST /agent/writing/run` |
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph "Runtime Agent Map"
|
||||
M["agents: {<br/> default: Agent,<br/> research: Agent,<br/> coding: Agent<br/>}"]
|
||||
end
|
||||
|
||||
subgraph Endpoints
|
||||
E1["POST /agent/default/run"]
|
||||
E2["POST /agent/research/run"]
|
||||
E3["POST /agent/coding/run"]
|
||||
end
|
||||
|
||||
subgraph Agent Instances
|
||||
A1["General Agent"]
|
||||
A2["Research Agent"]
|
||||
A3["Coding Agent"]
|
||||
end
|
||||
|
||||
E1 -->|"agents['default']"| A1
|
||||
E2 -->|"agents['research']"| A2
|
||||
E3 -->|"agents['coding']"| A3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Frontend: Select an Agent
|
||||
|
||||
### React
|
||||
|
||||
```tsx
|
||||
import { useAgent } from "@copilotkit/react-core";
|
||||
|
||||
function ResearchPanel() {
|
||||
// Gets the "research" agent
|
||||
const { agent } = useAgent({ agentId: "research" });
|
||||
|
||||
const sendMessage = async (text: string) => {
|
||||
agent.addMessage({ id: crypto.randomUUID(), role: "user", content: text });
|
||||
await copilotKit.runAgent({ agent });
|
||||
};
|
||||
|
||||
return <div>{/* research UI */}</div>;
|
||||
}
|
||||
|
||||
function CodingPanel() {
|
||||
// Gets the "coding" agent
|
||||
const { agent } = useAgent({ agentId: "coding" });
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### Using CopilotChat with agent IDs
|
||||
|
||||
```tsx
|
||||
import { CopilotChat } from "@copilotkit/react-core";
|
||||
|
||||
function App() {
|
||||
return (
|
||||
<CopilotKitProvider runtimeUrl="/api/copilotkit">
|
||||
<div style={{ display: "flex" }}>
|
||||
{/* Two separate chats, each talking to a different agent */}
|
||||
<CopilotChat agentId="research" threadId="research-1" />
|
||||
<CopilotChat agentId="coding" threadId="coding-1" />
|
||||
</div>
|
||||
</CopilotKitProvider>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Angular
|
||||
|
||||
```typescript
|
||||
@Component({
|
||||
/* ... */
|
||||
})
|
||||
export class MultiAgentComponent {
|
||||
private copilotKit = inject(CopilotKit);
|
||||
|
||||
researchStore = this.copilotKit.getAgentStore("research");
|
||||
codingStore = this.copilotKit.getAgentStore("coding");
|
||||
}
|
||||
```
|
||||
|
||||
### Vanilla JS
|
||||
|
||||
```typescript
|
||||
const researchAgent = copilotKit.getAgent("research");
|
||||
const codingAgent = copilotKit.getAgent("coding");
|
||||
|
||||
// Each agent has its own messages, state, and thread
|
||||
await copilotKit.runAgent({ agent: researchAgent });
|
||||
await copilotKit.runAgent({ agent: codingAgent });
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## The DEFAULT_AGENT_ID
|
||||
|
||||
When you don't specify an `agentId`, CopilotKit uses `"default"`:
|
||||
|
||||
```typescript
|
||||
// These are equivalent:
|
||||
useAgent(); // Uses "default"
|
||||
useAgent({ agentId: "default" }); // Explicit
|
||||
|
||||
// Your backend must have a "default" agent:
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: {
|
||||
default: myAgent, // This is required if any component omits agentId
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Agent Discovery
|
||||
|
||||
On mount, the frontend fetches available agents from the runtime:
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Core as CopilotKitCore
|
||||
participant Runtime as CopilotRuntime
|
||||
|
||||
Core->>Runtime: GET /info
|
||||
Runtime-->>Core: { agents: { research: { description: "..." }, coding: { description: "..." } } }
|
||||
Core->>Core: Create ProxiedAgent for each
|
||||
Core->>Core: Notify subscribers (onAgentsChanged)
|
||||
```
|
||||
|
||||
You can react to agent changes:
|
||||
|
||||
```typescript
|
||||
copilotKit.subscribe({
|
||||
onAgentsChanged: ({ agents }) => {
|
||||
console.log("Available agents:", Object.keys(agents));
|
||||
// e.g. ["default", "research", "coding"]
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Agent-Specific Tools
|
||||
|
||||
Tools can be scoped to specific agents:
|
||||
|
||||
```tsx
|
||||
// This tool is available to ALL agents
|
||||
useFrontendTool({
|
||||
name: "getCurrentTime",
|
||||
handler: async () => new Date().toISOString(),
|
||||
});
|
||||
|
||||
// This tool is ONLY available to the "research" agent
|
||||
useFrontendTool({
|
||||
name: "searchPapers",
|
||||
agentId: "research",
|
||||
parameters: z.object({ query: z.string() }),
|
||||
handler: async ({ query }) => await searchPapers(query),
|
||||
});
|
||||
|
||||
// This tool is ONLY available to the "coding" agent
|
||||
useFrontendTool({
|
||||
name: "runCode",
|
||||
agentId: "coding",
|
||||
parameters: z.object({ code: z.string(), language: z.string() }),
|
||||
handler: async ({ code, language }) => await executeCode(code, language),
|
||||
});
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Tool Registry"
|
||||
GT["getCurrentTime<br/><i>All agents</i>"]
|
||||
SP["searchPapers<br/><i>research only</i>"]
|
||||
RC["runCode<br/><i>coding only</i>"]
|
||||
end
|
||||
|
||||
subgraph Agents
|
||||
RA["research agent"]
|
||||
CA["coding agent"]
|
||||
end
|
||||
|
||||
GT --> RA
|
||||
GT --> CA
|
||||
SP --> RA
|
||||
RC --> CA
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Shared Context
|
||||
|
||||
Context is shared across all agents by default:
|
||||
|
||||
```tsx
|
||||
function App() {
|
||||
// Both research and coding agents can see this
|
||||
useAgentContext("Current user", { name: "Alice", role: "developer" });
|
||||
useAgentContext("Current project", {
|
||||
name: "my-app",
|
||||
language: "TypeScript",
|
||||
});
|
||||
|
||||
return (
|
||||
<>
|
||||
<CopilotChat agentId="research" />
|
||||
<CopilotChat agentId="coding" />
|
||||
</>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Shared Context"
|
||||
C1["Current user: Alice"]
|
||||
C2["Current project: my-app"]
|
||||
end
|
||||
|
||||
subgraph Agents
|
||||
RA["research agent"]
|
||||
CA["coding agent"]
|
||||
end
|
||||
|
||||
C1 --> RA
|
||||
C1 --> CA
|
||||
C2 --> RA
|
||||
C2 --> CA
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Thread Isolation
|
||||
|
||||
Each agent conversation runs on its own thread:
|
||||
|
||||
```tsx
|
||||
// These are separate conversations with separate histories
|
||||
<CopilotChat agentId="research" threadId="research-thread-1" />
|
||||
<CopilotChat agentId="coding" threadId="coding-thread-1" />
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph "Thread: research-1"
|
||||
RM1["User: Find papers on AI"]
|
||||
RM2["Agent: Here are 5 papers..."]
|
||||
end
|
||||
|
||||
subgraph "Thread: coding-1"
|
||||
CM1["User: Write a sort function"]
|
||||
CM2["Agent: Here's a quicksort..."]
|
||||
end
|
||||
|
||||
RM1 --> RM2
|
||||
CM1 --> CM2
|
||||
```
|
||||
|
||||
Each thread maintains its own:
|
||||
|
||||
- Message history
|
||||
- Agent state
|
||||
- Running status
|
||||
|
||||
---
|
||||
|
||||
## Full Example: Multi-Agent Dashboard
|
||||
|
||||
### Backend
|
||||
|
||||
```typescript
|
||||
import { CopilotRuntime } from "@copilotkit/runtime";
|
||||
import { createCopilotEndpointExpress } from "@copilotkit/runtime/express";
|
||||
import { BuiltInAgent } from "@copilotkit/runtime/v2";
|
||||
|
||||
const agents = {
|
||||
default: new BuiltInAgent({
|
||||
model: "openai/gpt-4o",
|
||||
systemPrompt: "You are a general assistant.",
|
||||
}),
|
||||
research: new BuiltInAgent({
|
||||
model: "openai/gpt-4o",
|
||||
systemPrompt:
|
||||
"You are a research specialist. Search for papers and summarize findings.",
|
||||
}),
|
||||
coding: new BuiltInAgent({
|
||||
model: "openai/gpt-4o",
|
||||
systemPrompt: "You are a coding expert. Write clean, tested code.",
|
||||
}),
|
||||
};
|
||||
|
||||
const runtime = new CopilotRuntime({ agents });
|
||||
app.use("/api/copilotkit", createCopilotEndpointExpress({ runtime }));
|
||||
```
|
||||
|
||||
### Frontend (React)
|
||||
|
||||
```tsx
|
||||
import {
|
||||
CopilotKitProvider,
|
||||
CopilotChat,
|
||||
useAgent,
|
||||
useFrontendTool,
|
||||
useAgentContext,
|
||||
} from "@copilotkit/react-core";
|
||||
import { z } from "zod";
|
||||
|
||||
export default function App() {
|
||||
return (
|
||||
<CopilotKitProvider runtimeUrl="/api/copilotkit">
|
||||
<SharedContext />
|
||||
<div style={{ display: "grid", gridTemplateColumns: "1fr 1fr" }}>
|
||||
<ResearchPanel />
|
||||
<CodingPanel />
|
||||
</div>
|
||||
</CopilotKitProvider>
|
||||
);
|
||||
}
|
||||
|
||||
// Shared context — all agents see this
|
||||
function SharedContext() {
|
||||
useAgentContext("Current project", {
|
||||
name: "my-saas-app",
|
||||
stack: "React + Node.js + PostgreSQL",
|
||||
description: "A SaaS platform for team collaboration",
|
||||
});
|
||||
return null;
|
||||
}
|
||||
|
||||
// Research agent with its own tools
|
||||
function ResearchPanel() {
|
||||
useFrontendTool({
|
||||
name: "saveFindings",
|
||||
agentId: "research",
|
||||
description: "Save research findings to the knowledge base",
|
||||
parameters: z.object({
|
||||
title: z.string(),
|
||||
summary: z.string(),
|
||||
sources: z.array(z.string()),
|
||||
}),
|
||||
handler: async ({ title, summary, sources }) => {
|
||||
await knowledgeBase.save({ title, summary, sources });
|
||||
return "Saved to knowledge base";
|
||||
},
|
||||
});
|
||||
|
||||
return (
|
||||
<div>
|
||||
<h2>Research Assistant</h2>
|
||||
<CopilotChat agentId="research" />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// Coding agent with its own tools
|
||||
function CodingPanel() {
|
||||
useFrontendTool({
|
||||
name: "createFile",
|
||||
agentId: "coding",
|
||||
description: "Create a new file in the project",
|
||||
parameters: z.object({
|
||||
path: z.string(),
|
||||
content: z.string(),
|
||||
}),
|
||||
handler: async ({ path, content }) => {
|
||||
await fileSystem.write(path, content);
|
||||
return `Created ${path}`;
|
||||
},
|
||||
});
|
||||
|
||||
return (
|
||||
<div>
|
||||
<h2>Coding Assistant</h2>
|
||||
<CopilotChat agentId="coding" />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,521 @@
|
||||
# Pluggable Architecture Guide
|
||||
|
||||
CopilotKit is built around extension points. Almost everything is optional and replaceable. This guide catalogs **every** pluggable part, where it's configured, and what happens when you don't provide it.
|
||||
|
||||
---
|
||||
|
||||
## Overview: All Extension Points
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Frontend (React / Angular / Vanilla)"
|
||||
FT["Frontend Tools<br/><i>Functions agents can call</i>"]
|
||||
CTX["Agent Context<br/><i>Data agents can read</i>"]
|
||||
RTC["Tool Call Renderers<br/><i>Custom UI for tool calls</i>"]
|
||||
HIL["Human-in-the-Loop<br/><i>Approval before execution</i>"]
|
||||
RAM["Activity Renderers<br/><i>Custom activity messages</i>"]
|
||||
RCM["Custom Message Renderers<br/><i>Inject UI before/after messages</i>"]
|
||||
SUG["Suggestions Config<br/><i>AI or static suggestions</i>"]
|
||||
SUBS["Event Subscribers<br/><i>React to lifecycle events</i>"]
|
||||
end
|
||||
|
||||
subgraph "Backend (Runtime)"
|
||||
BM["Before Middleware<br/><i>Auth, logging, transforms</i>"]
|
||||
AM["After Middleware<br/><i>Post-processing</i>"]
|
||||
RUNNER["Agent Runner<br/><i>How agents execute</i>"]
|
||||
TS["Transcription Service<br/><i>Audio → text</i>"]
|
||||
end
|
||||
|
||||
subgraph "Agent Level"
|
||||
MW["AG-UI Middleware<br/><i>Intercept agent pipeline</i>"]
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Frontend Extension Points
|
||||
|
||||
### 1. Frontend Tools
|
||||
|
||||
**What:** Functions in your app that agents can call during a conversation.
|
||||
|
||||
**Where configured:**
|
||||
|
||||
- React: `useFrontendTool()` hook or `frontendTools` provider prop
|
||||
- Angular: `copilotKit.addTool()` or `tools` in config
|
||||
- Vanilla: `copilotKit.addTool()`
|
||||
|
||||
**Default when not provided:** No tools — agent can only send text messages.
|
||||
|
||||
```typescript
|
||||
// Type signature
|
||||
type FrontendTool<T> = {
|
||||
name: string;
|
||||
description?: string;
|
||||
parameters?: z.ZodType<T>;
|
||||
handler?: (args: T, context: FrontendToolHandlerContext) => Promise<unknown>;
|
||||
followUp?: boolean; // Re-run agent after tool completes
|
||||
agentId?: string; // Scope to specific agent
|
||||
};
|
||||
```
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Agent
|
||||
participant Core as CopilotKitCore
|
||||
participant Tool as Your Tool Handler
|
||||
|
||||
Agent->>Core: TOOL_CALL_START { name: "myTool" }
|
||||
Agent->>Core: TOOL_CALL_ARGS { ... }
|
||||
Core->>Tool: handler(args)
|
||||
Tool-->>Core: result
|
||||
Core->>Agent: TOOL_CALL_RESULT
|
||||
opt followUp = true
|
||||
Core->>Agent: Re-run agent with result
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Agent Context
|
||||
|
||||
**What:** JSON data that gets sent to agents as context (like "the user is on the settings page").
|
||||
|
||||
**Where configured:**
|
||||
|
||||
- React: `useAgentContext()` hook
|
||||
- Angular / Vanilla: `copilotKit.addContext()` / `removeContext()`
|
||||
|
||||
**Default when not provided:** No extra context — agent only sees messages and tool definitions.
|
||||
|
||||
```typescript
|
||||
type AgentContextInput = {
|
||||
description: string; // Human-readable label
|
||||
value: JsonSerializable; // Any JSON value
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Tool Call Renderers
|
||||
|
||||
**What:** Custom React components that render while a tool is being called — showing progress, args, and results.
|
||||
|
||||
**Where configured:**
|
||||
|
||||
- React: `useRenderToolCall()` hook or `renderToolCalls` provider prop
|
||||
- Angular: `renderToolCalls` in config
|
||||
|
||||
**Default when not provided:** Generic built-in rendering.
|
||||
|
||||
```typescript
|
||||
type ReactToolCallRenderer<T> = {
|
||||
name: string; // Tool name to render
|
||||
args: z.ZodSchema<T>; // Schema for type-safe args
|
||||
agentId?: string; // Scope to specific agent
|
||||
render: React.ComponentType<
|
||||
| { status: "in-progress"; args: Partial<T>; result: undefined }
|
||||
| { status: "executing"; args: T; result: undefined }
|
||||
| { status: "complete"; args: T; result: string }
|
||||
>;
|
||||
};
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
IP["in-progress<br/><i>Args streaming in<br/>Partial<T> available</i>"]
|
||||
EX["executing<br/><i>Handler running<br/>Full args available</i>"]
|
||||
CO["complete<br/><i>Result available</i>"]
|
||||
IP --> EX --> CO
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4. Human-in-the-Loop
|
||||
|
||||
**What:** Tools that pause and wait for user input before continuing. The user sees a custom UI with approve/deny buttons.
|
||||
|
||||
**Where configured:**
|
||||
|
||||
- React: `useHumanInTheLoop()` hook or `humanInTheLoop` provider prop
|
||||
- Angular: `humanInTheLoop` in config
|
||||
|
||||
**Default when not provided:** No approval required — tools execute immediately.
|
||||
|
||||
```typescript
|
||||
type ReactHumanInTheLoop<T> = Omit<FrontendTool<T>, "handler"> & {
|
||||
render: React.ComponentType<{
|
||||
args: T;
|
||||
status: "in-progress" | "executing" | "complete";
|
||||
respond: (result: unknown) => Promise<void>; // Call this to approve/deny
|
||||
}>;
|
||||
};
|
||||
```
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Agent
|
||||
participant Core as CopilotKitCore
|
||||
participant UI as Your Approval UI
|
||||
participant User
|
||||
|
||||
Agent->>Core: TOOL_CALL { name: "deleteUser" }
|
||||
Core->>UI: Render with status: "executing"
|
||||
UI->>User: "Delete user X?"
|
||||
User->>UI: Clicks "Approve"
|
||||
UI->>Core: respond("approved")
|
||||
Core->>Agent: TOOL_CALL_RESULT
|
||||
Agent->>Agent: Continues
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. Activity Message Renderers
|
||||
|
||||
**What:** Custom UI for structured activity messages (non-chat messages like progress indicators or MCP app outputs).
|
||||
|
||||
**Where configured:**
|
||||
|
||||
- React: `useRenderActivityMessage()` hook or `renderActivityMessages` provider prop
|
||||
|
||||
**Default when not provided:** Built-in MCP Apps renderer is included. Other activity types show generic display.
|
||||
|
||||
```typescript
|
||||
type ReactActivityMessageRenderer<T> = {
|
||||
activityType: string; // Use "*" for wildcard
|
||||
agentId?: string;
|
||||
content: z.ZodSchema<T>;
|
||||
render: React.ComponentType<{
|
||||
activityType: string;
|
||||
content: T;
|
||||
message: ActivityMessage;
|
||||
agent: AbstractAgent | undefined;
|
||||
}>;
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 6. Custom Message Renderers
|
||||
|
||||
**What:** Inject custom UI before or after specific messages (e.g., add a "copy" button, show state snapshots).
|
||||
|
||||
**Where configured:**
|
||||
|
||||
- React: `useRenderCustomMessages()` hook or `renderCustomMessages` provider prop
|
||||
|
||||
**Default when not provided:** No custom rendering — standard message display.
|
||||
|
||||
```typescript
|
||||
type ReactCustomMessageRenderer = {
|
||||
agentId?: string;
|
||||
render: React.ComponentType<{
|
||||
message: Message;
|
||||
position: "before" | "after";
|
||||
runId: string;
|
||||
messageIndex: number;
|
||||
agentId: string;
|
||||
stateSnapshot: any;
|
||||
}> | null;
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 7. Suggestions Configuration
|
||||
|
||||
**What:** Configure AI-generated or static prompt suggestions shown to users.
|
||||
|
||||
**Where configured:**
|
||||
|
||||
- React: `useConfigureSuggestions()` hook
|
||||
- Core: `suggestionsConfig` in config
|
||||
|
||||
**Default when not provided:** No suggestions.
|
||||
|
||||
```typescript
|
||||
// AI-generated suggestions
|
||||
type DynamicSuggestionsConfig = {
|
||||
instructions: string; // What to suggest
|
||||
minSuggestions?: number; // Default: 1
|
||||
maxSuggestions?: number; // Default: 3
|
||||
available?: SuggestionAvailability; // When to show
|
||||
providerAgentId?: string; // Which agent generates them
|
||||
consumerAgentId?: string; // Which agent receives them ("*" = all)
|
||||
};
|
||||
|
||||
// Static suggestions
|
||||
type StaticSuggestionsConfig = {
|
||||
suggestions: Array<{ title: string; message: string }>;
|
||||
available?: SuggestionAvailability;
|
||||
consumerAgentId?: string;
|
||||
};
|
||||
|
||||
type SuggestionAvailability =
|
||||
| "before-first-message" // Default for static
|
||||
| "after-first-message" // Default for dynamic
|
||||
| "always"
|
||||
| "disabled";
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Suggestion Types"
|
||||
DYN["Dynamic<br/><i>AI generates suggestions<br/>from instructions</i>"]
|
||||
STA["Static<br/><i>You provide fixed<br/>suggestion list</i>"]
|
||||
end
|
||||
|
||||
subgraph "Availability"
|
||||
BFM["before-first-message"]
|
||||
AFM["after-first-message"]
|
||||
ALW["always"]
|
||||
DIS["disabled"]
|
||||
end
|
||||
|
||||
DYN -.->|default| AFM
|
||||
STA -.->|default| BFM
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 8. Event Subscribers
|
||||
|
||||
**What:** Listen to lifecycle events — connection status, tool execution, agent changes, errors.
|
||||
|
||||
**Where configured:**
|
||||
|
||||
- Any: `copilotKit.subscribe(subscriber)`
|
||||
- Returns: `{ unsubscribe() }` for cleanup
|
||||
|
||||
**Default when not provided:** No listeners — events still fire internally.
|
||||
|
||||
```typescript
|
||||
type CopilotKitCoreSubscriber = {
|
||||
onRuntimeConnectionStatusChanged?: (event) => void;
|
||||
onToolExecutionStart?: (event) => void;
|
||||
onToolExecutionEnd?: (event) => void;
|
||||
onAgentsChanged?: (event) => void;
|
||||
onContextChanged?: (event) => void;
|
||||
onSuggestionsChanged?: (event) => void;
|
||||
onSuggestionsStartedLoading?: (event) => void;
|
||||
onSuggestionsFinishedLoading?: (event) => void;
|
||||
onPropertiesChanged?: (event) => void;
|
||||
onHeadersChanged?: (event) => void;
|
||||
onError?: (event) => void;
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Backend Extension Points
|
||||
|
||||
### 9. Before Request Middleware
|
||||
|
||||
**What:** Intercept HTTP requests before they reach the handler. Use for auth, logging, request transformation.
|
||||
|
||||
**Where configured:** `CopilotRuntime` constructor — `beforeRequestMiddleware`
|
||||
|
||||
**Default when not provided:** Requests pass through unchanged.
|
||||
|
||||
```typescript
|
||||
type BeforeRequestMiddleware = (params: {
|
||||
runtime: CopilotRuntime;
|
||||
request: Request;
|
||||
path: string;
|
||||
}) => MaybePromise<Request | void>;
|
||||
// Return modified Request, or void to pass through
|
||||
// Return a Response to short-circuit (e.g., 401)
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
REQ["Incoming Request"]
|
||||
BM["beforeRequestMiddleware"]
|
||||
HANDLER["Route Handler"]
|
||||
REJECT["401 / Error Response"]
|
||||
|
||||
REQ --> BM
|
||||
BM -->|pass through| HANDLER
|
||||
BM -->|reject| REJECT
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 10. After Request Middleware
|
||||
|
||||
**What:** Run code after the response is prepared. Use for logging, metrics, cleanup.
|
||||
|
||||
**Where configured:** `CopilotRuntime` constructor — `afterRequestMiddleware`
|
||||
|
||||
**Default when not provided:** No post-processing.
|
||||
|
||||
```typescript
|
||||
type AfterRequestMiddleware = (params: {
|
||||
runtime: CopilotRuntime;
|
||||
response: Response;
|
||||
path: string;
|
||||
}) => MaybePromise<void>;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 11. Agent Runner
|
||||
|
||||
**What:** Controls how agents are executed and how thread state is managed.
|
||||
|
||||
**Where configured:** `CopilotRuntime` constructor — `runner`
|
||||
|
||||
**Default when not provided:** `InMemoryAgentRunner` — in-process, ephemeral (threads lost on restart).
|
||||
|
||||
```typescript
|
||||
abstract class AgentRunner {
|
||||
abstract run(request: AgentRunnerRunRequest): Observable<BaseEvent>;
|
||||
abstract connect(request: AgentRunnerConnectRequest): Observable<BaseEvent>;
|
||||
abstract isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean>;
|
||||
abstract stop(request: AgentRunnerStopRequest): Promise<boolean | undefined>;
|
||||
}
|
||||
```
|
||||
|
||||
| Implementation | Storage | Persistence | Use case |
|
||||
| --------------------- | ----------- | ----------- | -------------------------------- |
|
||||
| `InMemoryAgentRunner` | RAM | No | Development, stateless apps |
|
||||
| `SQLiteAgentRunner` | Disk | Yes | Production, long-running threads |
|
||||
| Custom | Your choice | Your choice | Redis, PostgreSQL, etc. |
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
RT["CopilotRuntime"]
|
||||
RUNNER["runner (AgentRunner)"]
|
||||
|
||||
RT --> RUNNER
|
||||
|
||||
subgraph Implementations
|
||||
IM["InMemoryAgentRunner<br/><i>Default — in-process</i>"]
|
||||
SQ["SQLiteAgentRunner<br/><i>Persistent on disk</i>"]
|
||||
CU["YourCustomRunner<br/><i>Redis, Postgres, etc.</i>"]
|
||||
end
|
||||
|
||||
RUNNER -.-> IM
|
||||
RUNNER -.-> SQ
|
||||
RUNNER -.-> CU
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 12. Transcription Service
|
||||
|
||||
**What:** Convert audio files to text. Enables the `/transcribe` endpoint.
|
||||
|
||||
**Where configured:** `CopilotRuntime` constructor — `transcriptionService`
|
||||
|
||||
**Default when not provided:** `/transcribe` endpoint returns 404.
|
||||
|
||||
```typescript
|
||||
abstract class TranscriptionService {
|
||||
abstract transcribeFile(options: {
|
||||
audioFile: File;
|
||||
mimeType?: string;
|
||||
size?: number;
|
||||
}): Promise<string>;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Agent-Level Extension Points
|
||||
|
||||
### 13. AG-UI Middleware
|
||||
|
||||
**What:** Intercept and transform the agent execution pipeline. Cross-cutting concerns like logging, filtering, and backward compatibility.
|
||||
|
||||
**Where configured:** At the agent level (outside CopilotKit core).
|
||||
|
||||
**Default when not provided:** Direct agent execution.
|
||||
|
||||
```typescript
|
||||
abstract class Middleware {
|
||||
abstract run(
|
||||
input: RunAgentInput,
|
||||
next: AbstractAgent,
|
||||
): Observable<BaseEvent>;
|
||||
}
|
||||
|
||||
// Built-in implementations:
|
||||
// - FunctionMiddleware — wrap a function as middleware
|
||||
// - FilterToolCallsMiddleware — filter which tools are sent
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
INPUT["RunAgentInput"]
|
||||
MW1["Middleware 1<br/><i>e.g., logging</i>"]
|
||||
MW2["Middleware 2<br/><i>e.g., tool filtering</i>"]
|
||||
AGENT["Agent.run()"]
|
||||
|
||||
INPUT --> MW1 --> MW2 --> AGENT
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Complete Map: Where Each Extension Plugs In
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Provider / Config"
|
||||
P["CopilotKitProvider<br/>or provideCopilotKit()"]
|
||||
P --> FT_P["frontendTools"]
|
||||
P --> RTC_P["renderToolCalls"]
|
||||
P --> RAM_P["renderActivityMessages"]
|
||||
P --> RCM_P["renderCustomMessages"]
|
||||
P --> HIL_P["humanInTheLoop"]
|
||||
P --> HDR["headers"]
|
||||
P --> CRD["credentials"]
|
||||
P --> PRP["properties"]
|
||||
P --> DC["showDevConsole"]
|
||||
end
|
||||
|
||||
subgraph "Hooks / Service Methods"
|
||||
UFT["useFrontendTool()"]
|
||||
UAC["useAgentContext()"]
|
||||
URT["useRenderToolCall()"]
|
||||
UHL["useHumanInTheLoop()"]
|
||||
UCS["useConfigureSuggestions()"]
|
||||
URA["useRenderActivityMessage()"]
|
||||
URC["useRenderCustomMessages()"]
|
||||
end
|
||||
|
||||
subgraph "CopilotRuntime"
|
||||
RT["new CopilotRuntime()"]
|
||||
RT --> AGENTS["agents (required)"]
|
||||
RT --> RUNNER["runner"]
|
||||
RT --> BM["beforeRequestMiddleware"]
|
||||
RT --> AM["afterRequestMiddleware"]
|
||||
RT --> TS["transcriptionService"]
|
||||
end
|
||||
|
||||
subgraph "Core API"
|
||||
SUB["copilotKit.subscribe()"]
|
||||
AT["copilotKit.addTool()"]
|
||||
AC["copilotKit.addContext()"]
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary Table
|
||||
|
||||
| Extension Point | Location | Config Method | Default | Optional |
|
||||
| ---------------------------- | -------- | ----------------------------- | ------------------- | -------- |
|
||||
| **Frontend Tools** | Frontend | Hook / Provider / `addTool()` | None | Yes |
|
||||
| **Agent Context** | Frontend | Hook / `addContext()` | None | Yes |
|
||||
| **Tool Call Renderers** | Frontend | Hook / Provider | Generic rendering | Yes |
|
||||
| **Human-in-the-Loop** | Frontend | Hook / Provider | Immediate execution | Yes |
|
||||
| **Activity Renderers** | Frontend | Hook / Provider | MCP Apps included | Yes |
|
||||
| **Custom Message Renderers** | Frontend | Hook / Provider | None | Yes |
|
||||
| **Suggestions Config** | Frontend | Hook / Config | None | Yes |
|
||||
| **Event Subscribers** | Frontend | `subscribe()` | None | Yes |
|
||||
| **Before Middleware** | Backend | Runtime constructor | Pass-through | Yes |
|
||||
| **After Middleware** | Backend | Runtime constructor | None | Yes |
|
||||
| **Agent Runner** | Backend | Runtime constructor | InMemoryAgentRunner | Yes |
|
||||
| **Transcription Service** | Backend | Runtime constructor | None (404) | Yes |
|
||||
| **AG-UI Middleware** | Agent | Agent-level config | Direct execution | Yes |
|
||||
@@ -0,0 +1,456 @@
|
||||
# Angular Setup Guide
|
||||
|
||||
This guide shows how to set up CopilotKit in an Angular app — from minimal to fully configured.
|
||||
|
||||
---
|
||||
|
||||
## What Talks to What
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Your Angular App
|
||||
DI["<b>provideCopilotKit()</b><br/><i>DI token</i>"]
|
||||
Service["<b>CopilotKit Service</b><br/><i>Injectable</i>"]
|
||||
Store["<b>AgentStore</b><br/><i>Signal-based state</i>"]
|
||||
Comp["Your Components"]
|
||||
end
|
||||
|
||||
subgraph Under the Hood
|
||||
Core["CopilotKitCore<br/><i>Orchestrator</i>"]
|
||||
Proxy["ProxiedAgent<br/><i>HTTP client</i>"]
|
||||
end
|
||||
|
||||
subgraph Your Server
|
||||
Runtime["CopilotRuntime<br/><i>Express / Hono</i>"]
|
||||
end
|
||||
|
||||
DI -->|configures| Service
|
||||
Service -->|wraps| Core
|
||||
Comp -->|injects| Service
|
||||
Service -->|creates| Store
|
||||
Store -->|wraps| Proxy
|
||||
Proxy -->|HTTP POST + SSE| Runtime
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Minimal Setup
|
||||
|
||||
### 1. Install
|
||||
|
||||
```bash
|
||||
npm install @copilotkit/angular
|
||||
```
|
||||
|
||||
### 2. Configure the DI token
|
||||
|
||||
```typescript
|
||||
// app.config.ts
|
||||
import { ApplicationConfig } from "@angular/core";
|
||||
import { provideCopilotKit } from "@copilotkit/angular";
|
||||
|
||||
export const appConfig: ApplicationConfig = {
|
||||
providers: [
|
||||
provideCopilotKit({
|
||||
runtimeUrl: "/api/copilotkit",
|
||||
}),
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
### 3. Use the service in a component
|
||||
|
||||
```typescript
|
||||
// chat.component.ts
|
||||
import { Component, inject } from "@angular/core";
|
||||
import { CopilotKit } from "@copilotkit/angular";
|
||||
|
||||
@Component({
|
||||
selector: "app-chat",
|
||||
template: `
|
||||
<div>
|
||||
<div *ngFor="let msg of agentStore.messages()">
|
||||
<b>{{ msg.role }}:</b> {{ msg.content }}
|
||||
</div>
|
||||
<input #input (keydown.enter)="send(input.value); input.value = ''" />
|
||||
</div>
|
||||
`,
|
||||
})
|
||||
export class ChatComponent {
|
||||
private copilotKit = inject(CopilotKit);
|
||||
agentStore = this.copilotKit.getAgentStore(); // default agent
|
||||
|
||||
async send(message: string) {
|
||||
this.agentStore.addMessage({
|
||||
id: crypto.randomUUID(),
|
||||
role: "user",
|
||||
content: message,
|
||||
});
|
||||
await this.copilotKit.runAgent({ agent: this.agentStore.agent });
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
That's it — the DI token creates a `CopilotKit` service backed by `CopilotKitCore`, and `AgentStore` gives you signal-based reactive state.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Config as app.config.ts
|
||||
participant Service as CopilotKit Service
|
||||
participant Core as CopilotKitCore
|
||||
participant Runtime as Your Server
|
||||
|
||||
Config->>Service: provideCopilotKit({ runtimeUrl })
|
||||
Service->>Core: new CopilotKitCore(config)
|
||||
Core->>Runtime: GET /info
|
||||
Runtime-->>Core: Available agents
|
||||
Note over Service: Ready — inject anywhere
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Angular Signals for Reactive State
|
||||
|
||||
`AgentStore` uses Angular signals, so your templates react to changes automatically:
|
||||
|
||||
```typescript
|
||||
@Component({
|
||||
template: `
|
||||
@if (agentStore.isRunning()) {
|
||||
<p>Agent is thinking...</p>
|
||||
}
|
||||
|
||||
@for (msg of agentStore.messages(); track msg.id) {
|
||||
<div [class]="msg.role">{{ msg.content }}</div>
|
||||
}
|
||||
|
||||
<pre>{{ agentStore.state() | json }}</pre>
|
||||
`,
|
||||
})
|
||||
export class ChatComponent {
|
||||
private copilotKit = inject(CopilotKit);
|
||||
agentStore = this.copilotKit.getAgentStore("my-agent");
|
||||
}
|
||||
```
|
||||
|
||||
### AgentStore Signals
|
||||
|
||||
| Signal | Type | What it tracks |
|
||||
| ------------- | ----------- | -------------------------------------- |
|
||||
| `messages()` | `Message[]` | All messages in the conversation |
|
||||
| `isRunning()` | `boolean` | Whether the agent is currently running |
|
||||
| `state()` | `any` | Agent state (arbitrary JSON) |
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph AgentStore
|
||||
Agent["AbstractAgent<br/><i>Subscribed to events</i>"]
|
||||
MS["messages()<br/><i>Signal<Message[]></i>"]
|
||||
IR["isRunning()<br/><i>Signal<boolean></i>"]
|
||||
ST["state()<br/><i>Signal<any></i>"]
|
||||
end
|
||||
|
||||
Agent -->|onMessagesChanged| MS
|
||||
Agent -->|onRunStarted/Finished| IR
|
||||
Agent -->|onStateChanged| ST
|
||||
|
||||
subgraph Template
|
||||
T["Your template auto-updates"]
|
||||
end
|
||||
|
||||
MS --> T
|
||||
IR --> T
|
||||
ST --> T
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Registering Tools
|
||||
|
||||
```typescript
|
||||
// In your component or service
|
||||
import { CopilotKit } from "@copilotkit/angular";
|
||||
import { z } from "zod";
|
||||
|
||||
@Component({
|
||||
/* ... */
|
||||
})
|
||||
export class ProductComponent implements OnInit, OnDestroy {
|
||||
private copilotKit = inject(CopilotKit);
|
||||
|
||||
ngOnInit() {
|
||||
// Register a tool the agent can call
|
||||
this.copilotKit.addTool({
|
||||
name: "addToCart",
|
||||
description: "Add a product to cart",
|
||||
parameters: z.object({
|
||||
productId: z.string(),
|
||||
quantity: z.number().default(1),
|
||||
}),
|
||||
handler: async ({ productId, quantity }) => {
|
||||
this.cartService.add(productId, quantity);
|
||||
return `Added ${quantity} item(s)`;
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
ngOnDestroy() {
|
||||
// Clean up when component is destroyed
|
||||
this.copilotKit.removeTool("addToCart");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Providing Context
|
||||
|
||||
```typescript
|
||||
@Component({
|
||||
/* ... */
|
||||
})
|
||||
export class DashboardComponent implements OnInit, OnDestroy {
|
||||
private copilotKit = inject(CopilotKit);
|
||||
private contextId?: string;
|
||||
|
||||
ngOnInit() {
|
||||
this.contextId = this.copilotKit.addContext({
|
||||
description: "Current dashboard metrics",
|
||||
value: JSON.stringify({
|
||||
revenue: this.metricsService.revenue(),
|
||||
activeUsers: this.metricsService.activeUsers(),
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
ngOnDestroy() {
|
||||
if (this.contextId) {
|
||||
this.copilotKit.removeContext(this.contextId);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Tool Call Rendering
|
||||
|
||||
Angular uses the `AngularToolCall` type for rendering tool calls:
|
||||
|
||||
```typescript
|
||||
import { AngularToolCall } from "@copilotkit/angular";
|
||||
|
||||
// Configure in provideCopilotKit
|
||||
provideCopilotKit({
|
||||
runtimeUrl: "/api/copilotkit",
|
||||
renderToolCalls: [
|
||||
{
|
||||
name: "searchProducts",
|
||||
// The Angular component receives the AngularToolCall
|
||||
},
|
||||
],
|
||||
});
|
||||
```
|
||||
|
||||
### AngularToolCall Status Flow
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
IP["in-progress<br/><i>Args still streaming</i>"]
|
||||
EX["executing<br/><i>Handler is running</i>"]
|
||||
CO["complete<br/><i>Result ready</i>"]
|
||||
IP --> EX --> CO
|
||||
```
|
||||
|
||||
| Field | Type | Description |
|
||||
| -------- | -------------------------------------------- | ---------------------------------------- |
|
||||
| `status` | `"in-progress" \| "executing" \| "complete"` | Current lifecycle stage |
|
||||
| `name` | `string` | Tool name |
|
||||
| `args` | `Partial<T>` or `T` | Tool arguments (partial while streaming) |
|
||||
| `result` | `string \| undefined` | Result (only when complete) |
|
||||
|
||||
---
|
||||
|
||||
## All Configuration Options
|
||||
|
||||
```typescript
|
||||
// app.config.ts
|
||||
import { provideCopilotKit } from "@copilotkit/angular";
|
||||
|
||||
provideCopilotKit({
|
||||
// Required
|
||||
runtimeUrl: "/api/copilotkit",
|
||||
|
||||
// Authentication
|
||||
headers: { Authorization: "Bearer token" },
|
||||
|
||||
// Custom properties forwarded to agents
|
||||
properties: { userId: "123", plan: "pro" },
|
||||
|
||||
// Local agents for development
|
||||
agents: { test: myTestAgent },
|
||||
|
||||
// Tools (can also add via service)
|
||||
tools: [
|
||||
{
|
||||
name: "myTool",
|
||||
parameters: z.object({ input: z.string() }),
|
||||
handler: async ({ input }) => `Processed: ${input}`,
|
||||
},
|
||||
],
|
||||
|
||||
// Tool call rendering
|
||||
renderToolCalls: [
|
||||
/* ... */
|
||||
],
|
||||
|
||||
// Frontend tools
|
||||
frontendTools: [
|
||||
/* ... */
|
||||
],
|
||||
|
||||
// Human-in-the-loop
|
||||
humanInTheLoop: [
|
||||
/* ... */
|
||||
],
|
||||
});
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "provideCopilotKit() Config"
|
||||
direction TB
|
||||
|
||||
subgraph Required
|
||||
URL["runtimeUrl"]
|
||||
end
|
||||
|
||||
subgraph "Optional: Auth"
|
||||
H["headers"]
|
||||
end
|
||||
|
||||
subgraph "Optional: Tools & Rendering"
|
||||
T["tools"]
|
||||
FT["frontendTools"]
|
||||
RTC["renderToolCalls"]
|
||||
HIL["humanInTheLoop"]
|
||||
end
|
||||
|
||||
subgraph "Optional: Other"
|
||||
P["properties"]
|
||||
AG["agents"]
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Full Example: Dashboard App
|
||||
|
||||
```typescript
|
||||
// app.config.ts
|
||||
import { ApplicationConfig } from "@angular/core";
|
||||
import { provideCopilotKit } from "@copilotkit/angular";
|
||||
|
||||
export const appConfig: ApplicationConfig = {
|
||||
providers: [
|
||||
provideCopilotKit({
|
||||
runtimeUrl: "/api/copilotkit",
|
||||
headers: { Authorization: `Bearer ${getToken()}` },
|
||||
}),
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
```typescript
|
||||
// dashboard.component.ts
|
||||
import { Component, inject, OnInit, OnDestroy } from "@angular/core";
|
||||
import { CopilotKit } from "@copilotkit/angular";
|
||||
import { z } from "zod";
|
||||
|
||||
@Component({
|
||||
selector: "app-dashboard",
|
||||
template: `
|
||||
<div class="dashboard">
|
||||
<app-metrics />
|
||||
|
||||
<div class="chat">
|
||||
@if (agentStore.isRunning()) {
|
||||
<div class="typing">Agent is thinking...</div>
|
||||
}
|
||||
|
||||
@for (msg of agentStore.messages(); track msg.id) {
|
||||
<div [class]="'message ' + msg.role">
|
||||
{{ msg.content }}
|
||||
</div>
|
||||
}
|
||||
|
||||
<input
|
||||
#input
|
||||
placeholder="Ask about your metrics..."
|
||||
(keydown.enter)="send(input.value); input.value = ''"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
`,
|
||||
})
|
||||
export class DashboardComponent implements OnInit, OnDestroy {
|
||||
private copilotKit = inject(CopilotKit);
|
||||
private metricsService = inject(MetricsService);
|
||||
|
||||
agentStore = this.copilotKit.getAgentStore();
|
||||
private contextId?: string;
|
||||
|
||||
ngOnInit() {
|
||||
// Provide context
|
||||
this.contextId = this.copilotKit.addContext({
|
||||
description: "Dashboard metrics",
|
||||
value: JSON.stringify({
|
||||
revenue: this.metricsService.revenue(),
|
||||
users: this.metricsService.activeUsers(),
|
||||
}),
|
||||
});
|
||||
|
||||
// Register tool
|
||||
this.copilotKit.addTool({
|
||||
name: "filterMetrics",
|
||||
description: "Filter dashboard metrics by date range",
|
||||
parameters: z.object({
|
||||
startDate: z.string(),
|
||||
endDate: z.string(),
|
||||
}),
|
||||
handler: async ({ startDate, endDate }) => {
|
||||
this.metricsService.setDateRange(startDate, endDate);
|
||||
return `Filtered to ${startDate} - ${endDate}`;
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
ngOnDestroy() {
|
||||
if (this.contextId) this.copilotKit.removeContext(this.contextId);
|
||||
this.copilotKit.removeTool("filterMetrics");
|
||||
}
|
||||
|
||||
async send(message: string) {
|
||||
this.agentStore.addMessage({
|
||||
id: crypto.randomUUID(),
|
||||
role: "user",
|
||||
content: message,
|
||||
});
|
||||
await this.copilotKit.runAgent({ agent: this.agentStore.agent });
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Differences from React
|
||||
|
||||
| Aspect | React | Angular |
|
||||
| ----------------- | ----------------------------------------- | --------------------------------------------------- |
|
||||
| Configuration | `<CopilotKitProvider>` JSX | `provideCopilotKit()` DI token |
|
||||
| Service access | `useCopilotKit()` hook | `inject(CopilotKit)` |
|
||||
| Agent state | `useAgent()` hook returns reactive values | `AgentStore` with Angular signals |
|
||||
| Tool registration | `useFrontendTool()` hook (auto-cleanup) | `addTool()` / `removeTool()` (manual cleanup) |
|
||||
| Context | `useAgentContext()` hook (auto-cleanup) | `addContext()` / `removeContext()` (manual cleanup) |
|
||||
| Reactivity | React re-renders on state change | Angular signals trigger change detection |
|
||||
@@ -0,0 +1,238 @@
|
||||
# Intelligence Setup Guide
|
||||
|
||||
This guide shows how to set up **CopilotKit Intelligence**: durable thread storage plus a websocket transport for realtime events.
|
||||
|
||||
Intelligence is designed to feel like a small runtime configuration change, not a separate product integration. You provide an Intelligence platform client to the runtime, and the rest of the stack switches from plain SSE mode into Intelligence mode automatically.
|
||||
|
||||
---
|
||||
|
||||
## What Changes in Intelligence Mode
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph Frontend
|
||||
App["React / Angular / Vanilla"]
|
||||
Core["CopilotKitCore"]
|
||||
Proxy["ProxiedCopilotRuntimeAgent"]
|
||||
IA["IntelligenceAgent<br/><i>chosen after /info</i>"]
|
||||
end
|
||||
|
||||
subgraph Your Server
|
||||
RT["CopilotRuntime"]
|
||||
CPK-I["CopilotKitIntelligence"]
|
||||
Runner["IntelligenceAgentRunner"]
|
||||
end
|
||||
|
||||
subgraph Intelligence
|
||||
API["Thread API<br/><i>durable storage</i>"]
|
||||
WS["Realtime WebSocket"]
|
||||
end
|
||||
|
||||
App --> Core
|
||||
Core --> Proxy
|
||||
Proxy -->|info handshake| RT
|
||||
RT --> CPK-I
|
||||
CPK-I --> API
|
||||
RT --> Runner
|
||||
Runner --> WS
|
||||
Proxy --> IA
|
||||
IA -->|REST bootstrap| RT
|
||||
IA -->|WebSocket events| WS
|
||||
```
|
||||
|
||||
### SSE Mode vs Intelligence Mode
|
||||
|
||||
| Mode | Thread storage | Realtime transport | `/info` reports |
|
||||
| ------------ | ------------------------------------------- | ------------------ | --------------------------------------------- |
|
||||
| SSE | Ephemeral unless your runner persists state | SSE | `mode: "sse"` |
|
||||
| Intelligence | Durable thread APIs | WebSocket | `mode: "intelligence"` + `intelligence.wsUrl` |
|
||||
|
||||
The important design rule is:
|
||||
|
||||
- The **runtime** decides the mode.
|
||||
- The **client** waits for `/info` before choosing the concrete remote agent implementation.
|
||||
- The **developer** only opts in by providing `intelligence`.
|
||||
|
||||
---
|
||||
|
||||
## Minimal Runtime Setup
|
||||
|
||||
### 1. Install runtime packages
|
||||
|
||||
```bash
|
||||
npm install @copilotkit/runtime
|
||||
```
|
||||
|
||||
### 2. Create the Intelligence platform client
|
||||
|
||||
```typescript
|
||||
import { CopilotKitIntelligence } from "@copilotkit/runtime";
|
||||
|
||||
const intelligence = new CopilotKitIntelligence({
|
||||
apiKey: process.env.COPILOTKIT_INTELLIGENCE_API_KEY!,
|
||||
organizationId: process.env.COPILOTKIT_INTELLIGENCE_ORGANIZATION_ID!,
|
||||
apiUrl: "https://your-intelligence-host/api",
|
||||
wsUrl: "wss://your-intelligence-host/socket",
|
||||
});
|
||||
```
|
||||
|
||||
### 3. Pass it to `CopilotRuntime`
|
||||
|
||||
```typescript
|
||||
import express from "express";
|
||||
import { CopilotRuntime } from "@copilotkit/runtime";
|
||||
import { createCopilotEndpointExpress } from "@copilotkit/runtime/express";
|
||||
|
||||
const app = express();
|
||||
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: {
|
||||
default: myAgent,
|
||||
},
|
||||
intelligence,
|
||||
});
|
||||
|
||||
app.use(
|
||||
"/api/copilotkit",
|
||||
createCopilotEndpointExpress({
|
||||
runtime,
|
||||
basePath: "/",
|
||||
}),
|
||||
);
|
||||
```
|
||||
|
||||
That is the mode switch. You do **not** separately configure Intelligence handlers in the endpoint layer. The runtime selects them.
|
||||
|
||||
---
|
||||
|
||||
## What `CopilotRuntime` Does For You
|
||||
|
||||
When `intelligence` is present, `CopilotRuntime`:
|
||||
|
||||
- switches its mode from `"sse"` to `"intelligence"`
|
||||
- uses the Intelligence handler path for `run`, `connect`, and `threads`
|
||||
- auto-configures the Intelligence runner from `intelligence.wsUrl`
|
||||
- reports Intelligence metadata from `/info`
|
||||
|
||||
Example `/info` response:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "1.x.x",
|
||||
"mode": "intelligence",
|
||||
"agents": {
|
||||
"default": {
|
||||
"name": "default",
|
||||
"description": "My agent",
|
||||
"className": "BuiltInAgent"
|
||||
}
|
||||
},
|
||||
"audioFileTranscriptionEnabled": false,
|
||||
"a2uiEnabled": false,
|
||||
"intelligence": {
|
||||
"wsUrl": "wss://your-intelligence-host/socket"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The frontend uses that response to decide whether to keep using the HTTP/SSE path or switch to the Intelligence websocket path.
|
||||
|
||||
---
|
||||
|
||||
## Frontend Behavior
|
||||
|
||||
You do not configure a special provider flag for Intelligence.
|
||||
|
||||
This stays the same:
|
||||
|
||||
```tsx
|
||||
import { CopilotKitProvider, CopilotChat } from "@copilotkit/react-core/v2";
|
||||
|
||||
export function App() {
|
||||
return (
|
||||
<CopilotKitProvider runtimeUrl="/api/copilotkit">
|
||||
<CopilotChat />
|
||||
</CopilotKitProvider>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
What changes under the hood:
|
||||
|
||||
1. The provider connects to the runtime as usual.
|
||||
2. `CopilotKitCore` fetches `/info`.
|
||||
3. `ProxiedCopilotRuntimeAgent` waits until the runtime reports its mode.
|
||||
4. If the mode is:
|
||||
- `"sse"`: normal HTTP/SSE behavior continues.
|
||||
- `"intelligence"`: the proxy uses `IntelligenceAgent` and the runtime-provided websocket URL.
|
||||
|
||||
This is why the runtime owns the mode decision instead of the frontend guessing from config.
|
||||
|
||||
---
|
||||
|
||||
## Durable Threads
|
||||
|
||||
Intelligence mode adds thread APIs on the runtime:
|
||||
|
||||
| Route | Method | Purpose |
|
||||
| ---------------------------- | ------ | ------------------------------------ |
|
||||
| `/threads` | GET | List durable threads |
|
||||
| `/threads/subscribe` | POST | Get credentials for realtime updates |
|
||||
| `/threads/:threadId` | PATCH | Update thread metadata |
|
||||
| `/threads/:threadId/archive` | POST | Archive a thread |
|
||||
| `/threads/:threadId` | DELETE | Delete a thread |
|
||||
|
||||
These routes are **Intelligence-only**.
|
||||
|
||||
In SSE mode they should reject with an explicit error, because SSE runtimes do not have the durable thread backend required to satisfy them.
|
||||
|
||||
---
|
||||
|
||||
## How Agent Runs Work in Intelligence Mode
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Client as Frontend
|
||||
participant Runtime as CopilotRuntime
|
||||
participant CPK-I as CopilotKitIntelligence
|
||||
participant WS as Intelligence WebSocket
|
||||
|
||||
Client->>Runtime: GET /info
|
||||
Runtime-->>Client: { mode: "intelligence", wsUrl: ... }
|
||||
|
||||
Client->>Runtime: POST /agent/default/run
|
||||
Runtime->>CPK-I: ensure thread exists + acquire lock
|
||||
CPK-I-->>Runtime: join token / join code
|
||||
Runtime-->>Client: bootstrap response
|
||||
|
||||
Client->>WS: join thread channel
|
||||
WS-->>Client: AG-UI events in realtime
|
||||
```
|
||||
|
||||
The runtime is still the contract boundary the frontend talks to. Intelligence is not exposed as a separate frontend integration surface.
|
||||
|
||||
---
|
||||
|
||||
## Local Agents vs Runtime-Discovered Agents
|
||||
|
||||
Local or self-managed agents still matter in Intelligence mode.
|
||||
|
||||
The intended precedence is:
|
||||
|
||||
1. local/self-managed agents
|
||||
2. runtime-discovered remote agents
|
||||
|
||||
That lets application code override a runtime-reported agent with a local implementation for development, testing, or custom routing behavior.
|
||||
|
||||
---
|
||||
|
||||
## Recommended Mental Model
|
||||
|
||||
Think of Intelligence as a **runtime capability**, not a second transport API developers need to learn.
|
||||
|
||||
- `CopilotKitIntelligence` configures the runtime's Intelligence backend.
|
||||
- `CopilotRuntime` exposes that capability through the same frontend-facing contract.
|
||||
- `/info` tells the client which concrete remote-agent implementation to use.
|
||||
- Frontend app code stays mostly unchanged.
|
||||
|
||||
If the setup feels bigger than “add the CPK-I to the runtime,” the abstraction is probably leaking.
|
||||
@@ -0,0 +1,448 @@
|
||||
# React Setup Guide
|
||||
|
||||
This guide shows how to set up CopilotKit in a React app — from minimal to fully configured.
|
||||
|
||||
---
|
||||
|
||||
## What Talks to What
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Your React App
|
||||
Provider["<b>CopilotKitProvider</b><br/><i>Wraps your app</i>"]
|
||||
Chat["<b>CopilotChat</b><br/><i>Chat UI component</i>"]
|
||||
Hook1["useFrontendTool()"]
|
||||
Hook2["useAgentContext()"]
|
||||
Hook3["useAgent()"]
|
||||
end
|
||||
|
||||
subgraph Under the Hood
|
||||
Core["CopilotKitCore<br/><i>Orchestrator</i>"]
|
||||
Proxy["ProxiedAgent<br/><i>HTTP client</i>"]
|
||||
end
|
||||
|
||||
subgraph Your Server
|
||||
Runtime["CopilotRuntime<br/><i>Express / Hono</i>"]
|
||||
end
|
||||
|
||||
Provider -->|creates| Core
|
||||
Chat -->|uses| Hook3
|
||||
Hook1 -->|registers tool in| Core
|
||||
Hook2 -->|registers context in| Core
|
||||
Hook3 -->|gets agent from| Core
|
||||
Core -->|creates| Proxy
|
||||
Proxy -->|HTTP POST + SSE| Runtime
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Minimal Setup (V1 — recommended starting point)
|
||||
|
||||
### 1. Install
|
||||
|
||||
```bash
|
||||
npm install @copilotkit/react-core @copilotkit/react-ui
|
||||
```
|
||||
|
||||
### 2. Wrap your app with the provider
|
||||
|
||||
```tsx
|
||||
// app.tsx
|
||||
import { CopilotKit } from "@copilotkit/react-core";
|
||||
import "@copilotkit/react-ui/styles.css";
|
||||
|
||||
export default function App() {
|
||||
return (
|
||||
<CopilotKit runtimeUrl="/api/copilotkit">
|
||||
<YourApp />
|
||||
</CopilotKit>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Add a chat component
|
||||
|
||||
```tsx
|
||||
// components/chat.tsx
|
||||
import { CopilotPopup } from "@copilotkit/react-ui";
|
||||
|
||||
export function ChatWidget() {
|
||||
return (
|
||||
<CopilotPopup
|
||||
labels={{ title: "AI Assistant", initial: "How can I help?" }}
|
||||
/>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
That's it — you now have a working AI chat. The provider connects to your runtime, fetches available agents, and the popup gives users a chat interface.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant App as React App
|
||||
participant Provider as CopilotKit Provider
|
||||
participant Runtime as Your Server
|
||||
|
||||
App->>Provider: Mounts with runtimeUrl
|
||||
Provider->>Runtime: GET /info
|
||||
Runtime-->>Provider: Available agents
|
||||
Note over Provider: Ready for chat
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## V2 Setup (direct)
|
||||
|
||||
If you're building new features and want the V2 API directly:
|
||||
|
||||
```bash
|
||||
npm install @copilotkit/react
|
||||
```
|
||||
|
||||
```tsx
|
||||
import { CopilotKitProvider, CopilotChat } from "@copilotkit/react-core";
|
||||
|
||||
export default function App() {
|
||||
return (
|
||||
<CopilotKitProvider runtimeUrl="/api/copilotkit">
|
||||
<CopilotChat />
|
||||
</CopilotKitProvider>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
> V1's `<CopilotKit>` wraps V2's `<CopilotKitProvider>` under the hood, so both work the same way.
|
||||
|
||||
---
|
||||
|
||||
## Adding Tools
|
||||
|
||||
Tools are functions the AI agent can call. They run in the browser.
|
||||
|
||||
```tsx
|
||||
import { useFrontendTool } from "@copilotkit/react-core";
|
||||
// or: import { useCopilotAction } from "@copilotkit/react-core"; (V1 equivalent)
|
||||
import { z } from "zod";
|
||||
|
||||
function ProductPage({ products }) {
|
||||
// The agent can now call "addToCart" during a conversation
|
||||
useFrontendTool({
|
||||
name: "addToCart",
|
||||
description: "Add a product to the user's shopping cart",
|
||||
parameters: z.object({
|
||||
productId: z.string().describe("The product ID to add"),
|
||||
quantity: z.number().default(1).describe("How many to add"),
|
||||
}),
|
||||
handler: async ({ productId, quantity }) => {
|
||||
await cartApi.add(productId, quantity);
|
||||
return `Added ${quantity} item(s) to cart`;
|
||||
},
|
||||
});
|
||||
|
||||
return <div>{/* your product UI */}</div>;
|
||||
}
|
||||
```
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Agent as AI Agent
|
||||
participant Runtime as CopilotRuntime
|
||||
participant Core as CopilotKitCore
|
||||
participant Tool as addToCart handler
|
||||
|
||||
Agent->>Runtime: TOOL_CALL_START { name: "addToCart" }
|
||||
Agent->>Runtime: TOOL_CALL_ARGS { productId: "abc", quantity: 2 }
|
||||
Runtime->>Core: SSE events
|
||||
Core->>Tool: Execute handler({ productId: "abc", quantity: 2 })
|
||||
Tool-->>Core: "Added 2 item(s) to cart"
|
||||
Core->>Runtime: TOOL_CALL_RESULT
|
||||
Runtime->>Agent: Agent continues with result
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Providing Context
|
||||
|
||||
Context tells the agent about what the user currently sees.
|
||||
|
||||
```tsx
|
||||
import { useAgentContext } from "@copilotkit/react-core";
|
||||
// or: import { useCopilotReadable } from "@copilotkit/react-core"; (V1 equivalent)
|
||||
|
||||
function Dashboard({ user, metrics }) {
|
||||
// The agent now knows about the current user and their metrics
|
||||
useAgentContext("Current user and dashboard metrics", {
|
||||
user: { name: user.name, role: user.role },
|
||||
metrics: { revenue: metrics.revenue, activeUsers: metrics.activeUsers },
|
||||
});
|
||||
|
||||
return <div>{/* your dashboard UI */}</div>;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Custom Tool Rendering
|
||||
|
||||
Show custom UI while a tool is being called:
|
||||
|
||||
```tsx
|
||||
import { useRenderToolCall } from "@copilotkit/react-core";
|
||||
import { z } from "zod";
|
||||
|
||||
function App() {
|
||||
useRenderToolCall({
|
||||
name: "searchProducts",
|
||||
args: z.object({ query: z.string() }),
|
||||
render: ({ args, status, result }) => {
|
||||
if (status === "in-progress") {
|
||||
return <div>Searching for "{args.query}"...</div>;
|
||||
}
|
||||
if (status === "executing") {
|
||||
return <Spinner>Running search...</Spinner>;
|
||||
}
|
||||
// status === "complete"
|
||||
return <div>Found results: {result}</div>;
|
||||
},
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Tool Call Lifecycle
|
||||
IP["in-progress<br/><i>Args streaming in</i>"]
|
||||
EX["executing<br/><i>Handler running</i>"]
|
||||
CO["complete<br/><i>Result available</i>"]
|
||||
IP --> EX --> CO
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Human-in-the-Loop
|
||||
|
||||
Require user approval before a tool executes:
|
||||
|
||||
```tsx
|
||||
import { useHumanInTheLoop } from "@copilotkit/react-core";
|
||||
import { z } from "zod";
|
||||
|
||||
function App() {
|
||||
useHumanInTheLoop({
|
||||
name: "deleteAccount",
|
||||
description: "Permanently delete a user account",
|
||||
parameters: z.object({ userId: z.string() }),
|
||||
render: ({ args, status, respond }) => {
|
||||
if (status === "executing") {
|
||||
return (
|
||||
<div>
|
||||
<p>Delete account {args.userId}?</p>
|
||||
<button onClick={() => respond("approved")}>Approve</button>
|
||||
<button onClick={() => respond("denied")}>Deny</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
if (status === "complete") {
|
||||
return <div>Action completed</div>;
|
||||
}
|
||||
return <div>Preparing...</div>;
|
||||
},
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Suggestions
|
||||
|
||||
Auto-generate prompt suggestions for users:
|
||||
|
||||
```tsx
|
||||
import { useConfigureSuggestions } from "@copilotkit/react-core";
|
||||
|
||||
function App() {
|
||||
useConfigureSuggestions({
|
||||
instructions: "Suggest questions about the user's dashboard data",
|
||||
minSuggestions: 2,
|
||||
maxSuggestions: 4,
|
||||
available: "always", // "before-first-message" | "after-first-message" | "always" | "disabled"
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## All Provider Props (optional)
|
||||
|
||||
```tsx
|
||||
<CopilotKitProvider
|
||||
// Required
|
||||
runtimeUrl="/api/copilotkit"
|
||||
// Authentication
|
||||
headers={{ Authorization: "Bearer token" }}
|
||||
credentials="include" // Forward cookies
|
||||
publicApiKey="ck_..." // CopilotKit Cloud key
|
||||
// Custom properties forwarded to agents
|
||||
properties={{ userId: "123", plan: "pro" }}
|
||||
// Tools & rendering (can also use hooks instead)
|
||||
frontendTools={
|
||||
[
|
||||
/* ... */
|
||||
]
|
||||
}
|
||||
renderToolCalls={
|
||||
[
|
||||
/* ... */
|
||||
]
|
||||
}
|
||||
renderActivityMessages={
|
||||
[
|
||||
/* ... */
|
||||
]
|
||||
}
|
||||
renderCustomMessages={
|
||||
[
|
||||
/* ... */
|
||||
]
|
||||
}
|
||||
humanInTheLoop={
|
||||
[
|
||||
/* ... */
|
||||
]
|
||||
}
|
||||
// Dev tools
|
||||
showDevConsole="auto" // true | false | "auto"
|
||||
// Advanced: local agents for development
|
||||
agents__unsafe_dev_only={{ test: myTestAgent }}
|
||||
/>
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "CopilotKitProvider Props"
|
||||
direction TB
|
||||
|
||||
subgraph Required
|
||||
URL["runtimeUrl"]
|
||||
end
|
||||
|
||||
subgraph "Optional: Auth"
|
||||
H["headers"]
|
||||
C["credentials"]
|
||||
K["publicApiKey"]
|
||||
end
|
||||
|
||||
subgraph "Optional: Tools & Rendering"
|
||||
FT["frontendTools"]
|
||||
RTC["renderToolCalls"]
|
||||
RAM["renderActivityMessages"]
|
||||
RCM["renderCustomMessages"]
|
||||
HIL["humanInTheLoop"]
|
||||
end
|
||||
|
||||
subgraph "Optional: Other"
|
||||
P["properties"]
|
||||
DC["showDevConsole"]
|
||||
AG["agents__unsafe_dev_only"]
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Chat Component Variants
|
||||
|
||||
```tsx
|
||||
import {
|
||||
CopilotChat, // Inline chat, fills its container
|
||||
CopilotPopup, // Floating popup button + chat
|
||||
CopilotSidebar, // Side panel
|
||||
CopilotPanel, // Inline panel
|
||||
} from "@copilotkit/react-ui";
|
||||
|
||||
// All accept the same core props:
|
||||
<CopilotChat
|
||||
agentId="research" // Which agent to talk to (default: "default")
|
||||
labels={{
|
||||
title: "Research Assistant",
|
||||
initial: "What would you like to research?",
|
||||
placeholder: "Ask me anything...",
|
||||
}}
|
||||
/>;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Full Example: E-Commerce App
|
||||
|
||||
```tsx
|
||||
import { CopilotKit } from "@copilotkit/react-core";
|
||||
import { CopilotSidebar } from "@copilotkit/react-ui";
|
||||
import "@copilotkit/react-ui/styles.css";
|
||||
import { z } from "zod";
|
||||
|
||||
export default function App() {
|
||||
return (
|
||||
<CopilotKit
|
||||
runtimeUrl="/api/copilotkit"
|
||||
headers={{ Authorization: `Bearer ${getToken()}` }}
|
||||
>
|
||||
<CopilotSidebar labels={{ title: "Shopping Assistant" }}>
|
||||
<ProductCatalog />
|
||||
</CopilotSidebar>
|
||||
</CopilotKit>
|
||||
);
|
||||
}
|
||||
|
||||
function ProductCatalog() {
|
||||
const [products] = useProducts();
|
||||
const [cart, setCart] = useCart();
|
||||
|
||||
// Context: tell the agent what the user sees
|
||||
useAgentContext("Product catalog the user is browsing", {
|
||||
products: products.map((p) => ({ id: p.id, name: p.name, price: p.price })),
|
||||
cartTotal: cart.total,
|
||||
cartItems: cart.items.length,
|
||||
});
|
||||
|
||||
// Tool: agent can add items to cart
|
||||
useFrontendTool({
|
||||
name: "addToCart",
|
||||
description: "Add a product to the shopping cart",
|
||||
parameters: z.object({
|
||||
productId: z.string(),
|
||||
quantity: z.number().default(1),
|
||||
}),
|
||||
handler: async ({ productId, quantity }) => {
|
||||
setCart((prev) => addItem(prev, productId, quantity));
|
||||
return "Added to cart";
|
||||
},
|
||||
});
|
||||
|
||||
// Tool: agent can search products
|
||||
useFrontendTool({
|
||||
name: "searchProducts",
|
||||
description: "Search for products by name or category",
|
||||
parameters: z.object({ query: z.string() }),
|
||||
handler: async ({ query }) => {
|
||||
const results = products.filter((p) =>
|
||||
p.name.toLowerCase().includes(query.toLowerCase()),
|
||||
);
|
||||
return JSON.stringify(
|
||||
results.map((p) => ({ id: p.id, name: p.name, price: p.price })),
|
||||
);
|
||||
},
|
||||
});
|
||||
|
||||
// Suggestions
|
||||
useConfigureSuggestions({
|
||||
instructions:
|
||||
"Suggest shopping-related questions based on the product catalog",
|
||||
maxSuggestions: 3,
|
||||
available: "always",
|
||||
});
|
||||
|
||||
return <div>{/* product grid UI */}</div>;
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,493 @@
|
||||
# Runtime / Backend Setup Guide
|
||||
|
||||
This guide shows how to set up the CopilotKit backend — from minimal to fully configured with all optional extension points.
|
||||
|
||||
---
|
||||
|
||||
## What Talks to What
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph Frontends
|
||||
React["React App"]
|
||||
Angular["Angular App"]
|
||||
Vanilla["Vanilla JS"]
|
||||
end
|
||||
|
||||
subgraph Your Server
|
||||
EP["Express / Hono<br/><i>Endpoint handler</i>"]
|
||||
BM["beforeRequestMiddleware<br/><i>(optional)</i>"]
|
||||
RT["<b>CopilotRuntime</b>"]
|
||||
AM["afterRequestMiddleware<br/><i>(optional)</i>"]
|
||||
Runner["<b>AgentRunner</b><br/><i>InMemory (default)<br/>or SQLite</i>"]
|
||||
TS["TranscriptionService<br/><i>(optional)</i>"]
|
||||
end
|
||||
|
||||
subgraph Agents
|
||||
A1["Agent 1<br/><i>LangGraph</i>"]
|
||||
A2["Agent 2<br/><i>CrewAI</i>"]
|
||||
A3["Agent 3<br/><i>Custom</i>"]
|
||||
end
|
||||
|
||||
React -->|HTTP| EP
|
||||
Angular -->|HTTP| EP
|
||||
Vanilla -->|HTTP| EP
|
||||
EP --> BM
|
||||
BM --> RT
|
||||
RT --> AM
|
||||
RT --> Runner
|
||||
RT --> TS
|
||||
Runner -->|AG-UI events| A1
|
||||
Runner -->|AG-UI events| A2
|
||||
Runner -->|AG-UI events| A3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Minimal Setup (Express)
|
||||
|
||||
### 1. Install
|
||||
|
||||
```bash
|
||||
npm install @copilotkit/runtime express
|
||||
```
|
||||
|
||||
### 2. Create the runtime
|
||||
|
||||
```typescript
|
||||
// server.ts
|
||||
import express from "express";
|
||||
import { CopilotRuntime } from "@copilotkit/runtime";
|
||||
import { createCopilotEndpointExpress } from "@copilotkit/runtime/express";
|
||||
|
||||
const app = express();
|
||||
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: {
|
||||
default: myAgent, // Any AbstractAgent implementation
|
||||
},
|
||||
});
|
||||
|
||||
app.use("/api/copilotkit", createCopilotEndpointExpress({ runtime }));
|
||||
|
||||
app.listen(3000);
|
||||
```
|
||||
|
||||
That's it. The endpoint handler creates these routes automatically:
|
||||
|
||||
| Route | Method | What it does |
|
||||
| ----------------------------------------------- | ------ | ----------------------------------- |
|
||||
| `/api/copilotkit/info` | GET | Returns list of available agents |
|
||||
| `/api/copilotkit/agent/:agentId/run` | POST | Run an agent (returns SSE stream) |
|
||||
| `/api/copilotkit/agent/:agentId/connect` | POST | Connect/reconnect to a thread |
|
||||
| `/api/copilotkit/agent/:agentId/stop/:threadId` | POST | Stop a running agent |
|
||||
| `/api/copilotkit/transcribe` | POST | Audio transcription (if configured) |
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Client as Frontend
|
||||
participant EP as Express Router
|
||||
participant RT as CopilotRuntime
|
||||
participant Agent as AI Agent
|
||||
|
||||
Client->>EP: GET /api/copilotkit/info
|
||||
EP->>RT: List agents
|
||||
RT-->>Client: [{ id: "default", description: "..." }]
|
||||
|
||||
Client->>EP: POST /agent/default/run
|
||||
EP->>RT: handleRunAgent({ agentId: "default" })
|
||||
RT->>Agent: runner.run()
|
||||
Agent-->>Client: SSE: TEXT_MESSAGE_START
|
||||
Agent-->>Client: SSE: TEXT_MESSAGE_CONTENT
|
||||
Agent-->>Client: SSE: TEXT_MESSAGE_END
|
||||
Agent-->>Client: SSE: RUN_FINISHED
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Hono Setup
|
||||
|
||||
```typescript
|
||||
import { Hono } from "hono";
|
||||
import { CopilotRuntime } from "@copilotkit/runtime";
|
||||
import { createCopilotEndpointHono } from "@copilotkit/runtime/hono";
|
||||
|
||||
const app = new Hono();
|
||||
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: { default: myAgent },
|
||||
});
|
||||
|
||||
app.route("/api/copilotkit", createCopilotEndpointHono({ runtime }));
|
||||
|
||||
export default app;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Using the Built-in Agent
|
||||
|
||||
CopilotKit includes a built-in agent powered by the Vercel AI SDK:
|
||||
|
||||
```typescript
|
||||
import { CopilotRuntime } from "@copilotkit/runtime";
|
||||
import { BuiltInAgent } from "@copilotkit/runtime/v2";
|
||||
|
||||
const agent = new BuiltInAgent({
|
||||
model: "openai/gpt-4o",
|
||||
systemPrompt: "You are a helpful shopping assistant.",
|
||||
});
|
||||
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: { default: agent },
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Lazy-Loaded Agents
|
||||
|
||||
Agents can be a `Promise` — useful for dynamic loading:
|
||||
|
||||
```typescript
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: loadAgents(), // Returns Promise<Record<string, AbstractAgent>>
|
||||
});
|
||||
|
||||
async function loadAgents() {
|
||||
const config = await fetchConfig();
|
||||
return {
|
||||
default: new BuiltInAgent({ model: config.model }),
|
||||
research: new HttpAgent({ url: config.researchAgentUrl }),
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## All CopilotRuntime Options
|
||||
|
||||
```typescript
|
||||
const runtime = new CopilotRuntime({
|
||||
// Required: map of agent IDs to agent instances
|
||||
agents: {
|
||||
default: defaultAgent,
|
||||
research: researchAgent,
|
||||
coding: codingAgent,
|
||||
},
|
||||
|
||||
// Optional: how agents are executed
|
||||
runner: new InMemoryAgentRunner(), // default
|
||||
|
||||
// Optional: audio → text
|
||||
transcriptionService: myTranscriptionService,
|
||||
|
||||
// Optional: intercept requests before processing
|
||||
beforeRequestMiddleware: async ({ request, path }) => {
|
||||
console.log(`[${path}] Request received`);
|
||||
// Return modified request, or void to pass through
|
||||
},
|
||||
|
||||
// Optional: run after response is prepared
|
||||
afterRequestMiddleware: async ({ response, path }) => {
|
||||
console.log(`[${path}] Response sent`);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "CopilotRuntime Options"
|
||||
direction TB
|
||||
|
||||
subgraph "Required"
|
||||
AGENTS["agents<br/><i>Record<string, AbstractAgent></i>"]
|
||||
end
|
||||
|
||||
subgraph "Optional"
|
||||
RUNNER["runner<br/><i>AgentRunner</i><br/><i>Default: InMemoryAgentRunner</i>"]
|
||||
TS["transcriptionService<br/><i>TranscriptionService</i>"]
|
||||
BM["beforeRequestMiddleware<br/><i>(request, path) → Request | void</i>"]
|
||||
AM["afterRequestMiddleware<br/><i>(response, path) → void</i>"]
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## AgentRunner: How Agents Execute
|
||||
|
||||
The `AgentRunner` is the abstraction that actually executes agents. It manages threads, streaming, and agent lifecycle.
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "AgentRunner (Abstract)"
|
||||
RUN["run(request)<br/><i>Execute agent, return Observable<BaseEvent></i>"]
|
||||
CONNECT["connect(request)<br/><i>Reconnect to existing thread</i>"]
|
||||
RUNNING["isRunning(request)<br/><i>Check if thread is active</i>"]
|
||||
STOP["stop(request)<br/><i>Abort a running thread</i>"]
|
||||
end
|
||||
|
||||
subgraph Implementations
|
||||
IM["<b>InMemoryAgentRunner</b><br/><i>Default — in-process, ephemeral</i>"]
|
||||
SQ["<b>SQLiteAgentRunner</b><br/><i>Persistent state on disk</i>"]
|
||||
CU["<b>Your Custom Runner</b><br/><i>Extend AgentRunner</i>"]
|
||||
end
|
||||
|
||||
IM --> RUN
|
||||
SQ --> RUN
|
||||
CU --> RUN
|
||||
```
|
||||
|
||||
### InMemoryAgentRunner (default)
|
||||
|
||||
Stores agent threads in memory. Simple, no persistence. Threads are lost on server restart.
|
||||
|
||||
```typescript
|
||||
import { InMemoryAgentRunner } from "@copilotkit/runtime";
|
||||
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: { default: myAgent },
|
||||
runner: new InMemoryAgentRunner(), // This is the default
|
||||
});
|
||||
```
|
||||
|
||||
### SQLiteAgentRunner (persistent)
|
||||
|
||||
Stores agent threads in SQLite. Survives restarts.
|
||||
|
||||
```typescript
|
||||
import { SQLiteAgentRunner } from "@copilotkit/sqlite-runner";
|
||||
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: { default: myAgent },
|
||||
runner: new SQLiteAgentRunner({ dbPath: "./agent-state.db" }),
|
||||
});
|
||||
```
|
||||
|
||||
### Custom Runner
|
||||
|
||||
```typescript
|
||||
import { AgentRunner } from "@copilotkit/runtime";
|
||||
import { Observable } from "rxjs";
|
||||
|
||||
class RedisAgentRunner extends AgentRunner {
|
||||
async run(request) {
|
||||
// Store in Redis, return event stream
|
||||
return new Observable((subscriber) => {
|
||||
// ... your implementation
|
||||
});
|
||||
}
|
||||
|
||||
async connect(request) {
|
||||
// Reconnect to existing Redis-stored thread
|
||||
}
|
||||
|
||||
async isRunning(request) {
|
||||
// Check Redis for active thread
|
||||
}
|
||||
|
||||
async stop(request) {
|
||||
// Signal thread to stop
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Middleware
|
||||
|
||||
Middleware lets you intercept requests before and after processing.
|
||||
|
||||
### Before Request Middleware
|
||||
|
||||
Runs before any handler. Use it for auth, logging, request modification.
|
||||
|
||||
```typescript
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: { default: myAgent },
|
||||
beforeRequestMiddleware: async ({ request, path, runtime }) => {
|
||||
// Example: verify auth token
|
||||
const token = request.headers.get("authorization");
|
||||
if (!token) {
|
||||
return new Response("Unauthorized", { status: 401 });
|
||||
}
|
||||
|
||||
// Example: add user context to request
|
||||
const user = await verifyToken(token);
|
||||
request.headers.set("x-user-id", user.id);
|
||||
|
||||
// Return modified request (or void to pass through)
|
||||
return request;
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### After Request Middleware
|
||||
|
||||
Runs after the response is prepared but before it's sent.
|
||||
|
||||
```typescript
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: { default: myAgent },
|
||||
afterRequestMiddleware: async ({ response, path, runtime }) => {
|
||||
// Example: log responses
|
||||
console.log(`[${path}] Response status: ${response.status}`);
|
||||
|
||||
// Example: add custom headers
|
||||
// (Note: response may be SSE stream)
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Client
|
||||
participant Before as beforeRequestMiddleware
|
||||
participant Handler as Route Handler
|
||||
participant After as afterRequestMiddleware
|
||||
|
||||
Client->>Before: HTTP Request
|
||||
alt Middleware rejects
|
||||
Before-->>Client: 401 Unauthorized
|
||||
else Middleware passes
|
||||
Before->>Handler: (modified) Request
|
||||
Handler->>After: Response
|
||||
After-->>Client: Final Response
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Transcription Service
|
||||
|
||||
Enable audio-to-text transcription:
|
||||
|
||||
```typescript
|
||||
import { TranscriptionService } from "@copilotkit/runtime";
|
||||
|
||||
class OpenAITranscription extends TranscriptionService {
|
||||
async transcribeFile({ audioFile, mimeType, size }) {
|
||||
const formData = new FormData();
|
||||
formData.append("file", audioFile);
|
||||
formData.append("model", "whisper-1");
|
||||
|
||||
const response = await fetch(
|
||||
"https://api.openai.com/v1/audio/transcriptions",
|
||||
{
|
||||
method: "POST",
|
||||
headers: { Authorization: `Bearer ${process.env.OPENAI_API_KEY}` },
|
||||
body: formData,
|
||||
},
|
||||
);
|
||||
|
||||
const result = await response.json();
|
||||
return result.text;
|
||||
}
|
||||
}
|
||||
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: { default: myAgent },
|
||||
transcriptionService: new OpenAITranscription(),
|
||||
});
|
||||
```
|
||||
|
||||
The `/transcribe` endpoint is only active when `transcriptionService` is configured.
|
||||
|
||||
---
|
||||
|
||||
## Request Flow Detail
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Client as Frontend
|
||||
participant CORS as CORS Handler
|
||||
participant Before as Before Middleware
|
||||
participant Router as Route Handler
|
||||
participant Runtime as CopilotRuntime
|
||||
participant Runner as AgentRunner
|
||||
participant Agent as AI Agent
|
||||
participant After as After Middleware
|
||||
|
||||
Client->>CORS: POST /agent/default/run
|
||||
CORS->>Before: Check middleware
|
||||
Before->>Router: Forward request
|
||||
Router->>Runtime: handleRunAgent()
|
||||
|
||||
Note over Runtime: 1. Resolve agents (await if Promise)
|
||||
Note over Runtime: 2. Find agent by ID
|
||||
Note over Runtime: 3. Clone agent (avoid shared state)
|
||||
Note over Runtime: 4. Parse RunAgentInput from body
|
||||
Note over Runtime: 5. Set messages, state, threadId
|
||||
|
||||
Runtime->>Runner: runner.run({ agent, input })
|
||||
Runner->>Agent: agent.runAgent(input)
|
||||
|
||||
loop SSE Stream
|
||||
Agent-->>Client: event: TEXT_MESSAGE_CONTENT
|
||||
end
|
||||
|
||||
Agent-->>Client: event: RUN_FINISHED
|
||||
Router->>After: Response complete
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Full Example: Express + Multiple Agents + Middleware
|
||||
|
||||
```typescript
|
||||
import express from "express";
|
||||
import { CopilotRuntime } from "@copilotkit/runtime";
|
||||
import { createCopilotEndpointExpress } from "@copilotkit/runtime/express";
|
||||
import { BuiltInAgent } from "@copilotkit/runtime/v2";
|
||||
import { SQLiteAgentRunner } from "@copilotkit/sqlite-runner";
|
||||
|
||||
const app = express();
|
||||
|
||||
// Create agents
|
||||
const generalAgent = new BuiltInAgent({
|
||||
model: "openai/gpt-4o",
|
||||
systemPrompt: "You are a helpful assistant.",
|
||||
});
|
||||
|
||||
const codeAgent = new BuiltInAgent({
|
||||
model: "openai/gpt-4o",
|
||||
systemPrompt: "You are a coding expert. Always provide code examples.",
|
||||
});
|
||||
|
||||
// Create runtime with all optional features
|
||||
const runtime = new CopilotRuntime({
|
||||
agents: {
|
||||
default: generalAgent,
|
||||
coding: codeAgent,
|
||||
},
|
||||
|
||||
// Persistent agent state
|
||||
runner: new SQLiteAgentRunner({ dbPath: "./data/agents.db" }),
|
||||
|
||||
// Auth middleware
|
||||
beforeRequestMiddleware: async ({ request, path }) => {
|
||||
const token = request.headers.get("authorization")?.replace("Bearer ", "");
|
||||
if (!token) {
|
||||
return new Response(JSON.stringify({ error: "Unauthorized" }), {
|
||||
status: 401,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}
|
||||
// Validate token...
|
||||
},
|
||||
|
||||
// Logging middleware
|
||||
afterRequestMiddleware: async ({ path }) => {
|
||||
console.log(`[CopilotKit] ${new Date().toISOString()} ${path}`);
|
||||
},
|
||||
});
|
||||
|
||||
// Mount CopilotKit endpoints
|
||||
app.use("/api/copilotkit", createCopilotEndpointExpress({ runtime }));
|
||||
|
||||
app.listen(3000, () => {
|
||||
console.log("Server running on :3000");
|
||||
console.log("CopilotKit endpoints at /api/copilotkit/*");
|
||||
});
|
||||
```
|
||||
@@ -0,0 +1,535 @@
|
||||
# Vanilla JavaScript Setup Guide
|
||||
|
||||
This guide shows how to use CopilotKit without React or Angular — using the core API directly. This works with any framework (Vue, Svelte, vanilla JS, Node.js, etc).
|
||||
|
||||
---
|
||||
|
||||
## What Talks to What
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Your Code
|
||||
App["Your Application"]
|
||||
Sub["Event Subscribers"]
|
||||
end
|
||||
|
||||
subgraph CopilotKit Core
|
||||
Core["<b>CopilotKitCore</b><br/><i>Orchestrator</i>"]
|
||||
AR["AgentRegistry"]
|
||||
RH["RunHandler"]
|
||||
CS["ContextStore"]
|
||||
end
|
||||
|
||||
subgraph Transport
|
||||
Proxy["ProxiedAgent<br/><i>HTTP + SSE</i>"]
|
||||
end
|
||||
|
||||
subgraph Your Server
|
||||
Runtime["CopilotRuntime"]
|
||||
end
|
||||
|
||||
App -->|creates| Core
|
||||
App -->|subscribes| Sub
|
||||
Sub -->|listens to| Core
|
||||
Core --> AR
|
||||
Core --> RH
|
||||
Core --> CS
|
||||
Core -->|creates| Proxy
|
||||
Proxy -->|HTTP POST + SSE| Runtime
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Minimal Setup
|
||||
|
||||
### 1. Install
|
||||
|
||||
```bash
|
||||
npm install @copilotkit/core
|
||||
```
|
||||
|
||||
### 2. Create the core instance
|
||||
|
||||
```typescript
|
||||
import { CopilotKitCore } from "@copilotkit/core";
|
||||
|
||||
const copilotKit = new CopilotKitCore({
|
||||
runtimeUrl: "http://localhost:3000/api/copilotkit",
|
||||
});
|
||||
```
|
||||
|
||||
### 3. Get an agent and send a message
|
||||
|
||||
```typescript
|
||||
// Wait for runtime connection
|
||||
const subscription = copilotKit.subscribe({
|
||||
onRuntimeConnectionStatusChanged: async ({ status }) => {
|
||||
if (status === "connected") {
|
||||
// Agents are now available
|
||||
const agent = copilotKit.getAgent("default");
|
||||
|
||||
// Add a user message
|
||||
agent.addMessage({
|
||||
id: crypto.randomUUID(),
|
||||
role: "user",
|
||||
content: "Hello, what can you do?",
|
||||
});
|
||||
|
||||
// Run the agent
|
||||
await copilotKit.runAgent({ agent });
|
||||
}
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
That's it — `CopilotKitCore` handles connecting to the runtime, fetching agents, and managing the event lifecycle.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant App as Your Code
|
||||
participant Core as CopilotKitCore
|
||||
participant Runtime as CopilotRuntime
|
||||
|
||||
App->>Core: new CopilotKitCore({ runtimeUrl })
|
||||
Core->>Runtime: GET /info
|
||||
Runtime-->>Core: Available agents
|
||||
Core->>App: onRuntimeConnectionStatusChanged("connected")
|
||||
App->>Core: getAgent("default")
|
||||
App->>Core: runAgent({ agent })
|
||||
Core->>Runtime: POST /agent/default/run
|
||||
Runtime-->>Core: SSE events stream
|
||||
Core->>App: Messages update via subscription
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Subscribing to Events
|
||||
|
||||
The core provides a rich subscription system — this is how you react to changes without a framework:
|
||||
|
||||
```typescript
|
||||
const subscription = copilotKit.subscribe({
|
||||
// Connection lifecycle
|
||||
onRuntimeConnectionStatusChanged: ({ status }) => {
|
||||
// "disconnected" | "connecting" | "connected" | "error"
|
||||
updateConnectionUI(status);
|
||||
},
|
||||
|
||||
// Agent availability
|
||||
onAgentsChanged: ({ agents }) => {
|
||||
console.log("Available agents:", Object.keys(agents));
|
||||
// agents is Record<string, AbstractAgent>
|
||||
},
|
||||
|
||||
// Tool execution
|
||||
onToolExecutionStart: ({ toolName, args, agentId }) => {
|
||||
showToolSpinner(toolName);
|
||||
},
|
||||
onToolExecutionEnd: ({ toolName, result, error }) => {
|
||||
hideToolSpinner(toolName);
|
||||
if (error) showError(error);
|
||||
},
|
||||
|
||||
// Context changes
|
||||
onContextChanged: ({ context }) => {
|
||||
console.log("Context updated:", context);
|
||||
},
|
||||
|
||||
// Suggestions
|
||||
onSuggestionsChanged: ({ agentId, suggestions }) => {
|
||||
renderSuggestionChips(suggestions);
|
||||
},
|
||||
|
||||
// Errors
|
||||
onError: ({ error, code, context }) => {
|
||||
// code: "AGENT_CONNECT_FAILED" | "AGENT_RUN_FAILED" |
|
||||
// "TOOL_HANDLER_FAILED" | "TOOL_ARGUMENT_PARSE_FAILED" |
|
||||
// "RUNTIME_INFO_FETCH_FAILED"
|
||||
console.error(`[${code}]`, error.message, context);
|
||||
},
|
||||
});
|
||||
|
||||
// Clean up when done
|
||||
subscription.unsubscribe();
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "CopilotKitCore Events"
|
||||
direction TB
|
||||
|
||||
subgraph Connection
|
||||
RCS["onRuntimeConnectionStatusChanged"]
|
||||
end
|
||||
|
||||
subgraph Agents
|
||||
AC["onAgentsChanged"]
|
||||
end
|
||||
|
||||
subgraph "Tool Execution"
|
||||
TES["onToolExecutionStart"]
|
||||
TEE["onToolExecutionEnd"]
|
||||
end
|
||||
|
||||
subgraph Context
|
||||
CC["onContextChanged"]
|
||||
end
|
||||
|
||||
subgraph Suggestions
|
||||
SC["onSuggestionsChanged"]
|
||||
SSL["onSuggestionsStartedLoading"]
|
||||
SFL["onSuggestionsFinishedLoading"]
|
||||
end
|
||||
|
||||
subgraph Errors
|
||||
ERR["onError"]
|
||||
end
|
||||
end
|
||||
|
||||
Your["Your subscriber"] --> RCS
|
||||
Your --> AC
|
||||
Your --> TES
|
||||
Your --> TEE
|
||||
Your --> CC
|
||||
Your --> SC
|
||||
Your --> ERR
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Subscribing to Agent Messages
|
||||
|
||||
In addition to core events, you can subscribe directly to an agent's events:
|
||||
|
||||
```typescript
|
||||
const agent = copilotKit.getAgent("default");
|
||||
|
||||
const agentSub = agent.subscribe({
|
||||
// Messages changed (streaming text, new messages, etc.)
|
||||
onMessagesChanged: ({ messages }) => {
|
||||
renderChatMessages(messages);
|
||||
},
|
||||
|
||||
// Agent state changed
|
||||
onStateChanged: ({ state }) => {
|
||||
updateStateDisplay(state);
|
||||
},
|
||||
|
||||
// Run lifecycle
|
||||
onRunInitialized: () => {
|
||||
showTypingIndicator();
|
||||
},
|
||||
onRunFinalized: () => {
|
||||
hideTypingIndicator();
|
||||
},
|
||||
onRunFailed: ({ error }) => {
|
||||
showError(error);
|
||||
},
|
||||
|
||||
// Granular event tracking
|
||||
onToolCallStartEvent: ({ event }) => {
|
||||
console.log("Tool call:", event.name);
|
||||
},
|
||||
onToolCallEndEvent: ({ toolCallArgs }) => {
|
||||
console.log("Tool args:", toolCallArgs);
|
||||
},
|
||||
onToolCallResultEvent: ({ event }) => {
|
||||
console.log("Tool result:", event.result);
|
||||
},
|
||||
});
|
||||
|
||||
// Clean up
|
||||
agentSub.unsubscribe();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Registering Tools
|
||||
|
||||
```typescript
|
||||
import { z } from "zod";
|
||||
|
||||
// Add a tool
|
||||
copilotKit.addTool({
|
||||
name: "getWeather",
|
||||
description: "Get current weather for a location",
|
||||
parameters: z.object({
|
||||
city: z.string().describe("City name"),
|
||||
unit: z.enum(["celsius", "fahrenheit"]).default("celsius"),
|
||||
}),
|
||||
handler: async ({ city, unit }) => {
|
||||
const data = await fetch(`/api/weather?city=${city}&unit=${unit}`);
|
||||
return await data.text();
|
||||
},
|
||||
followUp: true, // Agent will continue after getting the result
|
||||
});
|
||||
|
||||
// Remove a tool
|
||||
copilotKit.removeTool("getWeather");
|
||||
```
|
||||
|
||||
### Agent-Specific Tools
|
||||
|
||||
```typescript
|
||||
// This tool is only available to the "research" agent
|
||||
copilotKit.addTool({
|
||||
name: "searchPapers",
|
||||
description: "Search academic papers",
|
||||
agentId: "research", // Only this agent can call it
|
||||
parameters: z.object({ query: z.string() }),
|
||||
handler: async ({ query }) => {
|
||||
return await searchPapers(query);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Providing Context
|
||||
|
||||
```typescript
|
||||
// Add context (returns an ID for later removal)
|
||||
const contextId = copilotKit.addContext({
|
||||
description: "Current user session",
|
||||
value: JSON.stringify({
|
||||
userId: "user_123",
|
||||
role: "admin",
|
||||
currentPage: "/dashboard",
|
||||
}),
|
||||
});
|
||||
|
||||
// Update context (remove + re-add)
|
||||
copilotKit.removeContext(contextId);
|
||||
const newContextId = copilotKit.addContext({
|
||||
description: "Current user session",
|
||||
value: JSON.stringify({
|
||||
userId: "user_123",
|
||||
role: "admin",
|
||||
currentPage: "/settings",
|
||||
}),
|
||||
});
|
||||
|
||||
// Remove when no longer relevant
|
||||
copilotKit.removeContext(newContextId);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Constructor Options
|
||||
|
||||
```typescript
|
||||
const copilotKit = new CopilotKitCore({
|
||||
// Required
|
||||
runtimeUrl: "http://localhost:3000/api/copilotkit",
|
||||
|
||||
// Authentication
|
||||
headers: { Authorization: "Bearer my-token" },
|
||||
credentials: "include", // Forward cookies
|
||||
|
||||
// Runtime transport mode
|
||||
runtimeTransport: "rest", // "rest" (default) or "single"
|
||||
|
||||
// Custom properties forwarded to agents
|
||||
properties: {
|
||||
userId: "user_123",
|
||||
environment: "production",
|
||||
},
|
||||
|
||||
// Initial tools
|
||||
tools: [
|
||||
{
|
||||
name: "myTool",
|
||||
parameters: z.object({ input: z.string() }),
|
||||
handler: async ({ input }) => `Result: ${input}`,
|
||||
},
|
||||
],
|
||||
|
||||
// Local agents (dev only — normally fetched from runtime)
|
||||
agents__unsafe_dev_only: {
|
||||
test: myLocalAgent,
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "CopilotKitCore Config"
|
||||
direction TB
|
||||
|
||||
subgraph Required
|
||||
URL["runtimeUrl"]
|
||||
end
|
||||
|
||||
subgraph "Optional: Auth"
|
||||
H["headers"]
|
||||
C["credentials"]
|
||||
end
|
||||
|
||||
subgraph "Optional: Transport"
|
||||
RT["runtimeTransport<br/><i>'rest' or 'single'</i>"]
|
||||
end
|
||||
|
||||
subgraph "Optional: Data"
|
||||
P["properties"]
|
||||
T["tools"]
|
||||
SC["suggestionsConfig"]
|
||||
end
|
||||
|
||||
subgraph "Optional: Dev"
|
||||
AG["agents__unsafe_dev_only"]
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Using HttpAgent Directly (No CopilotKit)
|
||||
|
||||
For the simplest possible setup, you can skip CopilotKit entirely and use AG-UI's `HttpAgent` directly:
|
||||
|
||||
```typescript
|
||||
import { HttpAgent } from "@ag-ui/client";
|
||||
|
||||
const agent = new HttpAgent({
|
||||
agentId: "my-agent",
|
||||
url: "http://localhost:3000/api/copilotkit/agent/my-agent/run",
|
||||
headers: { Authorization: "Bearer token" },
|
||||
});
|
||||
|
||||
// Subscribe to messages
|
||||
agent.subscribe({
|
||||
onMessagesChanged: ({ messages }) => {
|
||||
console.log("Messages:", messages);
|
||||
},
|
||||
});
|
||||
|
||||
// Send a message and run
|
||||
agent.addMessage({
|
||||
id: crypto.randomUUID(),
|
||||
role: "user",
|
||||
content: "Hello!",
|
||||
});
|
||||
|
||||
await agent.runAgent();
|
||||
```
|
||||
|
||||
> **When to use this:** Only if you want zero abstraction and just need to talk to a single agent. You lose tools, context, suggestions, and multi-agent orchestration.
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph "HttpAgent (Minimal)"
|
||||
HA["HttpAgent"]
|
||||
end
|
||||
|
||||
subgraph "CopilotKitCore (Full)"
|
||||
CKC["CopilotKitCore"]
|
||||
Tools["Tool Registry"]
|
||||
Context["Context Store"]
|
||||
Suggestions["Suggestions"]
|
||||
Multi["Multi-Agent"]
|
||||
CKC --> Tools
|
||||
CKC --> Context
|
||||
CKC --> Suggestions
|
||||
CKC --> Multi
|
||||
end
|
||||
|
||||
HA -->|HTTP + SSE| Server["Your Server"]
|
||||
CKC -->|HTTP + SSE| Server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Full Example: Simple Chat App (No Framework)
|
||||
|
||||
```typescript
|
||||
import { CopilotKitCore } from "@copilotkit/core";
|
||||
import { z } from "zod";
|
||||
|
||||
// DOM elements
|
||||
const messagesDiv = document.getElementById("messages")!;
|
||||
const input = document.getElementById("input") as HTMLInputElement;
|
||||
const sendBtn = document.getElementById("send")!;
|
||||
const statusSpan = document.getElementById("status")!;
|
||||
|
||||
// Initialize CopilotKit
|
||||
const copilotKit = new CopilotKitCore({
|
||||
runtimeUrl: "/api/copilotkit",
|
||||
});
|
||||
|
||||
let currentAgent: any = null;
|
||||
|
||||
// Subscribe to core events
|
||||
copilotKit.subscribe({
|
||||
onRuntimeConnectionStatusChanged: ({ status }) => {
|
||||
statusSpan.textContent = status;
|
||||
if (status === "connected") {
|
||||
currentAgent = copilotKit.getAgent("default");
|
||||
setupAgentSubscription();
|
||||
input.disabled = false;
|
||||
}
|
||||
},
|
||||
onToolExecutionStart: ({ toolName }) => {
|
||||
appendMessage("system", `Running tool: ${toolName}...`);
|
||||
},
|
||||
onError: ({ error, code }) => {
|
||||
appendMessage("error", `[${code}] ${error.message}`);
|
||||
},
|
||||
});
|
||||
|
||||
// Subscribe to agent messages
|
||||
function setupAgentSubscription() {
|
||||
currentAgent.subscribe({
|
||||
onMessagesChanged: ({ messages }) => {
|
||||
messagesDiv.innerHTML = "";
|
||||
messages.forEach((msg) => appendMessage(msg.role, msg.content));
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// Register a tool
|
||||
copilotKit.addTool({
|
||||
name: "getCurrentTime",
|
||||
description: "Get the current date and time",
|
||||
parameters: z.object({}),
|
||||
handler: async () => new Date().toISOString(),
|
||||
});
|
||||
|
||||
// Send message
|
||||
async function sendMessage() {
|
||||
const text = input.value.trim();
|
||||
if (!text || !currentAgent) return;
|
||||
|
||||
input.value = "";
|
||||
currentAgent.addMessage({
|
||||
id: crypto.randomUUID(),
|
||||
role: "user",
|
||||
content: text,
|
||||
});
|
||||
|
||||
await copilotKit.runAgent({ agent: currentAgent });
|
||||
}
|
||||
|
||||
sendBtn.addEventListener("click", sendMessage);
|
||||
input.addEventListener("keydown", (e) => {
|
||||
if (e.key === "Enter") sendMessage();
|
||||
});
|
||||
|
||||
// Helper
|
||||
function appendMessage(role: string, content: string) {
|
||||
const div = document.createElement("div");
|
||||
div.className = `message ${role}`;
|
||||
div.textContent = `${role}: ${content}`;
|
||||
messagesDiv.appendChild(div);
|
||||
}
|
||||
```
|
||||
|
||||
```html
|
||||
<!-- index.html -->
|
||||
<div id="app">
|
||||
<div>Status: <span id="status">connecting...</span></div>
|
||||
<div id="messages"></div>
|
||||
<input id="input" disabled placeholder="Connecting..." />
|
||||
<button id="send">Send</button>
|
||||
</div>
|
||||
<script type="module" src="./main.ts"></script>
|
||||
```
|
||||
@@ -0,0 +1,45 @@
|
||||
# Browser Compatibility
|
||||
|
||||
## Browserslist Matrix
|
||||
|
||||
The `.browserslistrc` at the repo root declares the intended browser support policy. It is useful for tools that read browserslist directly (e.g. PostCSS autoprefixer, documentation generators). To see the current resolved matrix, run:
|
||||
|
||||
```
|
||||
npx browserslist
|
||||
```
|
||||
|
||||
The resolved set changes over time as `defaults` is a dynamic query maintained by the browserslist project — there is no fixed version table in this doc.
|
||||
|
||||
One entry is explicitly excluded: `not kaios 2.5`. KaiOS 2.5 ships a Gecko 48-era engine and is documented here as out-of-scope so that consumers of this policy (autoprefixer and similar tools) know not to target it.
|
||||
|
||||
Note: `.browserslistrc` does **not** drive `compat-check`. The `es-check` targets (ES2022 for ESM/CJS; ES2018 for UMD, except `@copilotkit/react-core` UMD which uses ES2020 — see "What `compat-check` Does" below) are hardcoded in the `compat-check` scripts and are independent of the browserslist query. See the "Decoupled" section below for why these two concerns are kept separate.
|
||||
|
||||
## What `compat-check` Does
|
||||
|
||||
`compat-check` runs `es-check` against the built `dist/` output after a build:
|
||||
|
||||
- **ESM and CJS files** are checked against **ES2022** (matches the `tsdown` `target: "es2022"` setting for all packages).
|
||||
- **UMD files** are checked against **ES2018** for all packages except `@copilotkit/react-core`, which uses **ES2020** because its source contains dynamic `import()` expressions that cannot be downcompiled to ES2018 by rollup when the imported modules are external.
|
||||
|
||||
If any file uses syntax above the target level, `es-check` fails loudly with the offending file and the problematic feature. The check runs in CI so failures surface before a release.
|
||||
|
||||
## Why It Is Decoupled from the `tsdown` Target
|
||||
|
||||
`tsdown`'s `target` option tells the compiler what it _should_ emit — but there are two ways the actual output syntax can exceed that target without tsdown itself introducing the violation:
|
||||
|
||||
1. **Transitive dependencies.** Bundled code from a dep can carry syntax that was never downcompiled because tsdown only transforms its own output, not pre-compiled dep artifacts.
|
||||
2. **tsdown version bumps.** A new tsdown version may change how it handles certain patterns, inadvertently emitting newer syntax.
|
||||
|
||||
Neither of those failures would be caught by reading the tsdown config. The `compat-check` catches them by inspecting the real built artifacts, so drift is detected before a customer encounters a parse error in a supported browser.
|
||||
|
||||
## Handling a Failure
|
||||
|
||||
When `compat-check` fails, the output from `es-check` will name the offending file and the syntax it objected to. The decision tree is:
|
||||
|
||||
1. **Identify the offending file.** Look at the `es-check` error output — it will point to a specific file in `dist/`.
|
||||
2. **Trace it to a dep or to first-party code.** Check whether the syntax comes from a bundled dependency or from code we wrote. Source maps or a quick `grep` of the dist file for a known identifier usually clarifies this.
|
||||
3. **Decide:**
|
||||
- If the violation came from **a dep update** that unintentionally introduced newer syntax: pin or override the dep, or open an issue upstream asking them to ship a downcompiled artifact.
|
||||
- If we **intentionally dropped support** for an older browser tier: raise the `es-check` target in the `compat-check` script and update this document to match. Note: updating `.browserslistrc` has no effect on `compat-check` — the es-check targets are hardcoded in the scripts and are independent of the browserslist query.
|
||||
|
||||
Do not suppress the failure or widen the allowed syntax band without updating the `compat-check` targets and this document to match.
|
||||
@@ -0,0 +1,87 @@
|
||||
# Bundle Size Tracking
|
||||
|
||||
## How it works — two tiers
|
||||
|
||||
### Tier 1: CI (compressed-size-action)
|
||||
|
||||
`static_bundle_size.yml` runs on every PR via `preactjs/compressed-size-action@v2.9.1`. It scans a glob (`packages/{...}/dist/**/*.{mjs,js,cjs}`), computes the gzip size of each matched file (the action's default compression; the workflow sets no `compression` input), and posts a PR comment showing per-file diffs. It has **no hard-fail** (Phase 1).
|
||||
|
||||
> **Fork PRs:** `pull_request` runs triggered from a fork receive a read-only `GITHUB_TOKEN`, so `compressed-size-action` cannot post or update the PR comment — it prints the size report to the job logs instead. The measurement still runs; only the comment is unavailable. This is an accepted Phase 1 limitation (the report is informational and there is no hard-fail). If the PR comment ever becomes a required signal, switch to a `pull_request_target` + `workflow_run` relay pattern so the comment is posted from a trusted context without exposing write tokens to fork code.
|
||||
|
||||
Key facts:
|
||||
|
||||
- Reports by **file path**, not by named entry — it does not read `.size-limit.json` at all.
|
||||
- The action runs `build-script: build` (the root `build` script — `nx run-many -t build` over all `packages/**`) on both the PR branch and the base branch, then measures only the files matched by the `pattern` glob. The root `build` script is used (rather than a bundle-size-specific one) because the action must build the base branch too, and `build` exists on every branch. No separate build step is needed before the workflow triggers — the action handles both builds.
|
||||
- PR comments show paths like `packages/react-core/dist/index.mjs (+1.2 kB gzip)`.
|
||||
|
||||
### The CopilotChat regression signal (job summary, not the PR comment)
|
||||
|
||||
The `copilotchat-import-size` job in `static_bundle_size.yml` measures what an app
|
||||
importing `{ CopilotChat }` from `@copilotkit/react-core/v2` bundles, via
|
||||
`packages/react-core/scripts/measure-copilotchat.mjs` (run locally with
|
||||
`pnpm --filter @copilotkit/react-core size:headline`). It drives `esbuild`
|
||||
directly — bundling `{ CopilotChat }` minified, with `react`/`react-dom` external
|
||||
and CSS/fonts stubbed to `empty` (we measure JS) — and writes the total gzipped
|
||||
JS to the GitHub **job summary**.
|
||||
|
||||
**This is a _relative_ regression signal, not a production figure.** Its absolute
|
||||
value (currently ~3 MB gzip) is an esbuild number; a real consumer bundler
|
||||
(Vite/Next/webpack) splits eager-vs-lazy differently and reports different
|
||||
absolutes — the Notion "Header Embed Bundle Readout" measured ~386 kB _main
|
||||
initial JS_ under Vite, with the shiki/mermaid language packs as separate
|
||||
generated chunks. The script's worth is **consistency**: the same measurement
|
||||
every PR, so a change that grows CopilotChat's JS shows up, and the number
|
||||
collapses once OSS-122 moves the language packs to a CDN. A faithful _production_
|
||||
headline (real Next 15 fixture + `@next/bundle-analyzer`) is OSS-122 Phase 0.
|
||||
|
||||
Why a custom script and not `size-limit`: CopilotChat pulls `katex`'s CSS, whose
|
||||
`url()` font refs crash `@size-limit/esbuild` (which exposes no loader hook).
|
||||
Driving esbuild directly lets us stub the CSS/font assets.
|
||||
|
||||
### Tier 2: Local dev (size-limit)
|
||||
|
||||
The four **bundled** packages (`core`, `react-core`, `react-ui`, `react-textarea`) each have a `.size-limit.json` at their root listing one or more named entries pointing at `dist/` paths. Run locally via:
|
||||
|
||||
```
|
||||
pnpm --filter <pkg> size
|
||||
```
|
||||
|
||||
The five unbundled packages (`shared`, `runtime-client-gql`, `web-inspector`, `voice`, `a2ui-renderer`) have no `.size-limit.json` and no `size` script — their sizes are tracked by the CI glob only.
|
||||
|
||||
> **Node version requirement:** `size-limit@12.1.0` requires Node 20, 22, or 24+ (`^20 || ^22 || >=24`). Running `pnpm --filter <pkg> size` on Node 18 will produce an `EBADENGINE` error.
|
||||
|
||||
## Where configuration lives
|
||||
|
||||
`.size-limit.json` files live at the root of each bundled package (`core`, `react-core`, `react-ui`, `react-textarea`) and are used exclusively by the local `size` script. They are not read by CI.
|
||||
|
||||
## Adding a new measurement
|
||||
|
||||
Only bundled packages support local size tracking. For unbundled packages, CI covers all chunk files via the glob; no local config is needed.
|
||||
|
||||
To add a measurement to a bundled package:
|
||||
|
||||
1. Add an entry to the package's `.size-limit.json`:
|
||||
```json
|
||||
{ "name": "my-package: MyExport", "path": "dist/index.mjs", "gzip": true }
|
||||
```
|
||||
2. Build the package first: `pnpm --filter <pkg> build`
|
||||
3. Run locally: `pnpm --filter <pkg> size`
|
||||
4. Commit the updated `.size-limit.json`.
|
||||
|
||||
Note: named entries appear in **local** size-limit output only. CI PR comments report by file path from the glob, not by these names.
|
||||
|
||||
> **Bundled vs. unbundled packages:** `@size-limit/file` reports accurate sizes for bundled packages (those that build a single-file bundle). For unbundled packages (those that emit re-export barrels with separate chunk files), `@size-limit/file` only counts the barrel file — the CI `compressed-size-action` glob covers all chunks correctly regardless.
|
||||
|
||||
## CI behavior (Phase 1 — current)
|
||||
|
||||
`static_bundle_size.yml` posts a comment with per-file gzip diffs on every PR. It has **no hard-fail**. Sizes today reflect pre-OSS-122 bloat; adding budget limits now would either lock in that bloat permanently or fail immediately on every PR. Neither is useful.
|
||||
|
||||
## Phase 2 — after OSS-122 (separate ticket, blocked)
|
||||
|
||||
Once OSS-122 has reduced the baseline:
|
||||
|
||||
1. Add `"limit"` fields to each `.size-limit.json` entry.
|
||||
2. Add a size-limit step to the CI workflow (currently the workflow has no size-limit step — Phase 2 adds one, it does not flip an existing step).
|
||||
3. PRs that regress past a limit will fail CI.
|
||||
|
||||
Do not add `"limit"` fields before OSS-122 lands.
|
||||
@@ -0,0 +1,441 @@
|
||||
# LangGraph-Python Column Wave 1 — Discovered Bugs & Descoped Cells
|
||||
|
||||
Wave 1 of the langgraph-python column completeness effort surfaced the
|
||||
following issues while authoring QA checklists, E2E specs, and ops probes.
|
||||
Each is tracked for follow-up separately from Wave 1's merge.
|
||||
|
||||
## How to read this
|
||||
|
||||
- **Descoped cell**: the Wave 1 "green the column" declaration explicitly
|
||||
excludes this cell. The dashboard will show amber/red for it until the
|
||||
underlying cause is addressed.
|
||||
- **Follow-up**: the issue doesn't block Wave 1 completion; filed here for
|
||||
later.
|
||||
|
||||
Entries are grouped by area (docs, backend-agent, probe plumbing, frontend
|
||||
/ CSS, test infra). Cross-references use the `W8-*` tag as it appears in
|
||||
`docs/superpowers/plans/langgraph-python-column-wave1-bugs-scratch.md`
|
||||
and in inline `// See W8-*` comments inside Playwright specs under
|
||||
`showcase/packages/langgraph-python/tests/e2e/`.
|
||||
|
||||
## Bugs
|
||||
|
||||
### B1 — probe-docs.ts does not read `packages/*/docs-links.json` overrides (W8-1)
|
||||
|
||||
- **Symptom:** `scripts/probe-docs.ts` only validates URLs in
|
||||
`shared/feature-registry.json`. Per-integration overrides in
|
||||
`packages/<slug>/docs-links.json` are invisible to the probe, so
|
||||
`showcase/shell/src/data/docs-status.json` can report `notfound` for a
|
||||
URL that actually resolves 200 — and conversely a broken override would
|
||||
not show red.
|
||||
- **Evidence:** Post-Task-1.4 probe aggregate is `ok=0 notfound=60 error=0
|
||||
missing=16` even though every langgraph-python cell except
|
||||
`chat-customization-css` renders ✓/✓ on the dashboard. The dashboard
|
||||
flips to green via
|
||||
`showcase/shell-dashboard/src/components/cell-pieces.tsx:36-57` which
|
||||
trusts the override. Example: `og_docs_url`
|
||||
`https://docs.copilotkit.ai/langgraph/prebuilt-components` in
|
||||
`packages/langgraph-python/docs-links.json` is 200-verified but shows
|
||||
`notfound` in the probe output.
|
||||
- **Suspected cause:** `probe-docs.ts` scope predates the
|
||||
`docs-links.json` override pattern; it reads only `REGISTRY_PATH` and
|
||||
never walks `packages/*/docs-links.json`.
|
||||
- **Suggested owner:** showcase ops.
|
||||
- **Next step:** either (a) extend `probe-docs.ts` to walk
|
||||
`packages/*/docs-links.json` and emit per-integration docs-status rows,
|
||||
or (b) teach `cell-pieces.tsx` to defer to probe state whenever a URL
|
||||
exists.
|
||||
- **Descoped cell(s):** none — dashboard is already green via the
|
||||
override. Affects probe accuracy column-wide but not visible cell state.
|
||||
|
||||
### B2 — Every `/features/<id>` URL in feature-registry soft-404s (W8-3)
|
||||
|
||||
- **Symptom:** Every `https://docs.copilotkit.ai/features/<id>` entry in
|
||||
`shared/feature-registry.json` returns the Next.js catch-all
|
||||
`[[...slug]]` page. This affects integrations that don't ship a
|
||||
`docs-links.json` override.
|
||||
- **Evidence:** Curl of any `/features/<id>` URL returns 200 with
|
||||
`x-matched-path: /[[...slug]]` or `/integrations/[[...slug]]`. Probe
|
||||
output's `notfound=60` aggregate is almost entirely these fallback
|
||||
URLs. See `docs/superpowers/plans/langgraph-python-docs-audit.md`
|
||||
surprise #3.
|
||||
- **Suspected cause:** registry URLs were written against an older docs
|
||||
IA (`/features/<id>`) that no longer exists.
|
||||
- **Suggested owner:** docs IA.
|
||||
- **Next step:** short-term, ensure every integration has a
|
||||
`docs-links.json` override. Long-term, update feature-registry URLs to
|
||||
point at integration-specific pages or drop the feature-level fallbacks.
|
||||
- **Descoped cell(s):** none for langgraph-python (overrides cover every
|
||||
cell). Other integration columns may still render red until each ships
|
||||
its own override.
|
||||
|
||||
### B3 — `chat-customization-css` has no dedicated docs page (W8-2)
|
||||
|
||||
- **Symptom:** langgraph-python ships a `chat-customization-css` demo but
|
||||
no dedicated CSS-customization page exists under docs.copilotkit.ai or
|
||||
shell-docs. The cell renders the "missing" state for og.
|
||||
- **Evidence:**
|
||||
- `packages/langgraph-python/docs-links.json` entry for
|
||||
`chat-customization-css` has `og_docs_url: null` and
|
||||
`shell_docs_path: "/custom-look-and-feel/css"`.
|
||||
- `https://docs.copilotkit.ai/langgraph/custom-look-and-feel/css`
|
||||
soft-404s (catch-all `[[...slug]]`).
|
||||
- `https://docs.copilotkit.ai/custom-look-and-feel/css` also soft-404s.
|
||||
- No `integrations/langgraph/custom-look-and-feel/css.mdx` exists
|
||||
under `showcase/shell-docs/src/content/docs/` (a non-scoped
|
||||
`custom-look-and-feel/css.mdx` does exist, which shell resolution
|
||||
matches).
|
||||
- **Suspected cause:** docs page was never authored.
|
||||
- **Suggested owner:** docs.
|
||||
- **Next step:** author `langgraph/custom-look-and-feel/css` (matching
|
||||
the `/slots` sibling) and the corresponding shell-docs mdx under
|
||||
`integrations/langgraph/custom-look-and-feel/css.mdx`. Then un-null
|
||||
`og_docs_url` in `packages/langgraph-python/docs-links.json`.
|
||||
- **Descoped cell(s):** `chat-customization-css` docs-og.
|
||||
|
||||
### B4 — `reasoning_agent` non-responsive on Railway (W8-3 E2E)
|
||||
|
||||
- **Symptom:** `/demos/agentic-chat-reasoning` on
|
||||
`showcase-langgraph-python-production.up.railway.app` loads fine, but
|
||||
any typed prompt produces no `[data-testid="reasoning-block"]` and no
|
||||
`[data-role="assistant"]` bubble within 60s.
|
||||
- **Evidence:**
|
||||
- Three consecutive E2E runs all time out at 60s on the
|
||||
reasoning-block locator.
|
||||
- Traces under
|
||||
`showcase/packages/langgraph-python/test-results/agentic-chat-reasoning-*`.
|
||||
- Same Railway host handles `frontend-tools` (5/5) and
|
||||
`frontend-tools-async` (2/3 LLM-dependent) — deployment is up; the
|
||||
`reasoning_agent` graph specifically is non-responsive.
|
||||
- Mitigation already landed in
|
||||
`showcase/packages/langgraph-python/tests/e2e/agentic-chat-reasoning.spec.ts`
|
||||
(three `test.skip`s with TODO).
|
||||
- **Suspected cause:** `deepagents.create_deep_agent` /
|
||||
`init_chat_model` path in `src/agents/reasoning_agent.py` may be
|
||||
missing a Python dep or an OpenAI Responses-API permission on Railway,
|
||||
or the agent name mapping in `src/app/api/copilotkit/route.ts:76-77`
|
||||
(`agentic-chat-reasoning` → `reasoning_agent`) fails at the runtime
|
||||
layer.
|
||||
- **Suggested owner:** showcase-langgraph-python deploy.
|
||||
- **Next step:** tail Railway logs while hitting `/api/copilotkit` POST
|
||||
with an `agentic-chat-reasoning` agent run; confirm whether
|
||||
`reasoning_agent.graph` actually imports.
|
||||
- **Descoped cell(s):** `agentic-chat-reasoning` E2E (reasoning-stream
|
||||
assertions skipped; page-load/submit-pipeline still live).
|
||||
|
||||
### B5 — `request_user_approval` does not fire on Railway within 60s (W8-5)
|
||||
|
||||
- **Symptom:** `/demos/hitl-in-app` on Railway loads fine; suggestion
|
||||
pills and the 3 ticket cards render. A typed prompt explicitly naming
|
||||
the tool and a ticket (e.g. "Use request_user_approval to ask me to
|
||||
approve a $50 refund on ticket #12345.") does not cause the agent to
|
||||
invoke the `useFrontendTool` handler. No
|
||||
`[data-testid="approval-dialog-overlay"]` portal appears; all three
|
||||
flows time out at 60s with two Playwright retries each.
|
||||
- **Evidence:** traces under
|
||||
`showcase/packages/langgraph-python/test-results/hitl-in-app-*`.
|
||||
Mitigation in `tests/e2e/hitl-in-app.spec.ts` — three approval flows
|
||||
marked `test.skip` with TODO; page-load / ticket-card / suggestion-pill
|
||||
assertions remain live.
|
||||
- **Suspected cause:** deployed `hitl_in_app_agent` graph may be missing
|
||||
the `request_user_approval` tool binding; or the agent-name mapping in
|
||||
`src/app/api/copilotkit/route.ts` does not route to a graph that
|
||||
receives frontend-tool registration; or the system prompt does not
|
||||
prime the model to call the tool for the typed prompt.
|
||||
- **Suggested owner:** showcase-langgraph-python agent authoring /
|
||||
deploy.
|
||||
- **Next step:** verify the HITL-in-app agent graph definition against
|
||||
the deployed image and confirm
|
||||
`useFrontendTool(request_user_approval)` is registered on the session
|
||||
by the time the user prompt is sent.
|
||||
- **Descoped cell(s):** `hitl-in-app` E2E (approval flows skipped).
|
||||
|
||||
### B6 — `useInterrupt` / `schedule_meeting` does not fire on Railway within 60s (W8-6)
|
||||
|
||||
- **Symptom:** `/demos/gen-ui-interrupt` on Railway loads fine; suggestion
|
||||
pills render. Typed prompts naming the backend tool (e.g. "Use
|
||||
schedule_meeting to book an intro call …") do not trigger the
|
||||
`interrupt_agent` graph's `interrupt()` within 60s; no inline
|
||||
`[data-testid="time-picker-card"]` renders; both pick-a-slot and cancel
|
||||
flows time out.
|
||||
- **Evidence:** traces under
|
||||
`showcase/packages/langgraph-python/test-results/gen-ui-interrupt-*`.
|
||||
Mitigation in `tests/e2e/gen-ui-interrupt.spec.ts` — two interrupt
|
||||
flows marked `test.skip` with TODO.
|
||||
- **Suspected cause:** likely same cluster as B4 / B5. Either the
|
||||
`interrupt_agent` graph (shared with `interrupt-headless`) is not
|
||||
reaching its `interrupt()` on Railway, the `useInterrupt({
|
||||
renderInChat: true })` primitive is not subscribing, or the
|
||||
`schedule_meeting` tool binding is stripped from the deployed graph.
|
||||
- **Suggested owner:** showcase-langgraph-python agent authoring /
|
||||
deploy.
|
||||
- **Next step:** hit `/api/copilotkit` with an `interrupt_agent` run
|
||||
while tailing Railway logs; confirm whether `schedule_meeting` is
|
||||
actually invoked and whether a LangGraph `interrupt()` is emitted on
|
||||
the SSE stream.
|
||||
- **Descoped cell(s):** `gen-ui-interrupt` E2E (interrupt flows
|
||||
skipped).
|
||||
|
||||
### B7 — `readonly-state-agent-context` LLM round-trip stalls past 60s on Railway (W8-READONLY-1)
|
||||
|
||||
- **Symptom:** `/demos/readonly-state-agent-context` on Railway loads, but
|
||||
LLM round-trip for the "Who am I?" suggestion and the equivalent typed
|
||||
prompt stalls past 60s. There is no deterministic frontend tool
|
||||
side-effect to race against (the page simply expects an assistant
|
||||
bubble).
|
||||
- **Evidence:** `showcase/packages/langgraph-python/tests/e2e/
|
||||
readonly-state-agent-context.spec.ts` marks both the suggestion flow
|
||||
and the typed-prompt flow `test.skip` with an inline "See
|
||||
W8-READONLY-1" pointer at `readonly-state-agent-context.spec.ts:76,96`.
|
||||
Scratch file does not mention this entry — **scratch not updated**.
|
||||
- **Suspected cause:** Railway round-trip flakiness; no frontend tool
|
||||
side-effect in the demo makes it impossible to distinguish slow-LLM
|
||||
from graph-dead.
|
||||
- **Suggested owner:** showcase-langgraph-python agent authoring /
|
||||
deploy. Parallel: demo authoring could add an
|
||||
`data-testid="assistant-message"` marker on the assistant bubble to
|
||||
give the spec a deterministic structural signal.
|
||||
- **Next step:** either fix the deployed agent's response latency or
|
||||
add the assistant-message testid so the spec can assert structural
|
||||
signal without waiting on LLM text.
|
||||
- **Descoped cell(s):** `readonly-state-agent-context` E2E (LLM
|
||||
round-trip assertions skipped).
|
||||
|
||||
### B8 — `open-gen-ui` iframe mount regularly exceeds 120s (W8-OGUI-1)
|
||||
|
||||
- **Symptom:** `/demos/open-gen-ui` iframe mount exceeds the 120s
|
||||
per-test budget because the LLM has to author full HTML/CSS/JS before
|
||||
the iframe can paint. No reliable post-mount signal.
|
||||
- **Evidence:** `showcase/packages/langgraph-python/tests/e2e/
|
||||
open-gen-ui.spec.ts` marks both the Quicksort suggestion path and the
|
||||
neural-network path `test.skip` with "See W8-OGUI-1" at
|
||||
`open-gen-ui.spec.ts:64,90`. Scratch file does not mention this entry
|
||||
— **scratch not updated**.
|
||||
- **Suspected cause:** demo is inherently LLM-authoring-bound. The
|
||||
iframe content is fully generated per request; there is no
|
||||
short-circuit signal (no testid on mount, iframe is srcdoc-loaded and
|
||||
opaque to the host).
|
||||
- **Suggested owner:** showcase-langgraph-python demo authoring.
|
||||
- **Next step:** emit a `data-testid="ogui-iframe"` on mount (short-
|
||||
circuits the LLM wait), or narrow the prompt to reduce authoring
|
||||
latency on Railway.
|
||||
- **Descoped cell(s):** `open-gen-ui` E2E (iframe-mount assertions
|
||||
skipped).
|
||||
|
||||
### B9 — `open-gen-ui-advanced` sandbox iframe round-trip unverifiable (W8-OGUI-2)
|
||||
|
||||
- **Symptom:** `/demos/open-gen-ui-advanced` mounts an
|
||||
`sandbox="allow-scripts"`-only iframe; the round-trip to the host
|
||||
(e.g. the `notifyHost` console log) cannot be asserted via
|
||||
Playwright's `contentFrame()` because `allow-scripts`-only iframes
|
||||
restrict cross-frame interaction.
|
||||
- **Evidence:** `showcase/packages/langgraph-python/tests/e2e/
|
||||
open-gen-ui-advanced.spec.ts` marks the Ping mount and the
|
||||
`notifyHost` round-trip `test.skip` with "See W8-OGUI-2" at
|
||||
`open-gen-ui-advanced.spec.ts:63,92`. Scratch file does not mention
|
||||
this entry — **scratch not updated**.
|
||||
- **Suspected cause:** shares B8's LLM-authoring latency; additionally
|
||||
the `allow-scripts` sandbox attribute by design prevents host-side
|
||||
introspection.
|
||||
- **Suggested owner:** showcase-langgraph-python demo authoring.
|
||||
- **Next step:** emit a post-mount testid or a host-visible console-log
|
||||
fixture the spec can assert against without crossing the sandbox
|
||||
boundary.
|
||||
- **Descoped cell(s):** `open-gen-ui-advanced` E2E (sandbox-attribute and
|
||||
round-trip assertions skipped).
|
||||
|
||||
### B10 — `declarative-gen-ui` `generate_a2ui` secondary LLM stalls for KPI/StatusReport prompts (W8-7)
|
||||
|
||||
- **Symptom:** `/demos/declarative-gen-ui` KPI-dashboard and
|
||||
StatusReport pill flows regularly exceed 60s on Railway when the
|
||||
secondary LLM stage (which authors the a2ui JSON) stalls.
|
||||
- **Evidence:** `showcase/packages/langgraph-python/tests/e2e/
|
||||
declarative-gen-ui.spec.ts` marks the KPI test and the StatusReport
|
||||
test `test.skip` with "See W8-7" at `declarative-gen-ui.spec.ts:118,140`.
|
||||
Scratch file does not mention this entry — **scratch not updated**.
|
||||
- **Suspected cause:** secondary LLM call in the `a2ui_dynamic` agent
|
||||
graph is slow/flaky on Railway. KPI is the slowest of the 4 pills.
|
||||
- **Suggested owner:** showcase-langgraph-python agent authoring.
|
||||
- **Next step:** measure secondary-LLM latency distribution on Railway;
|
||||
consider prompt shrinking or model swap for the secondary stage.
|
||||
- **Descoped cell(s):** `declarative-gen-ui` E2E (KPI + StatusReport
|
||||
flows skipped; ProductCard and VideoCard pills remain live).
|
||||
|
||||
### B11 — `a2ui-fixed-schema` `display_flight` secondary LLM occasionally stalls (W8-8)
|
||||
|
||||
- **Symptom:** `/demos/a2ui-fixed-schema` `display_flight` flow
|
||||
occasionally stalls the secondary LLM stage past its 60s render
|
||||
budget.
|
||||
- **Evidence:** `showcase/packages/langgraph-python/tests/e2e/
|
||||
a2ui-fixed-schema.spec.ts:31` — inline comment "W8-8: on Railway,
|
||||
`display_flight` occasionally stalls the secondary LLM stage; render
|
||||
budget is 60s." Spec still runs against the 60s budget — not skipped,
|
||||
but flaky. Scratch file does not mention this entry — **scratch not
|
||||
updated**.
|
||||
- **Suspected cause:** same secondary-LLM latency cluster as B10.
|
||||
- **Suggested owner:** showcase-langgraph-python agent authoring.
|
||||
- **Next step:** bundle with B10 investigation; possibly raise the
|
||||
render budget to 90s or switch the secondary stage model.
|
||||
- **Descoped cell(s):** none — test still runs; flake is documented, not
|
||||
skipped.
|
||||
|
||||
### B12 — `mcp-apps` Excalidraw MCP iframe fails to paint within 90s (W8-9)
|
||||
|
||||
- **Symptom:** The end-to-end MCP round-trip (agent → `create_view` →
|
||||
server-side resource fetch → activity event → iframe render) on
|
||||
`/demos/mcp-apps` regularly sits above 90s and intermittently fails
|
||||
to paint an iframe at all when the Excalidraw MCP server is slow.
|
||||
- **Evidence:** `showcase/packages/langgraph-python/tests/e2e/
|
||||
mcp-apps.spec.ts` marks the flowchart flow and the explicit
|
||||
`create_view`-prompt flow `test.skip` with "See W8-9" at
|
||||
`mcp-apps.spec.ts:60,80`. Scratch file does not mention this entry —
|
||||
**scratch not updated**.
|
||||
- **Suspected cause:** MCP Apps middleware latency or Excalidraw MCP
|
||||
upstream slowness.
|
||||
- **Suggested owner:** showcase-langgraph-python deploy + MCP
|
||||
infrastructure.
|
||||
- **Next step:** confirm whether the Excalidraw MCP server latency is
|
||||
the dominant factor; consider pre-warming or a cached-resource
|
||||
fallback.
|
||||
- **Descoped cell(s):** `mcp-apps` E2E (round-trip flows skipped;
|
||||
presence + sandbox-contract assertions live).
|
||||
|
||||
### B13 — `query_notes` occasionally does not fire without explicit keyword verb (W8-4)
|
||||
|
||||
- **Symptom:** `/demos/frontend-tools-async` `query_notes` tool fires
|
||||
reliably when the user prompt contains an explicit "search my notes"
|
||||
verb phrase, but the "Find project-planning notes" suggestion pill and
|
||||
the typed variant "Find my notes about project planning." occasionally
|
||||
do not trigger the tool within 45s — the agent answers in-context
|
||||
without firing.
|
||||
- **Evidence:** during e2e authoring, the pill-click variant and the
|
||||
typed-prompt variant both timed out waiting on
|
||||
`[data-testid="notes-card"]` at 45s. The "Search my notes for
|
||||
'auth'." typed variant and the zero-match "xyzzy-nonsense-keyword"
|
||||
variant succeeded reliably. Mitigation already landed in
|
||||
`showcase/packages/langgraph-python/tests/e2e/
|
||||
frontend-tools-async.spec.ts` — pill test substitutes an explicit
|
||||
typed "Search my notes for 'auth'." prompt; terminal assertion accepts
|
||||
either `notes-list` or the empty-state copy.
|
||||
- **Suspected cause:** `frontend_tools_async` graph's system prompt does
|
||||
not consistently bias the model towards `query_notes` for "find …
|
||||
notes" phrasing.
|
||||
- **Suggested owner:** showcase-langgraph-python agent authoring.
|
||||
- **Next step:** harden the system prompt to always prefer `query_notes`
|
||||
when the prompt contains "notes", or update the suggestion pill copy
|
||||
to begin with "Search my notes for …" verbatim.
|
||||
- **Descoped cell(s):** none — test still runs after the pill→typed
|
||||
substitution; flake is documented, not skipped.
|
||||
|
||||
### B14 — `chat-customization-css` theme.css loses cascade on Railway
|
||||
|
||||
- **Symptom:** On Railway the `chat-customization-css` demo
|
||||
intermittently loses the custom dashed-border and theme cascade — the
|
||||
`theme.css` overrides for `--copilot-kit-*` variables don't win over
|
||||
the default stylesheet load order.
|
||||
- **Evidence:** Memory-only from this session's dashboard walk (user
|
||||
note). Not captured in
|
||||
`tests/e2e/chat-customization-css.spec.ts` comments; the spec asserts
|
||||
`theme.css` CSS variables on the `.chat-css-demo-scope` wrapper but
|
||||
the reported Railway flake is about the dashed-border visual, not the
|
||||
computed variables. Scratch file does not mention this entry —
|
||||
**scratch not updated**.
|
||||
- **Suspected cause:** stylesheet load order on Railway's Next.js
|
||||
production build differs from local — `theme.css` is imported but not
|
||||
guaranteed to load after the default CopilotKit stylesheet under
|
||||
certain chunk-splitting conditions.
|
||||
- **Suggested owner:** showcase-langgraph-python demo authoring.
|
||||
- **Next step:** reproduce on Railway with a deterministic trigger;
|
||||
confirm import order in the production bundle; if needed, hoist
|
||||
`theme.css` import or add a `@layer` wrapper to force cascade.
|
||||
- **Descoped cell(s):** potentially `chat-customization-css` if the flake
|
||||
repros during Wave 1's final dashboard walk. Track but not
|
||||
pre-descoped.
|
||||
|
||||
### B15 — v2 `CopilotChatInput` Enter-key submit is flaky on slow networks
|
||||
|
||||
- **Symptom:** On slow networks the Enter-key submit path in v2
|
||||
`CopilotChatInput` intermittently drops the keystroke; tests using
|
||||
`page.keyboard.press("Enter")` after `fill()` flake. Workaround used
|
||||
across Wave 1 specs: click `[data-testid="copilot-send-button"]`
|
||||
instead.
|
||||
- **Evidence:** every Wave 1 spec
|
||||
(`showcase/packages/langgraph-python/tests/e2e/*.spec.ts`) uses the
|
||||
`[data-testid="copilot-send-button"]` locator rather than Enter. No
|
||||
dedicated comment in-spec explains why, but the workaround is
|
||||
uniform. Memory-only from this session. Scratch file does not mention
|
||||
this entry — **scratch not updated**.
|
||||
- **Suspected cause:** race between the controlled-input state update
|
||||
and the submit handler in v2 `CopilotChatInput` when Enter fires
|
||||
during an in-flight network tick.
|
||||
- **Suggested owner:** v2 chat-input component (packages/).
|
||||
- **Next step:** file an issue against the v2 chat-input package with a
|
||||
minimal repro; confirm whether the Enter handler awaits the latest
|
||||
controlled value.
|
||||
- **Descoped cell(s):** none — workaround is trivial.
|
||||
|
||||
### B16 — `agentic-chat` suite fails against Railway: `background-container` testid absent
|
||||
|
||||
- **Symptom:** The `agentic-chat.spec.ts` suite asserts
|
||||
`[data-testid="background-container"]`, but on the deployed Railway
|
||||
demo that testid is not emitted — the deployed demo has drifted from
|
||||
source.
|
||||
- **Evidence:** `showcase/packages/langgraph-python/tests/e2e/
|
||||
agentic-chat.spec.ts:13,20,89` all use
|
||||
`page.locator('[data-testid="background-container"]')`. The source
|
||||
under `src/app/demos/agentic-chat/page.tsx` does render the testid,
|
||||
but the Railway image appears to be from before a recent edit. Memory-
|
||||
only from this session. Scratch file does not mention this entry —
|
||||
**scratch not updated**.
|
||||
- **Suspected cause:** Railway build is stale relative to the source
|
||||
tree; redeploy needed, or the deployed branch diverges from the
|
||||
worktree.
|
||||
- **Suggested owner:** showcase-langgraph-python deploy.
|
||||
- **Next step:** redeploy Railway from current HEAD; re-run the
|
||||
`agentic-chat.spec.ts` suite and confirm all assertions pass.
|
||||
- **Descoped cell(s):** `agentic-chat` E2E remains pending a redeploy —
|
||||
track but not pre-descoped pending the Wave 1 post-merge dashboard
|
||||
walk.
|
||||
|
||||
### B17 — `chat-slots` manifest `highlight` list omits two components
|
||||
|
||||
- **Symptom:** `packages/langgraph-python/manifest.yaml` `chat-slots`
|
||||
entry lists only `custom-welcome-screen.tsx` under `highlight:`. The
|
||||
demo actually uses three custom slot components:
|
||||
`custom-assistant-message.tsx` and `custom-disclaimer.tsx` are missing
|
||||
from the highlight list.
|
||||
- **Evidence:**
|
||||
- `showcase/packages/langgraph-python/manifest.yaml:268-276`
|
||||
(`chat-slots` entry highlight list).
|
||||
- `showcase/packages/langgraph-python/src/app/demos/chat-slots/`
|
||||
contains `custom-assistant-message.tsx`, `custom-disclaimer.tsx`,
|
||||
`custom-welcome-screen.tsx`, and `page.tsx`.
|
||||
- Does not affect the dashboard (highlight list is not dashboard-
|
||||
consumed for this column). Minor hygiene only.
|
||||
- **Suspected cause:** original manifest author added the first slot
|
||||
component and later additions were not back-filled.
|
||||
- **Suggested owner:** showcase-langgraph-python demo authoring.
|
||||
- **Next step:** add the two missing files to the `highlight:` array.
|
||||
- **Descoped cell(s):** none.
|
||||
|
||||
## Summary
|
||||
|
||||
- **Total W8 / Wave 1 bug entries:** 17 (B1–B17).
|
||||
- **Descoped cells from Wave 1 completeness:** 7 —
|
||||
`chat-customization-css` (docs-og, via B3),
|
||||
`agentic-chat-reasoning` (E2E, via B4),
|
||||
`hitl-in-app` (E2E, via B5),
|
||||
`gen-ui-interrupt` (E2E, via B6),
|
||||
`readonly-state-agent-context` (E2E, via B7),
|
||||
`open-gen-ui` (E2E, via B8),
|
||||
`open-gen-ui-advanced` (E2E, via B9),
|
||||
plus partial descoping of `declarative-gen-ui` E2E (2 of 4 pills, via
|
||||
B10) and `mcp-apps` E2E (round-trip flows only, via B12).
|
||||
- **Follow-up-only (no cell impact):** 8 — B1, B2, B11, B13, B14, B15,
|
||||
B16, B17.
|
||||
|
||||
Entries B7–B12 and B14–B17 were captured in-code (Playwright spec
|
||||
comments, manifest, and session memory) but were not synced back to
|
||||
`docs/superpowers/plans/langgraph-python-column-wave1-bugs-scratch.md`
|
||||
during Wave 1. The scratch file currently covers only W8-1, W8-2, W8-3
|
||||
(docs), W8-3 (E2E), W8-4, W8-5, and W8-6.
|
||||
Reference in New Issue
Block a user