chore: import upstream snapshot with attribution
Publish CLI Package / publish-npm (push) Waiting to run
Publish Python SDK / publish-pypi (push) Waiting to run
Publish TypeScript SDK / publish-npm (push) Waiting to run
CI / Migrate Dev DB (push) Has been skipped
CI / Detect Version (push) Has been cancelled
CI / Migrate DB (push) Has been cancelled
CI / Build Dev ECR (./docker/app.Dockerfile, ECR_APP) (push) Has been cancelled
CI / Build Dev ECR (./docker/db.Dockerfile, ECR_MIGRATIONS) (push) Has been cancelled
CI / Build Dev ECR (./docker/pii.Dockerfile, ECR_PII) (push) Has been cancelled
CI / Build Dev ECR (./docker/realtime.Dockerfile, ECR_REALTIME) (push) Has been cancelled
CI / Deploy Trigger.dev (Dev) (push) Has been cancelled
CI / Build AMD64 (./docker/app.Dockerfile, ECR_APP, ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Build AMD64 (./docker/db.Dockerfile, ECR_MIGRATIONS, ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Build AMD64 (./docker/pii.Dockerfile, ECR_PII, ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Build AMD64 (./docker/realtime.Dockerfile, ECR_REALTIME, ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/app.Dockerfile, ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/db.Dockerfile, ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/pii.Dockerfile, ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/realtime.Dockerfile, ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Check Docs Changes (push) Has been cancelled
CI / Process Docs (push) Has been cancelled
CI / Create GitHub Release (push) Has been cancelled
CI / Test and Build (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:20:55 +08:00
commit d25d482dc2
13754 changed files with 4996608 additions and 0 deletions
+60
View File
@@ -0,0 +1,60 @@
---
description: Gate a block's visibility — ship an unreleased block as a preview (hidden until revealed via AppConfig/env), reveal it to admins/orgs, GA it, or kill-switch a shipped block
argument-hint: <block-type>
---
# Add Block Preview Skill
You manage **block visibility gating** in Sim — hiding blocks from every discovery surface (toolbar, cmd+K search, copilot @-mentions, agent tool picker, mothership VFS/metadata/tools, Access Control list, public docs/catalog) while **never** gating execution of already-placed instances.
## The model
Three levers, evaluated in `apps/sim/lib/core/config/block-visibility.ts` and folded into the registry accessors (`apps/sim/blocks/registry.ts`):
1. **`preview: true`** on the `BlockConfig` (static, in code) — the block is default-hidden EVERYWHERE (hosted, self-hosted, dev, SSR) until revealed. Fail-closed.
2. **The hosted `block-visibility` AppConfig document** — per-block rule keyed by the existing block type:
```jsonc
{
"<block-type>": {
"enabled": false, // required. true = GA (visible to everyone)
"orgIds": ["org_..."], // optional allowlist clauses (any match reveals)
"userIds": ["user_..."],
"adminEnabled": true // platform admins (user.role === 'admin')
}
}
```
3. **`PREVIEW_BLOCKS` env** (comma-separated block types) — the off-AppConfig reveal path for self-hosters and local dev.
A revealed block that is not globally GA (`enabled !== true`, or env-revealed) renders with a **" (Preview)"** name suffix on discovery surfaces. `getBlock()` stays pure, so placed instances keep their canonical name and always execute.
## Lifecycle of a preview block
1. **Author** the block normally (`/add-block` etc.) and set `preview: true` on its `BlockConfig`. **Ship no `BlockMeta` and no docs until GA** — `check-block-registry` deliberately skips preview blocks in meta coverage, and `generate-docs` skips them at every gate.
2. **Local dev:** set `PREVIEW_BLOCKS=<block-type>` in your env to see it (with the suffix).
3. **Merge/deploy.** The block's code is live everywhere but visible nowhere — no AppConfig rule exists and self-hosters have no env entry.
4. **Hosted preview:** add a rule to the `block-visibility` AppConfig document and start a deployment (no code deploy):
- Admins only: `{ "enabled": false, "adminEnabled": true }`
- Design-partner org: `{ "enabled": false, "orgIds": ["org_123"] }`
- GA via config (code cleanup pending): `{ "enabled": true }` — suffix disappears everywhere within ~30s (AppConfig TTL) + client refetch.
Same runbook as `feature-flags`: edit the hosted document, `aws appconfig start-deployment` with the `sim-<env>-fast` strategy (see the infra README).
5. **GA cleanup:** delete `preview: true` from the block (now visible to self-hosters on their next upgrade), add its `BlockMeta` + regen docs, and drop the AppConfig entry. For a v2 upgrade, this is also when v1 gets `hideFromToolbar: true` (the superseded-version paradigm).
## Kill switch (shipped blocks)
To pull an already-GA block from discovery surfaces on hosted (incident, deprecation): add `{ "<block-type>": { "enabled": false } }` to the document. Allowlist clauses can carve out exceptions. **Execution is NOT stopped** — workflows already using the block keep running; the kill switch only prevents new placement/discovery.
## Invariants (do not violate)
- **Execution is never gated.** The executor, serializer, drop-naming, and `isBlockTypeAccessControlExempt` resolve via pure `getBlock`. Do not add visibility checks to execution paths.
- **Clone-not-remove:** gated blocks stay in `getAllBlocks()` output as clones with `hideFromToolbar: true` — `.find`-by-type consumers rely on this. Never filter them out.
- **Keys are registry block types.** Never `custom_block_*` (parse drops them — custom blocks have their own enabled/disabled lifecycle).
- **The shared hidden-predicate is `isHiddenUnder`** (`apps/sim/blocks/visibility/context.ts`). Never restate the preview/disabled rule inline at a new consumer.
- **Process-global caches stay ungated.** `getStaticComponentFiles` (VFS) and `getExposedIntegrationTools` build the ungated universe; per-viewer filtering happens at stamp/consumer time. Never move gating into a shared builder.
- Gating is **surface hiding, not secrecy** — the full config ships in the client JS bundle. Anything truly secret cannot be a registered block.
## Tests
Evaluation semantics: `apps/sim/lib/core/config/block-visibility.test.ts`. Registry projection: `apps/sim/blocks/visibility/visibility.test.ts`. When gating behavior changes, extend those — mock `isPlatformAdmin` for the admin clause; use the local `withAppConfig` harness.
+874
View File
@@ -0,0 +1,874 @@
---
description: Create a block configuration for a Sim integration with proper subBlocks, conditions, and tool wiring
argument-hint: <service-name>
---
# Add Block Skill
You are an expert at creating block configurations for Sim. You understand the serializer, subBlock types, conditions, dependsOn, modes, and all UI patterns.
## Your Task
When the user asks you to create a block:
1. Create the block file in `apps/sim/blocks/blocks/{service}.ts`
2. Configure all subBlocks with proper types, conditions, and dependencies
3. Wire up tools correctly
## Block Configuration Structure
```typescript
import { {ServiceName}Icon } from '@/components/icons'
import type { BlockConfig } from '@/blocks/types'
import { AuthMode, IntegrationType } from '@/blocks/types'
import { getScopesForService } from '@/lib/oauth/utils'
export const {ServiceName}Block: BlockConfig = {
type: '{service}', // snake_case identifier
name: '{Service Name}', // Human readable
description: 'Brief description', // One sentence
longDescription: 'Detailed description for docs',
docsLink: 'https://docs.sim.ai/integrations/{service}',
category: 'tools', // 'tools' | 'blocks' | 'triggers'
integrationType: IntegrationType.X, // Primary category (see IntegrationType enum)
tags: ['oauth', 'api'], // Cross-cutting tags (see IntegrationTag type)
bgColor: '#HEXCOLOR', // Brand color
icon: {ServiceName}Icon,
// Auth mode
authMode: AuthMode.OAuth, // or AuthMode.ApiKey
subBlocks: [
// Define all UI fields here
],
tools: {
access: ['tool_id_1', 'tool_id_2'], // Array of tool IDs this block can use
config: {
tool: (params) => `{service}_${params.operation}`, // Tool selector function
params: (params) => ({
// Transform subBlock values to tool params
}),
},
},
inputs: {
// Optional: define expected inputs from other blocks
},
outputs: {
// Define outputs available to downstream blocks
},
}
```
## SubBlock Types Reference
**Critical:** Every subblock `id` must be unique within the block. Duplicate IDs cause conflicts even with different conditions.
### Text Inputs
```typescript
// Single-line input
{ id: 'field', title: 'Label', type: 'short-input', placeholder: '...' }
// Multi-line input
{ id: 'field', title: 'Label', type: 'long-input', placeholder: '...', rows: 6 }
// Password input
{ id: 'apiKey', title: 'API Key', type: 'short-input', password: true }
```
### Selection Inputs
```typescript
// Dropdown (static options)
{
id: 'operation',
title: 'Operation',
type: 'dropdown',
options: [
{ label: 'Create', id: 'create' },
{ label: 'Update', id: 'update' },
],
value: () => 'create', // Default value function
}
// Combobox (searchable dropdown)
{
id: 'field',
title: 'Label',
type: 'combobox',
options: [...],
searchable: true,
}
```
### Code/JSON Inputs
```typescript
{
id: 'code',
title: 'Code',
type: 'code',
language: 'javascript', // 'javascript' | 'json' | 'python'
placeholder: '// Enter code...',
}
```
### OAuth/Credentials
```typescript
{
id: 'credential',
title: 'Account',
type: 'oauth-input',
serviceId: '{service}', // Must match OAuth provider service key
requiredScopes: getScopesForService('{service}'), // Import from @/lib/oauth/utils
placeholder: 'Select account',
required: true,
}
```
**Scopes:** Always use `getScopesForService(serviceId)` from `@/lib/oauth/utils` for `requiredScopes`. Never hardcode scope arrays — the single source of truth is `OAUTH_PROVIDERS` in `lib/oauth/oauth.ts`.
**Scope descriptions:** When adding a new OAuth provider, also add human-readable descriptions for all scopes in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`.
### Selectors (with dynamic options)
```typescript
// Channel selector (Slack, Discord, etc.)
{
id: 'channel',
title: 'Channel',
type: 'channel-selector',
serviceId: '{service}',
placeholder: 'Select channel',
dependsOn: ['credential'],
}
// Project selector (Jira, etc.)
{
id: 'project',
title: 'Project',
type: 'project-selector',
serviceId: '{service}',
dependsOn: ['credential'],
}
// File selector (Google Drive, etc.)
{
id: 'file',
title: 'File',
type: 'file-selector',
serviceId: '{service}',
mimeType: 'application/pdf',
dependsOn: ['credential'],
}
// User selector
{
id: 'user',
title: 'User',
type: 'user-selector',
serviceId: '{service}',
dependsOn: ['credential'],
}
```
### Other Types
```typescript
// Switch/toggle
{ id: 'enabled', type: 'switch' }
// Slider
{ id: 'temperature', title: 'Temperature', type: 'slider', min: 0, max: 2, step: 0.1 }
// Table (key-value pairs)
{ id: 'headers', title: 'Headers', type: 'table', columns: ['Key', 'Value'] }
// File upload
{
id: 'files',
title: 'Attachments',
type: 'file-upload',
multiple: true,
acceptedTypes: 'image/*,application/pdf',
}
```
## File Input Handling
When your block accepts file uploads, use the basic/advanced mode pattern with `normalizeFileInput`.
### Basic/Advanced File Pattern
```typescript
// Basic mode: Visual file upload
{
id: 'uploadFile',
title: 'File',
type: 'file-upload',
canonicalParamId: 'file', // Both map to 'file' param
placeholder: 'Upload file',
mode: 'basic',
multiple: false,
required: true,
condition: { field: 'operation', value: 'upload' },
},
// Advanced mode: Reference from other blocks
{
id: 'fileRef',
title: 'File',
type: 'short-input',
canonicalParamId: 'file', // Both map to 'file' param
placeholder: 'Reference file (e.g., {{file_block.output}})',
mode: 'advanced',
required: true,
condition: { field: 'operation', value: 'upload' },
},
```
**Critical constraints:**
- `canonicalParamId` must NOT match any subblock's `id` in the same block
- Values are stored under subblock `id`, not `canonicalParamId`
### Normalizing File Input in tools.config
Use `normalizeFileInput` to handle all input variants:
```typescript
import { normalizeFileInput } from '@/blocks/utils'
tools: {
access: ['service_upload'],
config: {
tool: (params) => {
// Check all field IDs: uploadFile (basic), fileRef (advanced), fileContent (legacy)
const normalizedFile = normalizeFileInput(
params.uploadFile || params.fileRef || params.fileContent,
{ single: true }
)
if (normalizedFile) {
params.file = normalizedFile
}
return `service_${params.operation}`
},
},
}
```
**Why this pattern?**
- Values come through as `params.uploadFile` or `params.fileRef` (the subblock IDs)
- `canonicalParamId` only controls UI/schema mapping, not runtime values
- `normalizeFileInput` handles JSON strings from advanced mode template resolution
### File Input Types in `inputs`
Use `type: 'json'` for file inputs:
```typescript
inputs: {
uploadFile: { type: 'json', description: 'Uploaded file (UserFile)' },
fileRef: { type: 'json', description: 'File reference from previous block' },
// Legacy field for backwards compatibility
fileContent: { type: 'string', description: 'Legacy: base64 encoded content' },
}
```
### Multiple Files
For multiple file uploads:
```typescript
{
id: 'attachments',
title: 'Attachments',
type: 'file-upload',
multiple: true, // Allow multiple files
maxSize: 25, // Max size in MB per file
acceptedTypes: 'image/*,application/pdf,.doc,.docx',
}
// In tools.config:
const normalizedFiles = normalizeFileInput(
params.attachments || params.attachmentRefs,
// No { single: true } - returns array
)
if (normalizedFiles) {
params.files = normalizedFiles
}
```
## Condition Syntax
Controls when a field is shown based on other field values.
### Simple Condition
```typescript
condition: { field: 'operation', value: 'create' }
// Shows when operation === 'create'
```
### Multiple Values (OR)
```typescript
condition: { field: 'operation', value: ['create', 'update'] }
// Shows when operation is 'create' OR 'update'
```
### Negation
```typescript
condition: { field: 'operation', value: 'delete', not: true }
// Shows when operation !== 'delete'
```
### Compound (AND)
```typescript
condition: {
field: 'operation',
value: 'send',
and: {
field: 'type',
value: 'dm',
not: true,
}
}
// Shows when operation === 'send' AND type !== 'dm'
```
### Complex Example
```typescript
condition: {
field: 'operation',
value: ['list', 'search'],
not: true,
and: {
field: 'authMethod',
value: 'oauth',
}
}
// Shows when operation NOT in ['list', 'search'] AND authMethod === 'oauth'
```
## DependsOn Pattern
Controls when a field is enabled and when its options are refetched.
### Simple Array (all must be set)
```typescript
dependsOn: ['credential']
// Enabled only when credential has a value
// Options refetch when credential changes
dependsOn: ['credential', 'projectId']
// Enabled only when BOTH have values
```
### Complex (all + any)
```typescript
dependsOn: {
all: ['authMethod'], // All must be set
any: ['credential', 'apiKey'] // At least one must be set
}
// Enabled when authMethod is set AND (credential OR apiKey is set)
```
## Required Pattern
Can be boolean or condition-based.
### Simple Boolean
```typescript
required: true
required: false
```
### Conditional Required
```typescript
required: { field: 'operation', value: 'create' }
// Required only when operation === 'create'
required: { field: 'operation', value: ['create', 'update'] }
// Required when operation is 'create' OR 'update'
```
## Mode Pattern (Basic vs Advanced)
Controls which UI view shows the field.
### Mode Options
- `'basic'` - Only in basic view (default UI)
- `'advanced'` - Only in advanced view
- `'both'` - Both views (default if not specified)
- `'trigger'` - Only in trigger configuration
### canonicalParamId Pattern
Maps multiple UI fields to a single serialized parameter:
```typescript
// Basic mode: Visual selector
{
id: 'channel',
title: 'Channel',
type: 'channel-selector',
mode: 'basic',
canonicalParamId: 'channel', // Both map to 'channel' param
dependsOn: ['credential'],
}
// Advanced mode: Manual input
{
id: 'channelId',
title: 'Channel ID',
type: 'short-input',
mode: 'advanced',
canonicalParamId: 'channel', // Both map to 'channel' param
placeholder: 'Enter channel ID manually',
}
```
**How it works:**
- In basic mode: `channel` selector value → `params.channel`
- In advanced mode: `channelId` input value → `params.channel`
- The serializer consolidates based on current mode
**Critical constraints:**
- `canonicalParamId` must NOT match any other subblock's `id` in the same block (causes conflicts)
- A `canonicalParamId` links exactly one basic/advanced pair for a single logical parameter. Do NOT reuse the same `canonicalParamId` for different parameters, even under mutually-exclusive conditions/operations
- ONLY use `canonicalParamId` to link basic/advanced mode alternatives for the same logical parameter
- Do NOT use it for any other purpose
## WandConfig Pattern
Enables AI-assisted field generation.
```typescript
{
id: 'query',
title: 'Query',
type: 'code',
language: 'json',
wandConfig: {
enabled: true,
prompt: 'Generate a query based on the user request. Return ONLY the JSON.',
placeholder: 'Describe what you want to query...',
generationType: 'json-object', // Optional: affects AI behavior
maintainHistory: true, // Optional: keeps conversation context
},
}
```
### Generation Types
- `'javascript-function-body'` - JS code generation
- `'json-object'` - Raw JSON (adds "no markdown" instruction)
- `'json-schema'` - JSON Schema definitions
- `'sql-query'` - SQL statements
- `'timestamp'` - Adds current date/time context
## Tools Configuration
**Important:** `tools.config.tool` runs during serialization before variable resolution. Put `Number()` and other type coercions in `tools.config.params` instead, which runs at execution time after variables are resolved.
**Preferred:** Use tool names directly as dropdown option IDs to avoid switch cases:
```typescript
// Dropdown options use tool IDs directly
options: [
{ label: 'Create', id: 'service_create' },
{ label: 'Read', id: 'service_read' },
]
// Tool selector just returns the operation value
tool: (params) => params.operation,
```
### With Parameter Transformation
```typescript
tools: {
access: ['service_action'],
config: {
tool: (params) => 'service_action',
params: (params) => ({
id: params.resourceId,
data: typeof params.data === 'string' ? JSON.parse(params.data) : params.data,
}),
},
}
```
### V2 Versioned Tool Selector
```typescript
import { createVersionedToolSelector } from '@/blocks/utils'
tools: {
access: [
'service_create_v2',
'service_read_v2',
'service_update_v2',
],
config: {
tool: createVersionedToolSelector({
baseToolSelector: (params) => `service_${params.operation}`,
suffix: '_v2',
fallbackToolId: 'service_create_v2',
}),
},
}
```
## Outputs Definition
**IMPORTANT:** Block outputs have a simpler schema than tool outputs. Block outputs do NOT support:
- `optional: true` - This is only for tool outputs
- `items` property - This is only for tool outputs with array types
Block outputs only support:
- `type` - The data type ('string', 'number', 'boolean', 'json', 'array')
- `description` - Human readable description
- `condition` - Optional visibility condition
- `hiddenFromDisplay` - Optional flag to hide from the output display
**Nested object/`properties` outputs are tool-output-only and will fail TypeScript at build time on block outputs.** For complex shapes use `type: 'json'` and describe the inner fields in the `description` string.
```typescript
outputs: {
// Simple outputs
id: { type: 'string', description: 'Resource ID' },
success: { type: 'boolean', description: 'Whether operation succeeded' },
// Use type: 'json' for complex objects or arrays (NOT type: 'array' with items)
items: { type: 'json', description: 'List of items' },
metadata: { type: 'json', description: 'Response metadata' },
// Nested outputs (for structured data)
user: {
id: { type: 'string', description: 'User ID' },
name: { type: 'string', description: 'User name' },
email: { type: 'string', description: 'User email' },
},
}
```
### Typed JSON Outputs
When using `type: 'json'` and you know the object shape in advance, **describe the inner fields in the description** so downstream blocks know what properties are available. Block outputs have no nested `properties` form — always keep the output flat and put the shape in the `description`:
```typescript
outputs: {
// BAD: Opaque json with no info about what's inside
plan: { type: 'json', description: 'Zone plan information' },
// GOOD: Describe the known fields in the description
plan: {
type: 'json',
description: 'Zone plan information (id, name, price, currency, frequency, is_subscribed)',
},
}
```
Nested object outputs (`plan: { id: { type: 'string' }, ... }`) are a **tool-output** feature only — `OutputFieldDefinition` for blocks does not allow them and they fail TypeScript at build time.
## V2 Block Pattern
When creating V2 blocks (alongside legacy V1):
```typescript
// V1 Block - mark as legacy
export const ServiceBlock: BlockConfig = {
type: 'service',
name: 'Service (Legacy)',
hideFromToolbar: true, // Hide from toolbar
// ... rest of config
}
// V2 Block - visible, uses V2 tools
export const ServiceV2Block: BlockConfig = {
type: 'service_v2',
name: 'Service', // Clean name
hideFromToolbar: false, // Visible
subBlocks: ServiceBlock.subBlocks, // Reuse UI
tools: {
access: ServiceBlock.tools?.access?.map(id => `${id}_v2`) || [],
config: {
tool: createVersionedToolSelector({
baseToolSelector: (params) => (ServiceBlock.tools?.config as any)?.tool(params),
suffix: '_v2',
fallbackToolId: 'service_default_v2',
}),
params: ServiceBlock.tools?.config?.params,
},
},
outputs: {
// Flat, API-aligned outputs (not wrapped in content/metadata)
},
}
```
## Registering Blocks
After creating the block, remind the user to register it in `apps/sim/blocks/registry-maps.ts` (the data maps live here; `registry.ts` holds only the accessor functions). Add the import and an entry to each map alphabetically:
```typescript
import { ServiceBlock, ServiceBlockMeta } from '@/blocks/blocks/service'
export const BLOCK_REGISTRY: Record<string, BlockConfig> = {
// ... existing blocks ...
service: ServiceBlock,
}
export const BLOCK_META_REGISTRY: Record<string, BlockMeta> = {
// ... existing metas ...
service: ServiceBlockMeta,
}
```
## Complete Example
```typescript
import { ServiceIcon } from '@/components/icons'
import type { BlockConfig } from '@/blocks/types'
import { AuthMode, IntegrationType } from '@/blocks/types'
import { getScopesForService } from '@/lib/oauth/utils'
export const ServiceBlock: BlockConfig = {
type: 'service',
name: 'Service',
description: 'Integrate with Service API',
longDescription: 'Full description for documentation...',
docsLink: 'https://docs.sim.ai/integrations/service',
category: 'tools',
integrationType: IntegrationType.DeveloperTools,
tags: ['oauth', 'api'],
bgColor: '#FF6B6B',
icon: ServiceIcon,
authMode: AuthMode.OAuth,
subBlocks: [
{
id: 'operation',
title: 'Operation',
type: 'dropdown',
options: [
{ label: 'Create', id: 'create' },
{ label: 'Read', id: 'read' },
{ label: 'Update', id: 'update' },
{ label: 'Delete', id: 'delete' },
],
value: () => 'create',
},
{
id: 'credential',
title: 'Service Account',
type: 'oauth-input',
serviceId: 'service',
requiredScopes: getScopesForService('service'),
placeholder: 'Select account',
required: true,
},
{
id: 'resourceId',
title: 'Resource ID',
type: 'short-input',
placeholder: 'Enter resource ID',
condition: { field: 'operation', value: ['read', 'update', 'delete'] },
required: { field: 'operation', value: ['read', 'update', 'delete'] },
},
{
id: 'name',
title: 'Name',
type: 'short-input',
placeholder: 'Resource name',
condition: { field: 'operation', value: ['create', 'update'] },
required: { field: 'operation', value: 'create' },
},
],
tools: {
access: ['service_create', 'service_read', 'service_update', 'service_delete'],
config: {
tool: (params) => `service_${params.operation}`,
},
},
outputs: {
id: { type: 'string', description: 'Resource ID' },
name: { type: 'string', description: 'Resource name' },
createdAt: { type: 'string', description: 'Creation timestamp' },
},
}
```
## Connecting Blocks with Triggers
If the service supports webhooks, connect the block to its triggers.
```typescript
import { getTrigger } from '@/triggers'
export const ServiceBlock: BlockConfig = {
// ... basic config ...
triggers: {
enabled: true,
available: ['service_event_a', 'service_event_b', 'service_webhook'],
},
subBlocks: [
// Tool subBlocks first...
{ id: 'operation', /* ... */ },
// Then spread trigger subBlocks
...getTrigger('service_event_a').subBlocks,
...getTrigger('service_event_b').subBlocks,
...getTrigger('service_webhook').subBlocks,
],
}
```
See the `/add-trigger` skill for creating triggers.
## Icon Requirement
If the icon doesn't already exist in `@/components/icons.tsx`, **do NOT search for it yourself**. After completing the block, ask the user to provide the SVG:
```
The block is complete, but I need an icon for {Service}.
Please provide the SVG and I'll convert it to a React component.
You can usually find this in the service's brand/press kit page, or copy it from their website.
```
When converting the SVG: a **monochrome** logo (single white or black mark) must
use `fill='currentColor'`, never a hardcoded `#fff`/`#000000`. Block icons render
both inside their `bgColor` tile and "bare" on a neutral page (the home Suggested
actions list) in light and dark mode; a hardcoded white/black mark goes invisible
bare on the matching background. Multi-color brand logos keep their own fills.
Verify with `bun run check:bare-icons`.
## Advanced Mode for Optional Fields
Optional fields that are rarely used should be set to `mode: 'advanced'` so they don't clutter the basic UI. This includes:
- Pagination tokens
- Time range filters (start/end time)
- Sort order options
- Reply settings
- Rarely used IDs (e.g., reply-to tweet ID, quote tweet ID)
- Max results / limits
```typescript
{
id: 'startTime',
title: 'Start Time',
type: 'short-input',
placeholder: 'ISO 8601 timestamp',
condition: { field: 'operation', value: ['search', 'list'] },
mode: 'advanced', // Rarely used, hide from basic view
}
```
## WandConfig for Complex Inputs
Use `wandConfig` for fields that are hard to fill out manually, such as timestamps, comma-separated lists, and complex query strings. This gives users an AI-assisted input experience.
```typescript
// Timestamps - use generationType: 'timestamp' to inject current date context
{
id: 'startTime',
title: 'Start Time',
type: 'short-input',
mode: 'advanced',
wandConfig: {
enabled: true,
prompt: 'Generate an ISO 8601 timestamp based on the user description. Return ONLY the timestamp string.',
generationType: 'timestamp',
},
}
// Comma-separated lists - simple prompt without generationType
{
id: 'mediaIds',
title: 'Media IDs',
type: 'short-input',
mode: 'advanced',
wandConfig: {
enabled: true,
prompt: 'Generate a comma-separated list of media IDs. Return ONLY the comma-separated values.',
},
}
```
## Naming Convention
All tool IDs referenced in `tools.access` and returned by `tools.config.tool` MUST use `snake_case` (e.g., `x_create_tweet`, `slack_send_message`). Never use camelCase or PascalCase.
## BlockMeta (Required)
Every block file must export a `{Service}BlockMeta` alongside the block — **minimum 7 templates**. Look at existing examples in `apps/sim/blocks/blocks/` (e.g. `browser_use.ts`, `google_sheets.ts`) for the pattern.
```typescript
import type { BlockMeta } from '@/blocks/types'
export const {Service}BlockMeta = {
tags: ['tag1', 'tag2'], // IntegrationTag[]
url: 'https://{service}.com', // external service homepage (verify it resolves) — NOT docs.sim.ai
templates: [
{
icon: {Service}Icon,
title: '{Service} <use-case>', // 25 words
prompt: 'Build a workflow that...', // specific use case, 13 sentences
modules: ['agent', 'workflows'], // 'agent' | 'workflows' | 'tables' | 'files' | 'scheduled' | 'knowledge-base'
category: 'operations', // 'operations' | 'marketing' | 'sales' | 'engineering' | 'productivity' | 'support' | 'popular'
tags: ['automation'],
alsoIntegrations: ['slack'], // optional — other block IDs referenced in the prompt
featured: true, // optional
},
// ... at least 6 more
],
skills: [ // SuggestedSkill[] — 35 mainstream, 23 niche
{
name: 'summarize-thread', // kebab-case, ≤64 chars, unique, verb-led
description: 'One line: what it does and when to use it.', // ≤1024 chars
content:
'# Summarize Thread\n\n...\n\n## Steps\n1. ...\n\n## Output\n...', // markdown
},
// ... more
],
} as const satisfies BlockMeta
```
Derive templates from the service's real use cases. Each prompt should name a concrete trigger, transformation, and output — not a generic description of what the service does.
`skills` are curated, ready-to-add agent skills shown on the integration's detail page (users click **Add** to create them in their workspace). Two hard rules:
- **Ground every skill in operations the block actually exposes** — cross-check each skill's steps against `tools.access`. Never describe an action the integration cannot perform.
- **Derive skills from real, popular use cases found online — never invent them.** Web-search the service's documented use cases (vendor use-case/solutions pages, official docs describing the workflow, reputable "top automations for X" articles) and only add a skill you can source as something people genuinely do with the service. Do not hallucinate skills.
## Checklist Before Finishing
- [ ] `integrationType` is set to the correct `IntegrationType` enum value
- [ ] `tags` array includes all applicable `IntegrationTag` values
- [ ] All subBlocks have `id`, `title` (except switch), and `type`
- [ ] Conditions use correct syntax (field, value, not, and)
- [ ] DependsOn set for fields that need other values
- [ ] Required fields marked correctly (boolean or condition)
- [ ] OAuth inputs have correct `serviceId` and `requiredScopes: getScopesForService(serviceId)`
- [ ] Scope descriptions added to `SCOPE_DESCRIPTIONS` in `lib/oauth/utils.ts` for any new scopes
- [ ] Tools.access lists all tool IDs (snake_case)
- [ ] Tools.config.tool returns correct tool ID (snake_case)
- [ ] Outputs match tool outputs
- [ ] Block + meta registered in registry-maps.ts (`BLOCK_REGISTRY` / `BLOCK_META_REGISTRY`)
- [ ] If icon missing: asked user to provide SVG
- [ ] If triggers exist: `triggers` config set, trigger subBlocks spread
- [ ] Optional/rarely-used fields set to `mode: 'advanced'`
- [ ] Timestamps and complex inputs have `wandConfig` enabled
- [ ] Exported `{Service}BlockMeta` with at least 7 templates
- [ ] `url` set on `{Service}BlockMeta` to the external service's verified homepage (omit only for first-party blocks with no external service)
- [ ] `skills` added to `{Service}BlockMeta`, each grounded in `tools.access` and sourced from a real online use case (not invented)
## Final Validation (Required)
After creating the block, you MUST validate it against every tool it references:
1. **Read every tool definition** that appears in `tools.access` — do not skip any
2. **For each tool, verify the block has correct:**
- SubBlock inputs that cover all required tool params (with correct `condition` to show for that operation)
- SubBlock input types that match the tool param types (e.g., dropdown for enums, short-input for strings)
- `tools.config.params` correctly maps subBlock IDs to tool param names (if they differ)
- Type coercions in `tools.config.params` for any params that need conversion (Number(), Boolean(), JSON.parse())
3. **Verify block outputs** cover the key fields returned by all tools
4. **Verify conditions** — each subBlock should only show for the operations that actually use it
5. **Verify `{Service}BlockMeta` is exported** with at least 7 templates, each having `icon`, `title`, `prompt`, `modules`, `category`, and `tags`
+608
View File
@@ -0,0 +1,608 @@
---
description: Add a knowledge base connector for syncing documents from an external source
argument-hint: <service-name> [api-docs-url]
---
# Add Connector Skill
You are an expert at adding knowledge base connectors to Sim. A connector syncs documents from an external source (Confluence, Google Drive, Notion, etc.) into a knowledge base.
## Your Task
When the user asks you to create a connector:
1. Use Context7 or WebFetch to read the service's API documentation
2. Determine the auth mode: **OAuth** (if Sim already has an OAuth provider for the service) or **API key** (if the service uses API key / Bearer token auth)
3. Create the connector directory: a client-safe `meta.ts` (declarative metadata) plus the runtime module that spreads it
4. Register it in BOTH the server registry and the client-safe meta registry
## Directory Structure
Each connector is split into a client-safe metadata file and a server-only runtime file. This mirrors the `XBlockMeta` / `BLOCK_META_REGISTRY` split in `apps/sim/blocks` — client components (the knowledge UI) only need the metadata (icon, name, auth, config fields), so the runtime functions (which pull server-only helpers like `input-validation.server``undici``node:net`) must stay out of the client bundle.
Create files in `apps/sim/connectors/{service}/`:
```
connectors/{service}/
├── index.ts # Barrel export (re-exports the runtime connector)
├── meta.ts # ConnectorMeta — client-safe declarative metadata
└── {service}.ts # ConnectorConfig — spreads the meta + adds runtime functions
```
- `meta.ts` exports `{service}ConnectorMeta: ConnectorMeta`. It imports ONLY the icon from `@/components/icons`, `import type { ConnectorMeta } from '@/connectors/types'`, and any pure-data constants. It must NEVER import server/runtime code.
- `{service}.ts` exports `{service}Connector: ConnectorConfig`. It imports the meta via `import { {service}ConnectorMeta } from '@/connectors/{service}/meta'`, spreads it as the first property, and holds the runtime functions (which may import server-only helpers like `@/lib/knowledge/documents/utils`).
## Authentication
Connectors use a discriminated union for auth config (`ConnectorAuthConfig` in `connectors/types.ts`):
```typescript
type ConnectorAuthConfig =
| { mode: 'oauth'; provider: OAuthService; requiredScopes?: string[] }
| { mode: 'apiKey'; label?: string; placeholder?: string }
```
### OAuth mode
For services with existing OAuth providers in `apps/sim/lib/oauth/types.ts`. The `provider` must match an `OAuthService`. The modal shows a credential picker and handles token refresh automatically.
### API key mode
For services that use API key / Bearer token auth. The modal shows a password input with the configured `label` and `placeholder`. The API key is encrypted at rest using AES-256-GCM and stored in a dedicated `encryptedApiKey` column on the connector record. The sync engine decrypts it automatically — connectors receive the raw access token in `listDocuments`, `getDocument`, and `validateConfig`.
## Connector Structure (meta.ts + runtime)
The declarative metadata lives in `meta.ts` (`ConnectorMeta`). The runtime functions live in `{service}.ts` (`ConnectorConfig`), which spreads the meta as its first property.
### `meta.ts` — client-safe metadata
```typescript
import { {Service}Icon } from '@/components/icons'
import type { ConnectorMeta } from '@/connectors/types'
export const {service}ConnectorMeta: ConnectorMeta = {
id: '{service}',
name: '{Service}',
description: 'Sync documents from {Service} into your knowledge base',
version: '1.0.0',
icon: {Service}Icon,
auth: {
mode: 'oauth',
provider: '{service}', // Must match OAuthService in lib/oauth/types.ts
requiredScopes: ['read:...'],
},
configFields: [
// Rendered dynamically by the add-connector modal UI
// Supports 'short-input', 'dropdown', and 'selector' types — see ConfigField Types below
],
// Optional: tag definitions are metadata too — declare them here
// tagDefinitions: [ ... ],
}
```
Keep `meta.ts` free of any server/runtime import. Only the icon, the `ConnectorMeta` type, and pure-data constants belong here.
### `{service}.ts` — runtime (OAuth example)
```typescript
import { createLogger } from '@sim/logger'
import { fetchWithRetry } from '@/lib/knowledge/documents/utils'
import { {service}ConnectorMeta } from '@/connectors/{service}/meta'
import type { ConnectorConfig, ExternalDocument, ExternalDocumentList } from '@/connectors/types'
const logger = createLogger('{Service}Connector')
export const {service}Connector: ConnectorConfig = {
...{service}ConnectorMeta,
listDocuments: async (accessToken, sourceConfig, cursor) => {
// Return metadata stubs with contentDeferred: true (if per-doc content fetch needed)
// Or full documents with content (if list API returns content inline)
// Return { documents: ExternalDocument[], nextCursor?, hasMore }
},
getDocument: async (accessToken, sourceConfig, externalId) => {
// Fetch full content for a single document
// Return ExternalDocument with contentDeferred: false, or null
},
validateConfig: async (accessToken, sourceConfig) => {
// Return { valid: true } or { valid: false, error: 'message' }
},
// Optional: map source metadata to semantic tag keys (translated to slots by sync engine)
mapTags: (metadata) => {
// Return Record<string, unknown> with keys matching tagDefinitions[].id
},
}
```
### API key connector example
The split is identical — `auth` lives in `meta.ts`, runtime functions in `{service}.ts`.
```typescript
// meta.ts
export const {service}ConnectorMeta: ConnectorMeta = {
id: '{service}',
name: '{Service}',
description: 'Sync documents from {Service} into your knowledge base',
version: '1.0.0',
icon: {Service}Icon,
auth: {
mode: 'apiKey',
label: 'API Key', // Shown above the input field
placeholder: 'Enter your {Service} API key', // Input placeholder
},
configFields: [ /* ... */ ],
}
// {service}.ts
export const {service}Connector: ConnectorConfig = {
...{service}ConnectorMeta,
listDocuments: async (accessToken, sourceConfig, cursor) => { /* ... */ },
getDocument: async (accessToken, sourceConfig, externalId) => { /* ... */ },
validateConfig: async (accessToken, sourceConfig) => { /* ... */ },
}
```
## ConfigField Types
The add-connector modal renders these automatically — no custom UI needed.
Three field types are supported: `short-input`, `dropdown`, and `selector`.
```typescript
// Text input
{
id: 'domain',
title: 'Domain',
type: 'short-input',
placeholder: 'yoursite.example.com',
required: true,
}
// Dropdown (static options)
{
id: 'contentType',
title: 'Content Type',
type: 'dropdown',
required: false,
options: [
{ label: 'Pages only', id: 'page' },
{ label: 'Blog posts only', id: 'blogpost' },
{ label: 'All content', id: 'all' },
],
}
```
## Dynamic Selectors (Canonical Pairs)
Use `type: 'selector'` to fetch options dynamically from the existing selector registry (`hooks/selectors/registry.ts`). Selectors are always paired with a manual fallback input using the **canonical pair** pattern — a `selector` field (basic mode) and a `short-input` field (advanced mode) linked by `canonicalParamId`.
The user sees a toggle button (ArrowLeftRight) to switch between the selector dropdown and manual text input. On submit, the modal resolves each canonical pair to the active mode's value, keyed by `canonicalParamId`.
### Rules
1. **Every selector field MUST have a canonical pair** — a corresponding `short-input` (or `dropdown`) field with the same `canonicalParamId` and `mode: 'advanced'`.
2. **`required` must be set identically on both fields** in a pair. If the selector is required, the manual input must also be required.
3. **`canonicalParamId` must match the key the connector expects in `sourceConfig`** (e.g. `baseId`, `channel`, `teamId`). The advanced field's `id` should typically match `canonicalParamId`.
4. **`dependsOn` references the selector field's `id`**, not the `canonicalParamId`. The modal propagates dependency clearing across canonical siblings automatically — changing either field in a parent pair clears dependent children.
### Selector canonical pair example (Airtable base → table cascade)
```typescript
configFields: [
// Base: selector (basic) + manual (advanced)
{
id: 'baseSelector',
title: 'Base',
type: 'selector',
selectorKey: 'airtable.bases', // Must exist in hooks/selectors/registry.ts
canonicalParamId: 'baseId',
mode: 'basic',
placeholder: 'Select a base',
required: true,
},
{
id: 'baseId',
title: 'Base ID',
type: 'short-input',
canonicalParamId: 'baseId',
mode: 'advanced',
placeholder: 'e.g. appXXXXXXXXXXXXXX',
required: true,
},
// Table: selector depends on base (basic) + manual (advanced)
{
id: 'tableSelector',
title: 'Table',
type: 'selector',
selectorKey: 'airtable.tables',
canonicalParamId: 'tableIdOrName',
mode: 'basic',
dependsOn: ['baseSelector'], // References the selector field ID
placeholder: 'Select a table',
required: true,
},
{
id: 'tableIdOrName',
title: 'Table Name or ID',
type: 'short-input',
canonicalParamId: 'tableIdOrName',
mode: 'advanced',
placeholder: 'e.g. Tasks',
required: true,
},
// Non-selector fields stay as-is
{ id: 'maxRecords', title: 'Max Records', type: 'short-input', ... },
]
```
### Selector with domain dependency (Jira/Confluence pattern)
When a selector depends on a plain `short-input` field (no canonical pair), `dependsOn` references that field's `id` directly. The `domain` field's value maps to `SelectorContext.domain` automatically via `SELECTOR_CONTEXT_FIELDS`.
```typescript
configFields: [
{
id: 'domain',
title: 'Jira Domain',
type: 'short-input',
placeholder: 'yoursite.atlassian.net',
required: true,
},
{
id: 'projectSelector',
title: 'Project',
type: 'selector',
selectorKey: 'jira.projects',
canonicalParamId: 'projectKey',
mode: 'basic',
dependsOn: ['domain'],
placeholder: 'Select a project',
required: true,
},
{
id: 'projectKey',
title: 'Project Key',
type: 'short-input',
canonicalParamId: 'projectKey',
mode: 'advanced',
placeholder: 'e.g. ENG, PROJ',
required: true,
},
]
```
### How `dependsOn` maps to `SelectorContext`
The connector selector field builds a `SelectorContext` from dependency values. For the mapping to work, each dependency's `canonicalParamId` (or field `id` for non-canonical fields) must exist in `SELECTOR_CONTEXT_FIELDS` (`lib/workflows/subblocks/context.ts`):
```
oauthCredential, domain, teamId, projectId, knowledgeBaseId, planId,
siteId, collectionId, spreadsheetId, fileId, baseId, datasetId, serviceDeskId
```
### Available selector keys
Check `hooks/selectors/types.ts` for the full `SelectorKey` union. Common ones for connectors:
| SelectorKey | Context Deps | Returns |
|-------------|-------------|---------|
| `airtable.bases` | credential | Base ID + name |
| `airtable.tables` | credential, `baseId` | Table ID + name |
| `slack.channels` | credential | Channel ID + name |
| `gmail.labels` | credential | Label ID + name |
| `google.calendar` | credential | Calendar ID + name |
| `linear.teams` | credential | Team ID + name |
| `linear.projects` | credential, `teamId` | Project ID + name |
| `jira.projects` | credential, `domain` | Project key + name |
| `confluence.spaces` | credential, `domain` | Space key + name |
| `notion.databases` | credential | Database ID + name |
| `asana.workspaces` | credential | Workspace GID + name |
| `microsoft.teams` | credential | Team ID + name |
| `microsoft.channels` | credential, `teamId` | Channel ID + name |
| `webflow.sites` | credential | Site ID + name |
| `outlook.folders` | credential | Folder ID + name |
## ExternalDocument Shape
Every document returned from `listDocuments`/`getDocument` must include:
```typescript
{
externalId: string // Source-specific unique ID
title: string // Document title
content: string // Extracted plain text (or '' if contentDeferred)
contentDeferred?: boolean // true = content will be fetched via getDocument
mimeType: 'text/plain' // Always text/plain (content is extracted)
contentHash: string // Metadata-based hash for change detection
sourceUrl?: string // Link back to original (stored on document record)
metadata?: Record<string, unknown> // Source-specific data (fed to mapTags)
}
```
## Content Deferral (Required for file/content-download connectors)
**All connectors that require per-document API calls to fetch content MUST use `contentDeferred: true`.** This is the standard pattern — `listDocuments` returns lightweight metadata stubs, and content is fetched lazily by the sync engine via `getDocument` only for new/changed documents.
This pattern is critical for reliability: the sync engine processes documents in batches and enqueues each batch for processing immediately. If a sync times out, all previously-batched documents are already queued. Without deferral, content downloads during listing can exhaust the sync task's time budget before any documents are saved.
### When to use `contentDeferred: true`
- The service's list API does NOT return document content (only metadata)
- Content requires a separate download/export API call per document
- Examples: Google Drive, OneDrive, SharePoint, Dropbox, Notion, Confluence, Gmail, Obsidian, Evernote, GitHub
### When NOT to use `contentDeferred`
- The list API already returns the full content inline (e.g., Slack messages, Reddit posts, HubSpot notes)
- No per-document API call is needed to get content
### Content Hash Strategy
Use a **metadata-based** `contentHash` — never a content-based hash. The hash must be derivable from the list response metadata alone, so the sync engine can detect changes without downloading content.
Good metadata hash sources:
- `modifiedTime` / `lastModifiedDateTime` — changes when file is edited
- Git blob SHA — unique per content version
- API-provided content hash (e.g., Dropbox `content_hash`)
- Version number (e.g., Confluence page version)
Format: `{service}:{id}:{changeIndicator}`
```typescript
// Google Drive: modifiedTime changes on edit
contentHash: `gdrive:${file.id}:${file.modifiedTime ?? ''}`
// GitHub: blob SHA is a content-addressable hash
contentHash: `gitsha:${item.sha}`
// Dropbox: API provides content_hash
contentHash: `dropbox:${entry.id}:${entry.content_hash ?? entry.server_modified}`
// Confluence: version number increments on edit
contentHash: `confluence:${page.id}:${page.version.number}`
```
**Critical invariant:** The `contentHash` MUST be identical whether produced by `listDocuments` (stub) or `getDocument` (full doc). Both should use the same stub function to guarantee this.
### Implementation Pattern
```typescript
// 1. Create a stub function (sync, no API calls)
function fileToStub(file: ServiceFile): ExternalDocument {
return {
externalId: file.id,
title: file.name || 'Untitled',
content: '',
contentDeferred: true,
mimeType: 'text/plain',
sourceUrl: `https://service.com/file/${file.id}`,
contentHash: `service:${file.id}:${file.modifiedTime ?? ''}`,
metadata: { /* fields needed by mapTags */ },
}
}
// 2. listDocuments returns stubs (fast, metadata only)
listDocuments: async (accessToken, sourceConfig, cursor) => {
const response = await fetchWithRetry(listUrl, { ... })
const files = (await response.json()).files
const documents = files.map(fileToStub)
return { documents, nextCursor, hasMore }
}
// 3. getDocument fetches content and returns full doc with SAME contentHash
getDocument: async (accessToken, sourceConfig, externalId) => {
const metadata = await fetchWithRetry(metadataUrl, { ... })
const file = await metadata.json()
if (file.trashed) return null
try {
const content = await fetchContent(accessToken, file)
if (!content.trim()) return null
const stub = fileToStub(file)
return { ...stub, content, contentDeferred: false }
} catch (error) {
logger.warn(`Failed to fetch content for: ${file.name}`, { error })
return null
}
}
```
### Reference Implementations
- **Google Drive**: `connectors/google-drive/google-drive.ts` — file download/export with `modifiedTime` hash
- **GitHub**: `connectors/github/github.ts` — git blob SHA hash
- **Notion**: `connectors/notion/notion.ts` — blocks API with `last_edited_time` hash
- **Confluence**: `connectors/confluence/confluence.ts` — version number hash
## tagDefinitions — Declared Tag Definitions
Declare which tags the connector populates using semantic IDs. Shown in the add-connector modal as opt-out checkboxes.
On connector creation, slots are **dynamically assigned** via `getNextAvailableSlot` — connectors never hardcode slot names.
```typescript
tagDefinitions: [
{ id: 'labels', displayName: 'Labels', fieldType: 'text' },
{ id: 'version', displayName: 'Version', fieldType: 'number' },
{ id: 'lastModified', displayName: 'Last Modified', fieldType: 'date' },
],
```
Each entry has:
- `id`: Semantic key matching a key returned by `mapTags` (e.g. `'labels'`, `'version'`)
- `displayName`: Human-readable name shown in the UI (e.g. "Labels", "Last Modified")
- `fieldType`: `'text'` | `'number'` | `'date'` | `'boolean'` — determines which slot pool to draw from
Users can opt out of specific tags in the modal. Disabled IDs are stored in `sourceConfig.disabledTagIds`.
The assigned mapping (`semantic id → slot`) is stored in `sourceConfig.tagSlotMapping`.
## `@/connectors/utils` Helpers
Reuse these instead of inlining the same logic (the validator enforces them):
- `htmlToPlainText(html)` — strip HTML to plain text before indexing `ExternalDocument.content`. Never index raw HTML.
- `computeContentHash(content)` — stable content hash for change detection.
- `parseTagDate(value)` — parse to a valid `Date` or `undefined` (guards Invalid Date). Use in `mapTags` for date fields.
- `joinTagArray(value)` — validate an array and join to a comma-separated string, or `undefined`. Use in `mapTags` for array/label fields.
- `parseMultiValue(value)` — normalize a value into a `string[]`.
## mapTags — Metadata to Semantic Keys
Maps source metadata to semantic tag keys. Required if `tagDefinitions` is set.
The sync engine calls this automatically and translates semantic keys to actual DB slots
using the `tagSlotMapping` stored on the connector.
Return keys must match the `id` values declared in `tagDefinitions`.
Use the `@/connectors/utils` helpers for the common transforms — don't hand-roll date/array validation:
```typescript
import { joinTagArray, parseTagDate } from '@/connectors/utils'
mapTags: (metadata: Record<string, unknown>): Record<string, unknown> => {
const result: Record<string, unknown> = {}
// joinTagArray validates the array and joins to a comma-separated string (undefined if empty)
const labels = joinTagArray(metadata.labels)
if (labels) result.labels = labels
// Validate numbers — guard against NaN
if (metadata.version != null) {
const num = Number(metadata.version)
if (!Number.isNaN(num)) result.version = num
}
// parseTagDate returns a valid Date or undefined (guards against Invalid Date)
const lastModified = parseTagDate(metadata.lastModified)
if (lastModified) result.lastModified = lastModified
return result
}
```
## External API Calls — Use `fetchWithRetry`
All external API calls must use `fetchWithRetry` from `@/lib/knowledge/documents/utils` instead of raw `fetch()`. This provides exponential backoff with retries on 429/502/503/504 errors. It returns a standard `Response` — all `.ok`, `.json()`, `.text()` checks work unchanged.
For `validateConfig` (user-facing, called on save), pass `VALIDATE_RETRY_OPTIONS` to cap wait time at ~7s. Background operations (`listDocuments`, `getDocument`) use the built-in defaults (5 retries, ~31s max).
```typescript
import { VALIDATE_RETRY_OPTIONS, fetchWithRetry } from '@/lib/knowledge/documents/utils'
// Background sync — use defaults
const response = await fetchWithRetry(url, {
method: 'GET',
headers: { Authorization: `Bearer ${accessToken}` },
})
// validateConfig — tighter retry budget
const response = await fetchWithRetry(url, { ... }, VALIDATE_RETRY_OPTIONS)
```
## sourceUrl
If `ExternalDocument.sourceUrl` is set, the sync engine stores it on the document record. Always construct the full URL (not a relative path).
## Capped or Incomplete Listings — `syncContext.listingCapped` (REQUIRED)
If `listDocuments` can ever return **less than the full source set** on a non-incremental sync — a `maxItems`/`maxDocuments`-style cap, or a transient per-item error that drops a still-existing document from the listing — it MUST set `syncContext.listingCapped = true` when that happens.
The sync engine reconciles deletions by comparing the full listing against stored documents: anything not seen is **hard-deleted** (sync-engine.ts, gated on `!syncContext?.listingCapped`). A truncated listing without this flag deletes every real document beyond the cap. This was the single most common bug found when auditing connectors — do not omit it.
```typescript
if (hitLimit && syncContext) {
syncContext.listingCapped = true
}
```
Rules:
- Set it when a user-configured cap truncates the listing while more documents exist
- Set it when a thrown error caused a still-present document to be skipped during listing
- Do NOT set it when the source is genuinely exhausted (deleted documents must still reconcile)
- Do NOT set it for intentional scope filters (e.g. a date cutoff) — out-of-scope documents should be reconciled normally
## Sync Engine Behavior (Do Not Modify)
The sync engine (`lib/knowledge/connectors/sync-engine.ts`) is connector-agnostic. It:
1. Calls `listDocuments` with pagination until `hasMore` is false
2. Compares `contentHash` to detect new/changed/unchanged documents
3. Stores `sourceUrl` and calls `mapTags` on insert/update automatically
4. Handles soft-delete of removed documents
5. Resolves access tokens automatically — OAuth tokens are refreshed, API keys are decrypted from the `encryptedApiKey` column
You never need to modify the sync engine when adding a connector.
## Icon
The `icon` field on `ConnectorConfig` is used throughout the UI — in the connector list, the add-connector modal, and as the document icon in the knowledge base table (replacing the generic file type icon for connector-sourced documents). The icon is read from `CONNECTOR_REGISTRY[connectorType].icon` at runtime — no separate icon map to maintain.
If the service already has an icon in `apps/sim/components/icons.tsx` (from a tool integration), reuse it. Otherwise, ask the user to provide the SVG.
## Registering
Register in BOTH registries, keeping the same alphabetical-by-id ordering in each.
1. **Server registry**`apps/sim/connectors/registry.server.ts` (server-only full registry; holds full connectors with runtime functions, imported by the sync engine and knowledge API routes):
```typescript
import { {service}Connector } from '@/connectors/{service}'
export const CONNECTOR_REGISTRY: ConnectorRegistry = {
// ... existing connectors ...
{service}: {service}Connector,
}
```
2. **Client-safe meta registry**`apps/sim/connectors/registry.ts` (imports each connector's `meta.ts` only, so client components can use it without pulling server-only code; the metadata counterpart to `BLOCK_META_REGISTRY`):
```typescript
import { {service}ConnectorMeta } from '@/connectors/{service}/meta'
export const CONNECTOR_META_REGISTRY: ConnectorMetaRegistry = {
// ... existing connector metas ...
{service}: {service}ConnectorMeta,
}
```
`registry.ts` exports `CONNECTOR_META_REGISTRY: ConnectorMetaRegistry` plus the helpers `getConnectorMeta(id)` and `getAllConnectorMeta()`, importing each `@/connectors/{service}/meta` directly — never the runtime module. `registry.server.ts` exports `CONNECTOR_REGISTRY: ConnectorRegistry`.
## Reference Implementations
- **OAuth + contentDeferred**: `apps/sim/connectors/google-drive/google-drive.ts` — file download with metadata-based hash, `orderBy` for deterministic pagination
- **OAuth + contentDeferred (blocks API)**: `apps/sim/connectors/notion/notion.ts` — complex block content extraction deferred to `getDocument`
- **OAuth + contentDeferred (git)**: `apps/sim/connectors/github/github.ts` — blob SHA hash, tree listing
- **OAuth + inline content**: `apps/sim/connectors/confluence/confluence.ts` — multiple config field types, `mapTags`, label fetching
- **API key**: `apps/sim/connectors/fireflies/fireflies.ts` — GraphQL API with Bearer token auth
## Checklist
- [ ] Created `connectors/{service}/meta.ts` with `{service}ConnectorMeta: ConnectorMeta` (icon, name, auth, configFields, tagDefinitions) — no server/runtime imports
- [ ] Created `connectors/{service}/{service}.ts` with `{service}Connector: ConnectorConfig` spreading the meta + runtime functions
- [ ] Created `connectors/{service}/index.ts` barrel export
- [ ] **Auth configured correctly:**
- OAuth: `auth.provider` matches an existing `OAuthService` in `lib/oauth/types.ts`
- API key: `auth.label` and `auth.placeholder` set appropriately
- [ ] **Selector fields configured correctly (if applicable):**
- Every `type: 'selector'` field has a canonical pair (`short-input` or `dropdown` with same `canonicalParamId` and `mode: 'advanced'`)
- `required` is identical on both fields in each canonical pair
- `selectorKey` exists in `hooks/selectors/registry.ts`
- `dependsOn` references selector field IDs (not `canonicalParamId`)
- Dependency `canonicalParamId` values exist in `SELECTOR_CONTEXT_FIELDS`
- [ ] `listDocuments` handles pagination with metadata-based content hashes
- [ ] `syncContext.listingCapped = true` set whenever the listing is truncated (max-items cap or transient per-item error) — required to prevent the engine's deletion reconciliation from removing unseen documents
- [ ] `contentDeferred: true` used if content requires per-doc API calls (file download, export, blocks fetch)
- [ ] `contentHash` is metadata-based (not content-based) and identical between stub and `getDocument`
- [ ] `sourceUrl` set on each ExternalDocument (full URL, not relative)
- [ ] `metadata` includes source-specific data for tag mapping
- [ ] `tagDefinitions` declared for each semantic key returned by `mapTags`
- [ ] `mapTags` implemented if source has useful metadata (labels, dates, versions)
- [ ] `validateConfig` verifies the source is accessible
- [ ] All external API calls use `fetchWithRetry` (not raw `fetch`)
- [ ] All optional config fields validated in `validateConfig`
- [ ] Icon exists in `components/icons.tsx` (or asked user to provide SVG)
- [ ] Registered the full connector in `connectors/registry.server.ts`
- [ ] Registered the meta in `connectors/registry.ts` (same alphabetical-by-id ordering as registry.server.ts)
+142
View File
@@ -0,0 +1,142 @@
---
description: Add a code-defined table enrichment (registry entry) backed by a provider cascade, ensuring each provider tool has hosted-key support
argument-hint: <enrichment-name>
---
# Adding a Table Enrichment
Enrichments are code-defined entries in `apps/sim/enrichments/` that run **directly per table row** (no workflow). Each enrichment declares inputs, outputs, and an ordered list of **providers**; the cascade runner tries providers in order and the first non-empty result fills the cell. Each provider calls one existing Sim tool via `executeTool`, which injects the workspace's BYOK key or a **hosted key** and bills usage automatically.
Because enrichments run on Sim's hosted keys by default, **every provider tool you reference must have hosted-key support** — otherwise it can only run when the workspace brings its own key. This command makes that check a required step.
## Overview
| Step | What | Where |
|------|------|-------|
| 1 | Pick the data-source tool(s) for each output | `tools/{service}/` + `tools/registry.ts` |
| 2 | **Verify each tool has `hosting`; if not, run `/add-hosted-key`** | `tools/{service}/{action}.ts` |
| 3 | Write the enrichment definition | `enrichments/{name}/{name}.ts` + `index.ts` |
| 4 | Register it | `enrichments/registry.ts` |
| 5 | Verify | tsc / biome / manual run |
## Architecture (what you're plugging into)
- **`enrichments/types.ts`** — `EnrichmentConfig { id, name, description, icon, inputs, outputs, providers }` and `EnrichmentProvider { id, label, toolId, buildParams, mapOutput }`. Providers are **plain data** (no `@/tools` import) so the catalog stays client-safe.
- **`enrichments/providers.ts`** — `toolProvider(...)` (typed passthrough) plus shared input helpers: `str(v)`, `normalizeDomain(v)`, `firstNonEmpty(arr)`, `splitName(fullName)`.
- **`enrichments/run.ts`** — the server-only cascade runner. Calls `executeTool(provider.toolId, { ...params, _context: { workspaceId } })`, accumulates hosted-key cost, returns the first non-empty mapped result. **You do not edit this** — it works for any registry entry.
- **`enrichments/registry.ts`** — `ENRICHMENT_REGISTRY` / `ALL_ENRICHMENTS` / `getEnrichment`. Register new entries here.
Outputs automatically become table columns; billing, the catalog/sidebar UI, the column meta-header icon, and per-row execution all work with no extra wiring.
## Step 1: Pick the data-source tool(s)
For each output the enrichment produces, decide which existing tool provides it. Look up the service's API and the tool in `apps/sim/tools/{service}/` (e.g. `hunter_email_finder`, `pdl_person_enrich`, `pdl_company_enrich`). Confirm:
- The tool id is registered in `apps/sim/tools/registry.ts`.
- Its `params` accept what you can derive from table columns (read the tool's `params`).
- Its `outputs` / `transformResponse` actually expose the field you need (read the real output shape — don't assume).
Order providers **cheapest / most-likely-to-hit first**; the cascade stops at the first non-empty result. Apollo / LinkedIn are not hosted-safe (ToS) — don't use them.
## Step 2: Verify hosted-key support — chain to `/add-hosted-key` if missing
**This is the required gate.** For every tool a provider calls, open `apps/sim/tools/{service}/{action}.ts` and check for a `hosting` block:
```typescript
hosting: {
envKeyPrefix: 'SERVICE_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'service',
pricing: { /* ... */ },
rateLimit: { /* ... */ },
}
```
- **If `hosting` is present** — good. Note the `envKeyPrefix`; the deployment needs `{PREFIX}_COUNT` + `{PREFIX}_1..N` env vars set for the hosted key to actually resolve at runtime (ops concern, not code). If those env vars aren't set in the target environment, the provider will only run with a workspace BYOK key.
- **If `hosting` is absent** — the tool can't use a Sim-provided key, so the enrichment would silently produce blank cells on hosted Sim. **Stop and run `/add-hosted-key <service>`** to add hosted-key support to that tool first, then come back. Do this for every provider tool that lacks it.
Why it matters: the cascade runner only bills (and only reads `output.cost.total`) when `executeTool` injected a hosted key, which requires the tool's `hosting` config. No `hosting` → no hosted key → the enrichment depends entirely on per-workspace BYOK.
## Step 3: Write the enrichment definition
Create `apps/sim/enrichments/{name}/{name}.ts` and a barrel `index.ts`. Mirror the existing entries (`work-email`, `phone-number`, `company-domain`, `company-info`).
```typescript
import { SomeIcon } from 'lucide-react'
import { filterUndefined } from '@sim/utils/object'
import { normalizeDomain, splitName, str, toolProvider } from '@/enrichments/providers'
import type { EnrichmentConfig } from '@/enrichments/types'
export const myEnrichment: EnrichmentConfig = {
id: 'my-enrichment',
name: 'My Enrichment',
description: 'One concise sentence describing what it finds.',
icon: SomeIcon,
inputs: [
// Person enrichments take a single canonical `fullName` (Clay-style);
// split it with splitName() for tools that need first/last.
{ id: 'fullName', name: 'Full name', type: 'string', required: true },
{ id: 'companyDomain', name: 'Company domain', type: 'string' },
],
outputs: [{ id: 'value', name: 'value', type: 'string' }],
providers: [
toolProvider({
id: 'provider-a',
label: 'Provider A',
toolId: 'service_action', // must have `hosting` (Step 2)
buildParams: (inputs) => {
// Return null when there aren't enough inputs → cascade skips this provider.
const name = splitName(inputs.fullName)
const domain = normalizeDomain(inputs.companyDomain)
if (!name || !domain) return null
return { domain, first_name: name.firstName, last_name: name.lastName }
},
mapOutput: (output) => {
// Return { [outputId]: value } on a hit, or null to fall through.
const value = str(output.value)
return value ? { value } : null
},
}),
// ...additional fallback providers, in priority order.
],
}
```
```typescript
// apps/sim/enrichments/{name}/index.ts
export { myEnrichment } from './my-enrichment'
```
Rules:
- Keep the file **client-safe**: import only `lucide-react`, `@sim/utils/*`, `@/enrichments/providers`, and the types. **Never import `@/tools`** here — the runner does the tool call.
- `buildParams` returns `null` when inputs are insufficient (provider skipped). `mapOutput` returns `null`/empty for a miss (falls through). Use `filterUndefined` when assembling optional tool params; coerce numbers explicitly (don't pass `''` to number outputs).
- Output `id`s are the keys `mapOutput` returns; output `name`s are the default column names (the user can rename them in the config).
## Step 4: Register it
In `apps/sim/enrichments/registry.ts`, import and add the entry (catalog order is registration order):
```typescript
import { myEnrichment } from '@/enrichments/my-enrichment'
export const ENRICHMENT_REGISTRY: EnrichmentRegistry = {
// ...existing
[myEnrichment.id]: myEnrichment,
}
```
## Step 5: Verify
1. `bunx tsc --noEmit` (from `apps/sim`, `NODE_OPTIONS=--max-old-space-size=8192`) and `bunx biome check` on the changed files.
2. In a table → **+ New column → Enrichments** → pick the new enrichment, map its inputs to columns, name the output column(s), Save. Confirm it appears in the catalog with its icon/description.
3. With hosted keys (or a workspace BYOK key) configured for each provider's service, run a row and confirm the cell fills; the dev-server log shows `Enrichment hit { provider }`. A row whose providers all miss completes blank; a row where every provider errored shows an error cell.
## Checklist
- [ ] Each output mapped to a real tool field (verified against the tool's `params`/`outputs`)
- [ ] **Every provider tool has a `hosting` block — ran `/add-hosted-key` for any that didn't**
- [ ] Providers ordered cheapest / most-likely-first; Apollo/LinkedIn not used
- [ ] Enrichment file is client-safe (no `@/tools` import); uses `toolProvider` + shared helpers
- [ ] `buildParams` returns `null` on insufficient inputs; `mapOutput` returns `null` on a miss
- [ ] Registered in `enrichments/registry.ts`
- [ ] tsc + biome clean; created and ran the column end-to-end
+72
View File
@@ -0,0 +1,72 @@
---
description: Add a runtime gated feature flag (AppConfig-backed on prod, secret fallback off-prod), gated by org id, user id, or admin
argument-hint: <flag-name>
---
# Add Feature Flag Skill
You add a **runtime, gated feature flag** to Sim — one that can be turned on for specific orgs, users, or admins and changed on prod with no redeploy (AWS AppConfig). When AppConfig isn't the source of truth, the flag falls back to a single **secret** (on/off only).
## When to use this vs `env-flags.ts`
- **Feature flag** (`@/lib/core/config/feature-flags.ts`): per-request, gated by `userId`/`orgId`/admin, changeable at runtime. This skill.
- **Env flag** (`@/lib/core/config/env-flags.ts`): deploy-time capability/environment detection (`isProd`, `isHosted`, `isBillingEnabled`). A module-load boolean. **Do not add gated flags here.**
If the user wants a fixed per-deployment toggle, send them to `env-flags.ts` instead.
## The flag model
A flag's **gating rule lives only in the hosted AppConfig document**. It is ON for a context when any clause matches:
```ts
interface FeatureFlagRule {
enabled?: boolean // global default for everyone
orgIds?: string[] // allowlisted organization ids
userIds?: string[] // allowlisted user ids
admins?: boolean // platform admins (user.role === 'admin')
}
```
Critically, **none of this is expressible in code** — gating (especially `admins`) can only be set through AppConfig, so no environment can grant access from a code literal. Off-AppConfig (self-hosted/OSS/local), a flag is simply on or off, derived from its fallback secret.
## Steps
1. **Define the flag.** Add one entry to the `FEATURE_FLAGS` registry in `apps/sim/lib/core/config/feature-flags.ts`. Each entry is the flag's whole definition — name (kebab-case key), `description`, and the `fallback` secret consulted when AppConfig isn't the source of truth (truthy ⇒ on globally):
```ts
const FEATURE_FLAGS = {
'<flag-name>': {
description: '<what this gates>',
fallback: '<FLAG_SECRET>',
},
}
```
`fallback` is the env/secret key (typed as `keyof typeof env`), so add `<FLAG_SECRET>` to `apps/sim/lib/core/config/env.ts` first (and the deployment's secret store) — it won't typecheck otherwise. Do **not** add org/user/admin defaults here — that gating exists only in AppConfig. Adding the entry makes `<flag-name>` a valid `FeatureFlagName`.
2. **Gate the call site.** Call `isFeatureEnabled` with whatever ids you have — admin status is resolved internally, so callers never pass it:
```ts
import { isFeatureEnabled } from '@/lib/core/config/feature-flags'
if (await isFeatureEnabled('<flag-name>', { userId, orgId })) {
// gated behavior
}
```
- Missing ids are fine — a clause with no matching id is skipped; with no `userId`, the admin clause resolves to `false` without a DB read.
- Admin routes that already know the caller is an admin may pass `{ userId, isAdmin: true }` to skip the role lookup.
- **Client/UI flags:** resolve server-side (in a server component, route, or loader) and pass the boolean down as a prop. There is no client AppConfig.
3. **(Prod) configure in AppConfig.** The infra `feature-flags` profile schema is permissive, so a new flag needs **no infra change**. Operators add the flag under `flags` in the hosted `feature-flags` document — including any `orgIds`/`userIds`/`admins` gating — and start a `sim-<env>-fast` deployment (see the AppConfig runbook in the infra README — same flow as `access-control`). The fallback secret only applies when AppConfig is disabled.
4. **Test.** Add a case to `apps/sim/lib/core/config/feature-flags.test.ts`: use `withAppConfig({ flags: { ... } })` to cover the gating rule (mock `isPlatformAdmin` for the `admins` clause), and toggle the fallback secret to cover the off-AppConfig path.
5. **Clean up after rollout.** When the feature ships to everyone, delete the flag's entry from `FEATURE_FLAGS`, the `<FLAG_SECRET>` env entry, the AppConfig document, the call sites, and the test. Leaving dead flags around is the main failure mode of flag systems.
## Notes
- Flag keys are `kebab-case`.
- Never read flags via raw `fetch` or a new AppConfig client — always go through `isFeatureEnabled` / `getFeatureFlags`.
- Never bake gating into code. The fallback is a single boolean secret; org/user/admin scoping is AppConfig-only.
- The admin check reads the DB **replica** (`dbReplica`) and is resolved lazily, so an admin-gated flag adds at most one cheap replica read, and only when `admins` is the deciding clause.
+301
View File
@@ -0,0 +1,301 @@
---
description: Add hosted API key support to a tool so Sim provides the key when users don't bring their own
argument-hint: <service-name>
---
# Adding Hosted Key Support to a Tool
When a tool has hosted key support, Sim provides its own API key if the user hasn't configured one (via BYOK or env var). Usage is metered and billed to the workspace.
## Overview
| Step | What | Where |
|------|------|-------|
| 1 | Register BYOK provider ID | `tools/types.ts`, `lib/api/contracts/byok-keys.ts` |
| 2 | Research the API's pricing and rate limits | API docs / pricing page (before writing any code) |
| 3 | Add `hosting` config to the tool | `tools/{service}/{action}.ts` |
| 4 | Hide API key field when hosted | `blocks/blocks/{service}.ts` |
| 5 | Add to BYOK settings UI | BYOK settings component (`byok.tsx`) |
| 6 | Summarize pricing and throttling comparison | Output to user (after all code changes) |
## Step 1: Register the BYOK Provider ID
Add the new provider to the `BYOKProviderId` union in `tools/types.ts`:
```typescript
export type BYOKProviderId =
| 'openai'
| 'anthropic'
// ...existing providers
| 'your_service'
```
Then add the same provider id to the `byokProviderIdSchema` enum in `lib/api/contracts/byok-keys.ts` (this is what the byok-keys route validates against):
```typescript
export const byokProviderIdSchema = z.enum([
'openai',
'anthropic',
// ...existing providers
'your_service',
])
```
## Step 2: Research the API's Pricing Model and Rate Limits
**Before writing any `getCost` or `rateLimit` code**, look up the service's official documentation for both pricing and rate limits. You need to understand:
### Pricing
1. **How the API charges** — per request, per credit, per token, per step, per minute, etc.
2. **Whether the API reports cost in its response** — look for fields like `creditsUsed`, `costDollars`, `tokensUsed`, or similar in the response body or headers
3. **Whether cost varies by endpoint/options** — some APIs charge more for certain features (e.g., Firecrawl charges 1 credit/page base but +4 for JSON format, +4 for enhanced mode)
4. **The dollar-per-unit rate** — what each credit/token/unit costs in dollars on our plan
### Rate Limits
1. **What rate limits the API enforces** — requests per minute/second, tokens per minute, concurrent requests, etc.
2. **Whether limits vary by plan tier** — free vs paid vs enterprise often have different ceilings
3. **Whether limits are per-key or per-account** — determines whether adding more hosted keys actually increases total throughput
4. **What the API returns when rate limited** — HTTP 429, `Retry-After` header, error body format, etc.
5. **Whether there are multiple dimensions** — some APIs limit both requests/min AND tokens/min independently
Search the API's docs/pricing page (use WebSearch/WebFetch). Capture the pricing model as a comment in `getCost` so future maintainers know the source of truth.
### Setting Our Rate Limits
Our rate limiter (`lib/core/rate-limiter/hosted-key/`) uses a token-bucket algorithm applied **per billing actor** (workspace). It supports two modes:
- **`per_request`** — simple; just `requestsPerMinute`. Good when the API charges flat per-request or cost doesn't vary much.
- **`custom`** — `requestsPerMinute` plus additional `dimensions` (e.g., `tokens`, `search_units`). Each dimension has its own `limitPerMinute` and an `extractUsage` function that reads actual usage from the response. Use when the API charges on a variable metric (tokens, credits) and you want to cap that metric too.
When choosing values for `requestsPerMinute` and any dimension limits:
- **Stay well below the API's per-key limit** — our keys are shared across all workspaces. If the API allows 60 RPM per key and we have 3 keys, the global ceiling is ~180 RPM. Set the per-workspace limit low enough (e.g., 20-60 RPM) that many workspaces can coexist without collectively hitting the API's ceiling.
- **Account for key pooling** — our round-robin distributes requests across `N` hosted keys, so the effective API-side rate per key is `(total requests) / N`. But per-workspace limits are enforced *before* key selection, so they apply regardless of key count.
- **Prefer conservative defaults** — it's easy to raise limits later but hard to claw back after users depend on high throughput.
## Step 3: Add `hosting` Config to the Tool
Add a `hosting` object to the tool's `ToolConfig`. This tells the execution layer how to acquire hosted keys, calculate cost, and rate-limit.
```typescript
hosting: {
envKeyPrefix: 'YOUR_SERVICE_API_KEY',
apiKeyParam: 'apiKey',
byokProviderId: 'your_service',
pricing: {
type: 'custom',
getCost: (_params, output) => {
if (output.creditsUsed == null) {
throw new Error('Response missing creditsUsed field')
}
const creditsUsed = output.creditsUsed as number
const cost = creditsUsed * 0.001 // dollars per credit
return { cost, metadata: { creditsUsed } }
},
},
rateLimit: {
mode: 'per_request',
requestsPerMinute: 100,
},
},
```
### Hosted Key Env Var Convention
Keys use a numbered naming pattern driven by a count env var:
```
YOUR_SERVICE_API_KEY_COUNT=3
YOUR_SERVICE_API_KEY_1=sk-...
YOUR_SERVICE_API_KEY_2=sk-...
YOUR_SERVICE_API_KEY_3=sk-...
```
The `envKeyPrefix` value (`YOUR_SERVICE_API_KEY`) determines which env vars are read at runtime. Adding more keys only requires bumping the count and adding the new env var.
### Pricing: Prefer API-Reported Cost
Always prefer using cost data returned by the API (e.g., `creditsUsed`, `costDollars`). This is the most accurate because it accounts for variable pricing tiers, feature modifiers, and plan-level discounts.
**When the API reports cost** — use it directly and throw if missing:
```typescript
pricing: {
type: 'custom',
getCost: (params, output) => {
if (output.creditsUsed == null) {
throw new Error('Response missing creditsUsed field')
}
// $0.001 per credit — from https://example.com/pricing
const cost = (output.creditsUsed as number) * 0.001
return { cost, metadata: { creditsUsed: output.creditsUsed } }
},
},
```
**When the API does NOT report cost** — compute it from params/output based on the pricing docs, but still validate the data you depend on:
```typescript
pricing: {
type: 'custom',
getCost: (params, output) => {
if (!Array.isArray(output.searchResults)) {
throw new Error('Response missing searchResults, cannot determine cost')
}
// Serper: 1 credit for <=10 results, 2 credits for >10 — from https://serper.dev/pricing
const credits = Number(params.num) > 10 ? 2 : 1
return { cost: credits * 0.001, metadata: { credits } }
},
},
```
**`getCost` must always throw** if it cannot determine cost. Never silently fall back to a default — this would hide billing inaccuracies.
### Capturing Cost Data from the API
If the API returns cost info, capture it in `transformResponse` so `getCost` can read it from the output:
```typescript
transformResponse: async (response: Response) => {
const data = await response.json()
return {
success: true,
output: {
results: data.results,
creditsUsed: data.creditsUsed, // pass through for getCost
},
}
},
```
For async/polling tools, capture it in `postProcess` when the job completes:
```typescript
if (jobData.status === 'completed') {
result.output = {
data: jobData.data,
creditsUsed: jobData.creditsUsed,
}
}
```
## Step 4: Hide the API Key Field When Hosted
In the block config (`blocks/blocks/{service}.ts`), add `hideWhenHosted: true` to the API key subblock. This hides the field on hosted Sim since the platform provides the key:
```typescript
{
id: 'apiKey',
title: 'API Key',
type: 'short-input',
placeholder: 'Enter your API key',
password: true,
required: true,
hideWhenHosted: true,
},
```
The visibility is controlled by `isSubBlockHidden()` in `lib/workflows/subblocks/visibility.ts`, which checks both the `isHosted` feature flag (`hideWhenHosted`) and optional env var conditions (`hideWhenEnvSet`).
### Excluding Specific Operations from Hosted Key Support
When a block has multiple operations but some operations should **not** use a hosted key (e.g., the underlying API is deprecated, unsupported, or too expensive), use the **duplicate apiKey subblock** pattern. This is the same pattern Exa uses for its `research` operation:
1. **Remove the `hosting` config** from the tool definition for that operation — it must not have a `hosting` object at all.
2. **Duplicate the `apiKey` subblock** in the block config with opposing conditions:
```typescript
// API Key — hidden when hosted for operations with hosted key support
{
id: 'apiKey',
title: 'API Key',
type: 'short-input',
placeholder: 'Enter your API key',
password: true,
required: true,
hideWhenHosted: true,
condition: { field: 'operation', value: 'unsupported_op', not: true },
},
// API Key — always visible for unsupported_op (no hosted key support)
{
id: 'apiKey',
title: 'API Key',
type: 'short-input',
placeholder: 'Enter your API key',
password: true,
required: true,
condition: { field: 'operation', value: 'unsupported_op' },
},
```
Both subblocks share the same `id: 'apiKey'`, so the same value flows to the tool. The conditions ensure only one is visible at a time. The first has `hideWhenHosted: true` and shows for all hosted operations; the second has no `hideWhenHosted` and shows only for the excluded operation — meaning users must always provide their own key for that operation.
To exclude multiple operations, use an array: `{ field: 'operation', value: ['op_a', 'op_b'] }`.
**Reference implementations:**
- **Exa** (`blocks/blocks/exa.ts`): `exa_research` operation excluded from hosting — duplicate `apiKey` pair around lines ~348-365
- **Google Maps** (`blocks/blocks/google_maps.ts`): `speed_limits` operation excluded from hosting (deprecated Roads API)
## Step 5: Add to the BYOK Settings UI
Add an entry to the `PROVIDERS` array in the BYOK settings component so users can bring their own key. You need the service icon from `components/icons.tsx`:
```typescript
{
id: 'your_service',
name: 'Your Service',
icon: YourServiceIcon,
description: 'What this service does',
placeholder: 'Enter your API key',
},
```
## Step 6: Summarize Pricing and Throttling Comparison
After all code changes are complete, output a detailed summary to the user covering:
### What to include
1. **API's pricing model** — how the service charges (per token, per credit, per request, etc.), the specific rates found in docs, and whether the API reports cost in responses.
2. **Our `getCost` approach** — how we calculate cost, what fields we depend on, and any assumptions or estimates (especially when the API doesn't report exact dollar cost).
3. **API's rate limits** — the documented limits (RPM, TPM, concurrent, etc.), which plan tier they apply to, and whether they're per-key or per-account.
4. **Our `rateLimit` config** — what we set for `requestsPerMinute` (and dimensions if custom mode), why we chose those values, and how they compare to the API's limits.
5. **Key pooling impact** — how many hosted keys we expect, and how round-robin distribution affects the effective per-key rate at the API.
6. **Gaps or risks** — anything the API charges for that we don't meter, rate limit dimensions we chose not to enforce, or pricing that may be inaccurate due to variable model/tier costs.
### Format
Present this as a structured summary with clear headings. Example:
```
### Pricing
- **API charges**: $X per 1M tokens (input), $Y per 1M tokens (output) — varies by model
- **Response reports cost?**: No — only token counts in `usage` field
- **Our getCost**: Estimates cost at $Z per 1M total tokens based on median model pricing
- **Risk**: Actual cost varies by model; our estimate may over/undercharge for cheap/expensive models
### Throttling
- **API limits**: 300 RPM per key (paid tier), 60 RPM (free tier)
- **Per-key or per-account**: Per key — more keys = more throughput
- **Our config**: 60 RPM per workspace (per_request mode)
- **With N keys**: Effective per-key rate is (total RPM across workspaces) / N
- **Headroom**: Comfortable — even 10 active workspaces at full rate = 600 RPM / 3 keys = 200 RPM per key, under the 300 RPM API limit
```
This summary helps reviewers verify that the pricing and rate limiting are well-calibrated and surfaces any risks that need monitoring.
## Checklist
- [ ] Provider added to `BYOKProviderId` in `tools/types.ts`
- [ ] Provider added to `byokProviderIdSchema` enum in `lib/api/contracts/byok-keys.ts`
- [ ] API pricing docs researched — understand per-unit cost and whether the API reports cost in responses
- [ ] API rate limits researched — understand RPM/TPM limits, per-key vs per-account, and plan tiers
- [ ] `hosting` config added to the tool with `envKeyPrefix`, `apiKeyParam`, `byokProviderId`, `pricing`, and `rateLimit`
- [ ] `getCost` throws if required cost data is missing from the response
- [ ] Cost data captured in `transformResponse` or `postProcess` if API provides it
- [ ] `hideWhenHosted: true` added to the API key subblock in the block config
- [ ] Provider entry added to the BYOK settings UI with icon and description
- [ ] Env vars documented: `{PREFIX}_COUNT` and `{PREFIX}_1..N`
- [ ] Pricing and throttling summary provided to reviewer
+818
View File
@@ -0,0 +1,818 @@
---
description: Add a complete integration to Sim (tools, block, icon, registration)
argument-hint: <service-name> [api-docs-url]
---
# Add Integration Skill
You are an expert at adding complete integrations to Sim. This skill orchestrates the full process of adding a new service integration.
## Overview
Adding an integration involves these steps in order:
1. **Research** - Read the service's API documentation
2. **Create Tools** - Build tool configurations for each API operation
3. **Create Block** - Build the block UI configuration
4. **Add Icon** - Add the service's brand icon
5. **Create Triggers** (optional) - If the service supports webhooks
6. **Register** - Register tools, block, and triggers in their registries
7. **Generate Docs** - Run the docs generation script
## Step 1: Research the API
Before writing any code:
1. Use Context7 to find official documentation: `mcp__context7__resolve-library-id`, then fetch with `mcp__context7__query-docs`
2. Or use WebFetch to read API docs directly
3. Identify:
- Authentication method (OAuth, API Key, both)
- Available operations (CRUD, search, etc.)
- Required vs optional parameters
- Response structures
## Step 2: Create Tools
### Directory Structure
```
apps/sim/tools/{service}/
├── index.ts # Barrel exports
├── types.ts # TypeScript interfaces
├── {action1}.ts # Tool for action 1
├── {action2}.ts # Tool for action 2
└── ...
```
### Key Patterns
**types.ts:**
```typescript
import type { ToolResponse } from '@/tools/types'
export interface {Service}{Action}Params {
accessToken: string // For OAuth services
// OR
apiKey: string // For API key services
requiredParam: string
optionalParam?: string
}
export interface {Service}Response extends ToolResponse {
output: {
// Define output structure
}
}
```
**Tool file pattern:**
```typescript
export const {service}{Action}Tool: ToolConfig<Params, Response> = {
id: '{service}_{action}',
name: '{Service} {Action}',
description: '...',
version: '1.0.0',
oauth: { required: true, provider: '{service}' }, // If OAuth
params: {
accessToken: { type: 'string', required: true, visibility: 'hidden', description: '...' },
// ... other params
},
request: { url, method, headers, body },
transformResponse: async (response) => {
const data = await response.json()
return {
success: true,
output: {
field: data.field ?? null, // Always handle nullables
},
}
},
outputs: { /* ... */ },
}
```
### Critical Rules
- `visibility: 'hidden'` for OAuth tokens
- `visibility: 'user-only'` for API keys and user credentials
- `visibility: 'user-or-llm'` for operation parameters
- Always use `?? null` for nullable API response fields
- Always use `?? []` for optional array fields
- Set `optional: true` for outputs that may not exist
- Never output raw JSON dumps - extract meaningful fields
- When using `type: 'json'` and you know the object shape, define `properties` with the inner fields so downstream consumers know the structure. Only use bare `type: 'json'` when the shape is truly dynamic
## Step 3: Create Block
### File Location
`apps/sim/blocks/blocks/{service}.ts`
### Block Structure
```typescript
import { {Service}Icon } from '@/components/icons'
import type { BlockConfig } from '@/blocks/types'
import { AuthMode, IntegrationType } from '@/blocks/types'
import { getScopesForService } from '@/lib/oauth/utils'
export const {Service}Block: BlockConfig = {
type: '{service}',
name: '{Service}',
description: '...',
longDescription: '...',
docsLink: 'https://docs.sim.ai/integrations/{service}',
category: 'tools',
integrationType: IntegrationType.X, // Primary category (see IntegrationType enum)
tags: ['oauth', 'api'], // Cross-cutting tags (see IntegrationTag type)
bgColor: '#HEXCOLOR',
icon: {Service}Icon,
authMode: AuthMode.OAuth, // or AuthMode.ApiKey
subBlocks: [
// Operation dropdown
{
id: 'operation',
title: 'Operation',
type: 'dropdown',
options: [
{ label: 'Operation 1', id: 'action1' },
{ label: 'Operation 2', id: 'action2' },
],
value: () => 'action1',
},
// Credential field
{
id: 'credential',
title: '{Service} Account',
type: 'oauth-input',
serviceId: '{service}',
requiredScopes: getScopesForService('{service}'),
required: true,
},
// Conditional fields per operation
// ...
],
tools: {
access: ['{service}_action1', '{service}_action2'],
config: {
tool: (params) => `{service}_${params.operation}`,
},
},
outputs: { /* ... */ },
}
```
### Key SubBlock Patterns
**Condition-based visibility:**
```typescript
{
id: 'resourceId',
title: 'Resource ID',
type: 'short-input',
condition: { field: 'operation', value: ['read', 'update', 'delete'] },
required: { field: 'operation', value: ['read', 'update', 'delete'] },
}
```
**DependsOn for cascading selectors:**
```typescript
{
id: 'project',
type: 'project-selector',
dependsOn: ['credential'],
},
{
id: 'issue',
type: 'file-selector',
dependsOn: ['credential', 'project'],
}
```
**Basic/Advanced mode for dual UX:**
```typescript
// Basic: Visual selector
{
id: 'channel',
type: 'channel-selector',
mode: 'basic',
canonicalParamId: 'channel',
dependsOn: ['credential'],
},
// Advanced: Manual input
{
id: 'channelId',
type: 'short-input',
mode: 'advanced',
canonicalParamId: 'channel',
}
```
**Critical Canonical Param Rules:**
- `canonicalParamId` must NOT match any subblock's `id` in the block
- `canonicalParamId` must be unique per operation/condition context
- Only use `canonicalParamId` to link basic/advanced alternatives for the same logical parameter
- `mode` only controls UI visibility, NOT serialization. Without `canonicalParamId`, both basic and advanced field values would be sent
- Every subblock `id` must be unique within the block. Duplicate IDs cause conflicts even with different conditions
- **Required consistency:** If one subblock in a canonical group has `required: true`, ALL subblocks in that group must have `required: true` (prevents bypassing validation by switching modes)
- **Inputs section:** Must list canonical param IDs (e.g., `fileId`), NOT raw subblock IDs (e.g., `fileSelector`, `manualFileId`)
- **Params function:** Must use canonical param IDs, NOT raw subblock IDs (raw IDs are deleted after canonical transformation)
### BlockMeta (Required)
Export a `{Service}BlockMeta` in the same file as the block — **minimum 7 templates**. See `add-block.md` → "BlockMeta (Required)" for valid `modules` and `category` values and the full pattern.
```typescript
export const {Service}BlockMeta = {
tags: ['tag1', 'tag2'],
templates: [
{
icon: {Service}Icon,
title: '{Service} <use-case>',
prompt: 'Build a workflow that...', // concrete trigger → transformation → output
modules: ['agent', 'workflows'],
category: 'operations',
tags: ['automation'],
alsoIntegrations: ['slack'], // when the prompt references another service
},
// ... at least 6 more
],
} as const satisfies BlockMeta
```
## Step 4: Add Icon
### File Location
`apps/sim/components/icons.tsx`
### Pattern
```typescript
export function {Service}Icon(props: SVGProps<SVGSVGElement>) {
return (
<svg
{...props}
viewBox="0 0 24 24"
fill="none"
xmlns="http://www.w3.org/2000/svg"
>
{/* SVG paths from user-provided SVG */}
</svg>
)
}
```
### Getting Icons
**Do NOT search for icons yourself.** At the end of implementation, ask the user to provide the SVG:
```
I've completed the integration. Before I can add the icon, please provide the SVG for {Service}.
You can usually find this in the service's brand/press kit page, or copy it from their website.
Paste the SVG code here and I'll convert it to a React component.
```
Once the user provides the SVG:
1. Extract the SVG paths/content
2. Create a React component that spreads props
3. Ensure viewBox is preserved from the original SVG
### Theme-safety (bare rendering) — REQUIRED
The icon renders both inside its colored `bgColor` tile AND "bare" (no tile) on a
neutral page — e.g. the home **Suggested actions** list — in both light and dark
mode. A monochrome logo whose paths hardcode a single near-white or near-black
fill is invisible bare on the matching background (white-on-white in light mode,
black-on-black in dark mode).
Rules when adding the SVG:
- **Monochrome logos** (a single white or black mark): draw the shape with
`fill='currentColor'`, not `fill='#fff'` / `fill='#000000'`. It then inherits
white inside dark tiles, near-black inside light tiles (via
`getTileIconColorClass`), and the theme-aware `var(--text-icon)` bare — legible
everywhere. Do NOT set `iconColor` for these.
- **Multi-color brand logos** (their own vivid fills): keep the hardcoded fills.
They read on any background. Only set `iconColor` (a vivid brand hex, never a
near-black/near-white tile color) if the bare icon should adopt a brand tint.
- A large white shape with a tiny vivid accent (e.g. a logo where the body is the
white negative space) still vanishes bare — convert the body to `currentColor`.
Verify with `bun run check:bare-icons` (also runs in CI). It flags purely
monochrome hazards; for partial-accent logos, eyeball the suggested-actions list
in both light and dark mode.
## Step 5: Create Triggers (Optional)
If the service supports webhooks, create triggers using the generic `buildTriggerSubBlocks` helper.
### Directory Structure
```
apps/sim/triggers/{service}/
├── index.ts # Barrel exports
├── utils.ts # Trigger options, setup instructions, extra fields
├── {event_a}.ts # Primary trigger (includes dropdown)
├── {event_b}.ts # Secondary triggers (no dropdown)
└── webhook.ts # Generic webhook (optional)
```
### Key Pattern
```typescript
import { buildTriggerSubBlocks } from '@/triggers'
import { {service}TriggerOptions, {service}SetupInstructions, build{Service}ExtraFields } from './utils'
// Primary trigger - includeDropdown: true
export const {service}EventATrigger: TriggerConfig = {
id: '{service}_event_a',
subBlocks: buildTriggerSubBlocks({
triggerId: '{service}_event_a',
triggerOptions: {service}TriggerOptions,
includeDropdown: true, // Only for primary trigger!
setupInstructions: {service}SetupInstructions('Event A'),
extraFields: build{Service}ExtraFields('{service}_event_a'),
}),
// ...
}
// Secondary triggers - no dropdown
export const {service}EventBTrigger: TriggerConfig = {
id: '{service}_event_b',
subBlocks: buildTriggerSubBlocks({
triggerId: '{service}_event_b',
triggerOptions: {service}TriggerOptions,
// No includeDropdown!
setupInstructions: {service}SetupInstructions('Event B'),
extraFields: build{Service}ExtraFields('{service}_event_b'),
}),
// ...
}
```
### Connect to Block
```typescript
import { getTrigger } from '@/triggers'
export const {Service}Block: BlockConfig = {
triggers: {
enabled: true,
available: ['{service}_event_a', '{service}_event_b'],
},
subBlocks: [
// Tool fields...
...getTrigger('{service}_event_a').subBlocks,
...getTrigger('{service}_event_b').subBlocks,
],
}
```
See `/add-trigger` skill for complete documentation.
## Step 6: Register Everything
### Tools Registry (`apps/sim/tools/registry.ts`)
```typescript
// Add import (alphabetically)
import {
{service}Action1Tool,
{service}Action2Tool,
} from '@/tools/{service}'
// Add to tools object (alphabetically)
export const tools: Record<string, ToolConfig> = {
// ... existing tools ...
{service}_action1: {service}Action1Tool,
{service}_action2: {service}Action2Tool,
}
```
### Block Registry (`apps/sim/blocks/registry-maps.ts`)
The data maps (`BLOCK_REGISTRY` + `BLOCK_META_REGISTRY`) live in `registry-maps.ts`; `registry.ts` holds only the accessor functions. Add the import and an entry to each map alphabetically:
```typescript
// Add import (alphabetically)
import { {Service}Block, {Service}BlockMeta } from '@/blocks/blocks/{service}'
// Add to the config map (alphabetically)
export const BLOCK_REGISTRY: Record<string, BlockConfig> = {
// ... existing blocks ...
{service}: {Service}Block,
}
// Add to the catalog-meta map (alphabetically)
export const BLOCK_META_REGISTRY: Record<string, BlockMeta> = {
// ... existing metas ...
{service}: {Service}BlockMeta,
}
```
### Trigger Registry (`apps/sim/triggers/registry.ts`) - If triggers exist
```typescript
// Add import (alphabetically)
import {
{service}EventATrigger,
{service}EventBTrigger,
{service}WebhookTrigger,
} from '@/triggers/{service}'
// Add to TRIGGER_REGISTRY (alphabetically)
export const TRIGGER_REGISTRY: TriggerRegistry = {
// ... existing triggers ...
{service}_event_a: {service}EventATrigger,
{service}_event_b: {service}EventBTrigger,
{service}_webhook: {service}WebhookTrigger,
}
```
## Step 7: Generate Docs
Run the documentation generator (from `apps/sim`):
```bash
bun run generate-docs
```
This creates `apps/docs/content/docs/en/tools/{service}.mdx`
## V2 Integration Pattern
If creating V2 versions (API-aligned outputs):
1. **V2 Tools** - Add `_v2` suffix, version `2.0.0`, flat outputs
2. **V2 Block** - Add `_v2` type, use `createVersionedToolSelector`
3. **V1 Block** - Add `(Legacy)` to name, set `hideFromToolbar: true`
4. **Registry** - Register both versions
```typescript
// In registry
{service}: {Service}Block, // V1 (legacy, hidden)
{service}_v2: {Service}V2Block, // V2 (visible)
```
## Complete Checklist
### Tools
- [ ] Created `tools/{service}/` directory
- [ ] Created `types.ts` with all interfaces
- [ ] Created tool file for each operation
- [ ] All params have correct visibility
- [ ] All nullable fields use `?? null`
- [ ] All optional outputs have `optional: true`
- [ ] Created `index.ts` barrel export
- [ ] Registered all tools in `tools/registry.ts`
### Block
- [ ] Created `blocks/blocks/{service}.ts`
- [ ] Set `integrationType` to the correct `IntegrationType` enum value
- [ ] Set `tags` array with all applicable `IntegrationTag` values
- [ ] Defined operation dropdown with all operations
- [ ] Added credential field with `requiredScopes: getScopesForService('{service}')`
- [ ] Added conditional fields per operation
- [ ] Set up dependsOn for cascading selectors
- [ ] Configured tools.access with all tool IDs
- [ ] Configured tools.config.tool selector
- [ ] Defined outputs matching tool outputs
- [ ] Registered block + meta in `blocks/registry-maps.ts` (`BLOCK_REGISTRY` / `BLOCK_META_REGISTRY`)
- [ ] If triggers: set `triggers.enabled` and `triggers.available`
- [ ] If triggers: spread trigger subBlocks with `getTrigger()`
- [ ] Exported `{Service}BlockMeta` with at least 7 templates
### OAuth Scopes (if OAuth service)
- [ ] Defined scopes in `lib/oauth/oauth.ts` under `OAUTH_PROVIDERS`
- [ ] Added scope descriptions in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`
- [ ] Used `getCanonicalScopesForProvider()` in `auth.ts` (never hardcode)
- [ ] Used `getScopesForService()` in block `requiredScopes` (never hardcode)
### Icon
- [ ] Asked user to provide SVG
- [ ] Added icon to `components/icons.tsx`
- [ ] Icon spreads props correctly
- [ ] Monochrome marks use `fill='currentColor'` (not hardcoded white/black) so the icon renders bare in light AND dark mode — verified with `bun run check:bare-icons`
### Triggers (if service supports webhooks)
- [ ] Created `triggers/{service}/` directory
- [ ] Created `utils.ts` with options, instructions, and extra fields helpers
- [ ] Primary trigger uses `includeDropdown: true`
- [ ] Secondary triggers do NOT have `includeDropdown`
- [ ] All triggers use `buildTriggerSubBlocks` helper
- [ ] Created `index.ts` barrel export
- [ ] Registered all triggers in `triggers/registry.ts`
### Docs
- [ ] Ran `bun run scripts/generate-docs.ts`
- [ ] Verified docs file created
### Final Validation (Required)
- [ ] Read every tool file and cross-referenced inputs/outputs against the API docs
- [ ] Verified block subBlocks cover all required tool params with correct conditions
- [ ] Verified block outputs match what the tools actually return
- [ ] Verified `tools.config.params` correctly maps and coerces all param types
- [ ] `{Service}BlockMeta` exported with at least 7 templates, each having `icon`, `title`, `prompt`, `modules`, `category`, and `tags`
## Example Command
When the user asks to add an integration:
```
User: Add a Stripe integration
You: I'll add the Stripe integration. Let me:
1. First, research the Stripe API using Context7
2. Create the tools for key operations (payments, subscriptions, etc.)
3. Create the block with operation dropdown
4. Register everything
5. Generate docs
6. Ask you for the Stripe icon SVG
[Proceed with implementation...]
[After completing steps 1-5...]
I've completed the Stripe integration. Before I can add the icon, please provide the SVG for Stripe.
You can usually find this in the service's brand/press kit page, or copy it from their website.
Paste the SVG code here and I'll convert it to a React component.
```
## File Handling
When your integration handles file uploads or downloads, follow these patterns to work with `UserFile` objects consistently.
### What is a UserFile?
A `UserFile` is the standard file representation in Sim:
```typescript
interface UserFile {
id: string // Unique identifier
name: string // Original filename
url: string // Presigned URL for download
size: number // File size in bytes
type: string // MIME type (e.g., 'application/pdf')
base64?: string // Optional base64 content (if small file)
key?: string // Internal storage key
context?: object // Storage context metadata
}
```
### File Input Pattern (Uploads)
For tools that accept file uploads, **always route through an internal API endpoint** rather than calling external APIs directly. This ensures proper file content retrieval.
#### 1. Block SubBlocks for File Input
Use the basic/advanced mode pattern:
```typescript
// Basic mode: File upload UI
{
id: 'uploadFile',
title: 'File',
type: 'file-upload',
canonicalParamId: 'file', // Maps to 'file' param
placeholder: 'Upload file',
mode: 'basic',
multiple: false,
required: true,
condition: { field: 'operation', value: 'upload' },
},
// Advanced mode: Reference from previous block
{
id: 'fileRef',
title: 'File',
type: 'short-input',
canonicalParamId: 'file', // Same canonical param
placeholder: 'Reference file (e.g., {{file_block.output}})',
mode: 'advanced',
required: true,
condition: { field: 'operation', value: 'upload' },
},
```
**Critical:** `canonicalParamId` must NOT match any subblock `id`.
#### 2. Normalize File Input in Block Config
In `tools.config.tool`, use `normalizeFileInput` to handle all input variants:
```typescript
import { normalizeFileInput } from '@/blocks/utils'
tools: {
config: {
tool: (params) => {
// Normalize file from basic (uploadFile), advanced (fileRef), or legacy (fileContent)
const normalizedFile = normalizeFileInput(
params.uploadFile || params.fileRef || params.fileContent,
{ single: true }
)
if (normalizedFile) {
params.file = normalizedFile
}
return `{service}_${params.operation}`
},
},
}
```
#### 3. Create Internal API Route
Create `apps/sim/app/api/tools/{service}/{action}/route.ts`:
```typescript
import { createLogger } from '@sim/logger'
import { NextResponse, type NextRequest } from 'next/server'
import { {service}UploadContract } from '@/lib/api/contracts/tools/{service}'
import { parseRequest } from '@/lib/api/server'
import { checkInternalAuth } from '@/lib/auth/hybrid'
import { generateRequestId } from '@/lib/core/utils/request'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { type RawFileInput } from '@/lib/uploads/utils/file-schemas'
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
const logger = createLogger('{Service}UploadAPI')
export const POST = withRouteHandler(async (request: NextRequest) => {
const requestId = generateRequestId()
// Auth always runs BEFORE parseRequest — never validate untrusted input before authenticating.
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
if (!authResult.success) {
return NextResponse.json({ success: false, error: 'Unauthorized' }, { status: 401 })
}
const parsed = await parseRequest({service}UploadContract, request, {})
if (!parsed.success) return parsed.response
const data = parsed.data.body
let fileBuffer: Buffer
let fileName: string
// Prefer UserFile input, fall back to legacy base64
if (data.file) {
const userFiles = processFilesToUserFiles([data.file as RawFileInput], requestId, logger)
if (userFiles.length === 0) {
return NextResponse.json({ success: false, error: 'Invalid file' }, { status: 400 })
}
const userFile = userFiles[0]
fileBuffer = await downloadFileFromStorage(userFile, requestId, logger)
fileName = userFile.name
} else if (data.fileContent) {
// Legacy: base64 string (backwards compatibility)
fileBuffer = Buffer.from(data.fileContent, 'base64')
fileName = 'file'
} else {
return NextResponse.json({ success: false, error: 'File required' }, { status: 400 })
}
// Now call external API with fileBuffer
const response = await fetch('https://api.{service}.com/upload', {
method: 'POST',
headers: { Authorization: `Bearer ${data.accessToken}` },
body: new Uint8Array(fileBuffer), // Convert Buffer for fetch
})
// ... handle response
})
```
#### 4. Update Tool to Use Internal Route
```typescript
export const {service}UploadTool: ToolConfig<Params, Response> = {
id: '{service}_upload',
// ...
params: {
file: { type: 'file', required: false, visibility: 'user-or-llm' },
fileContent: { type: 'string', required: false, visibility: 'hidden' }, // Legacy
},
request: {
url: '/api/tools/{service}/upload', // Internal route
method: 'POST',
body: (params) => ({
accessToken: params.accessToken,
file: params.file,
fileContent: params.fileContent,
}),
},
}
```
### File Output Pattern (Downloads)
For tools that return files, use `FileToolProcessor` to store files and return `UserFile` objects.
#### In Tool transformResponse
```typescript
import { FileToolProcessor } from '@/executor/utils/file-tool-processor'
transformResponse: async (response, context) => {
const data = await response.json()
// Process file outputs to UserFile objects
const fileProcessor = new FileToolProcessor(context)
const file = await fileProcessor.processFileData({
data: data.content, // base64 or buffer
mimeType: data.mimeType,
filename: data.filename,
})
return {
success: true,
output: { file },
}
}
```
#### In API Route (for complex file handling)
```typescript
// Return file data that FileToolProcessor can handle
return NextResponse.json({
success: true,
output: {
file: {
data: base64Content,
mimeType: 'application/pdf',
filename: 'document.pdf',
},
},
})
```
### Key Helpers Reference
| Helper | Location | Purpose |
|--------|----------|---------|
| `normalizeFileInput` | `@/blocks/utils` | Normalize file params in block config |
| `processFilesToUserFiles` | `@/lib/uploads/utils/file-utils` | Convert raw inputs to UserFile[] |
| `downloadFileFromStorage` | `@/lib/uploads/utils/file-utils.server` | Get file Buffer from UserFile |
| `FileToolProcessor` | `@/executor/utils/file-tool-processor` | Process tool output files |
| `isUserFile` | `@/lib/core/utils/user-file` | Type guard for UserFile objects |
| `FileInputSchema` | `@/lib/uploads/utils/file-schemas` | Zod schema for file validation |
### Advanced Mode for Optional Fields
Optional fields that are rarely used should be set to `mode: 'advanced'` so they don't clutter the basic UI. Examples: pagination tokens, time range filters, sort order, max results, reply settings.
### WandConfig for Complex Inputs
Use `wandConfig` for fields that are hard to fill out manually:
- **Timestamps**: Use `generationType: 'timestamp'` to inject current date context into the AI prompt
- **JSON arrays**: Use `generationType: 'json-object'` for structured data
- **Complex queries**: Use a descriptive prompt explaining the expected format
```typescript
{
id: 'startTime',
title: 'Start Time',
type: 'short-input',
mode: 'advanced',
wandConfig: {
enabled: true,
prompt: 'Generate an ISO 8601 timestamp. Return ONLY the timestamp string.',
generationType: 'timestamp',
},
}
```
### OAuth Scopes (Centralized System)
Scopes are maintained in a single source of truth and reused everywhere:
1. **Define scopes** in `lib/oauth/oauth.ts` under `OAUTH_PROVIDERS[provider].services[service].scopes`
2. **Add descriptions** in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` for the OAuth modal UI
3. **Reference in auth.ts** using `getCanonicalScopesForProvider(providerId)` from `@/lib/oauth/utils`
4. **Reference in blocks** using `getScopesForService(serviceId)` from `@/lib/oauth/utils`
**Never hardcode scope arrays** in `auth.ts` or block `requiredScopes`. Always import from the centralized source.
```typescript
// In auth.ts (Better Auth config)
scopes: getCanonicalScopesForProvider('{service}'),
// In block credential sub-block
requiredScopes: getScopesForService('{service}'),
```
### Common Gotchas
1. **OAuth serviceId must match** - The `serviceId` in oauth-input must match the OAuth provider configuration
2. **All tool IDs MUST be snake_case** - `stripe_create_payment`, not `stripeCreatePayment`. This applies to tool `id` fields, registry keys, `tools.access` arrays, and `tools.config.tool` return values
3. **Block type is snake_case** - `type: 'stripe'`, not `type: 'Stripe'`
4. **Alphabetical ordering** - Keep imports and registry entries alphabetically sorted
5. **Required can be conditional** - Use `required: { field: 'op', value: 'create' }` instead of always true
6. **DependsOn clears options** - When a dependency changes, selector options are refetched
7. **Never pass Buffer directly to fetch** - Convert to `new Uint8Array(buffer)` for TypeScript compatibility
8. **Always handle legacy file params** - Keep hidden `fileContent` params for backwards compatibility
9. **Optional fields use advanced mode** - Set `mode: 'advanced'` on rarely-used optional fields
10. **Complex inputs need wandConfig** - Timestamps, JSON arrays, and other hard-to-type values should have `wandConfig` enabled
11. **Never hardcode scopes** - Use `getScopesForService()` in blocks and `getCanonicalScopesForProvider()` in auth.ts
12. **Always add scope descriptions** - New scopes must have entries in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`
+209
View File
@@ -0,0 +1,209 @@
---
description: Add a new LLM model to apps/sim/providers/models.ts with specs verified against the provider's live API docs (no hallucination)
argument-hint: <provider> <model-id> [docs-url]
---
# Add Model Skill
You add a new model entry to `apps/sim/providers/models.ts`. **Every numeric and capability claim MUST be derived from a live web fetch of the provider's official docs in this session.** Marketing emails, training data, and your prior knowledge are not sources of truth — they routinely hallucinate pricing, context windows, and capability lists.
## Hard rules (do not skip)
1. **Live-fetch or refuse.** Before writing the entry, you must successfully WebFetch the provider's official models/pricing page in this session. If you cannot reach an authoritative source for any field, **mark the field as UNVERIFIED in your report and ask the user before guessing**. Never fill in pricing or capabilities from memory.
2. **Two-source rule for pricing.** Cross-check input/output/cached pricing against at least one secondary source (OpenRouter, Artificial Analysis, CloudPrice, mem0, intuitionlabs). If sources disagree, the provider's own docs win — but flag the disagreement.
3. **Read the code before setting capability flags.** Capability flags are dead unless the provider's implementation under `apps/sim/providers/{provider}/` actually consumes them (see Consumption Matrix below). Setting a flag the provider ignores is a silent bug.
4. **Cite every fact.** Your final report must list the URL each value came from. No URL → not verified.
## Your Task
1. Identify provider and model id from user args
2. Live-fetch official docs + pricing page + capability/parameter pages + at least one secondary source
3. Apply the Consumption Matrix to know which capability flags are real
4. Read 2-3 sibling entries in `models.ts` and match their pattern exactly
5. Check the repo-side touchpoints that are NOT data-driven (hosted-key billing, tests, provider code)
6. Insert the entry, run `bun run lint`, print the verification report
## Step 1: Live source-of-truth lookup
In priority order — fetch all that exist for the provider:
| Provider | Models index | Pricing | Reasoning/parameter caveats |
|---|---|---|---|
| OpenAI | platform.openai.com/docs/models | openai.com/api/pricing | platform.openai.com/docs/guides/reasoning |
| Anthropic | docs.anthropic.com/en/docs/about-claude/models | anthropic.com/pricing | docs.anthropic.com/en/docs/build-with-claude/extended-thinking |
| Google (Gemini) | ai.google.dev/gemini-api/docs/models | ai.google.dev/pricing | ai.google.dev/gemini-api/docs/thinking |
| xAI | docs.x.ai/developers/models | docs.x.ai/developers/models (per-model detail page) | docs.x.ai/developers/model-capabilities/text/reasoning |
| Mistral | docs.mistral.ai/getting-started/models/models_overview | mistral.ai/pricing | n/a |
| DeepSeek | api-docs.deepseek.com/quick_start/pricing | same | api-docs.deepseek.com/guides/reasoning_model |
| Groq | console.groq.com/docs/models | groq.com/pricing | n/a |
| Cerebras | inference-docs.cerebras.ai/models | cerebras.ai/pricing | n/a |
Secondary verification (use at least one): `openrouter.ai/<provider>/<model>`, `artificialanalysis.ai/models/<model>`, `cloudprice.net/models/<provider>-<model>`.
Use a precise WebFetch prompt: *"Extract for {model_id}: exact model id string, context window in tokens, input price per 1M, cached input price per 1M, output price per 1M, max output tokens, supported reasoning effort levels, accepted parameters (temperature, top_p), release date. Do not fill in fields you cannot find."*
## Step 2: Consumption Matrix (which provider honors which capability)
| Capability | Honored by | Effect if set elsewhere |
|---|---|---|
| `temperature` | All providers (passed through if set) | Safe but inert on always-reasoning models that reject it |
| `toolUsageControl` | All providers (provider-level, not per-model) | n/a — set on `ProviderDefinition`, not models |
| `reasoningEffort` | `openai/core.ts`, `azure-openai`, `anthropic/core.ts` (mapped to thinking), `gemini/core.ts` | **Dead on xai, deepseek, mistral, groq, cerebras, openrouter, fireworks, bedrock, vertex** unless their core consumes it — re-grep before assuming |
| `verbosity` | `openai/core.ts`, `azure-openai/index.ts` only | Dead elsewhere |
| `thinking` | `anthropic/core.ts`, `gemini/core.ts` | Dead elsewhere |
| `nativeStructuredOutputs` | `anthropic/core.ts`, `fireworks/index.ts`, `openrouter/index.ts` | Dead on openai, xai, google, vertex, bedrock, azure-openai, deepseek, mistral, groq, cerebras |
| `maxOutputTokens` | Read by UI + executor for token estimation | Always meaningful — set if provider documents a cap |
| `computerUse` | `anthropic/core.ts` | Dead elsewhere |
| `deepResearch` | UI flag for routing to deep-research SKUs | Set only on actual deep-research model IDs |
| `memory: false` | Conversation persistence opt-out | Set only when model genuinely cannot maintain history (e.g., deep-research) |
**Always re-grep before relying on this table** — the codebase moves:
```bash
rg "reasoningEffort|reasoning_effort" apps/sim/providers/<provider>/
rg "verbosity" apps/sim/providers/<provider>/
rg "request\.thinking|thinking:" apps/sim/providers/<provider>/
rg "supportsNativeStructuredOutputs|nativeStructuredOutputs" apps/sim/providers/<provider>/
```
## Step 3: Match the provider's existing entry pattern
Open `apps/sim/providers/models.ts`, find `PROVIDER_DEFINITIONS[<provider>].models`, read 2-3 sibling entries. Match field order exactly:
```ts
{
id: '<exact-api-id>',
pricing: {
input: <number>,
cachedInput: <number>, // omit if provider doesn't offer caching
output: <number>,
updatedAt: '<today YYYY-MM-DD>',
},
capabilities: {
// only flags the provider actually consumes — see matrix
},
contextWindow: <tokens>,
releaseDate: '<YYYY-MM-DD>',
recommended: true, // only if new flagship; ask user before swapping
speedOptimized: true, // only on smallest/fastest tier
deprecated: true, // only on retired models
}
```
### Reseller providers (azure-openai, azure-anthropic, vertex, bedrock, openrouter)
Model id MUST be prefixed: `azure/`, `azure-anthropic/`, `vertex/`, `bedrock/`, `openrouter/`. Pricing usually mirrors the upstream provider but verify on the reseller's own pricing page.
### Insertion order
Within a family, newest first (matches existing convention: GPT-5.5 above GPT-5.4 above GPT-5.2). Across families, biggest/flagship at top of list.
### `recommended` / `speedOptimized`
- At most one or two `recommended: true` per provider — the current flagship(s).
- If you're adding a new flagship, ask the user before removing `recommended` from the previous flagship. Never silently flip it.
- `speedOptimized: true` only on the smallest/fastest tier (nano, flash-lite, haiku class).
## Step 4: Repo-side touchpoints beyond the entry
Adding the `models.ts` entry is most of the job because nearly every consumer is **data-driven** and picks the model up automatically: the ~40 query helpers in `models.ts` / `providers/utils.ts`, the public `/models` catalog (`app/(landing)/models/utils.ts` iterates `PROVIDER_DEFINITIONS`), the agent-block model dropdown, and copilot's `isKnownModelId` / `suggestModelIdsForUnknownModel` validation. The touchpoints below are the exceptions — they are **not** data-driven, so check each one.
### Hosted = auto-billed, by provider
`getHostedModels()` in `apps/sim/providers/models.ts` returns **every** model under `openai`, `anthropic`, and `google`:
```ts
export function getHostedModels(): string[] {
return [
...getProviderModels('openai'),
...getProviderModels('anthropic'),
...getProviderModels('google'),
]
}
```
So a model added to any of those three providers is **automatically served with Sim's rotating hosted key and billed** to the workspace via `shouldBillModelUsage()` (`providers/utils.ts`). Before you insert:
- **If the model should be BYOK-only / never-billed**, do NOT drop it under `openai`/`anthropic`/`google` as-is — that silently enrolls it in hosted billing. Confirm hosting/billing intent with the user. (Precedent: Ollama Cloud is a deliberately separate `isReseller` provider specifically to stay BYOK-only/never-billed.)
- **If the model should be hosted**, the deployment must actually have a key for it — the provider's `{PREFIX}_COUNT` / `{PREFIX}_1..N` env vars must be set, or hosted runs fail at execution time.
- State the hosted/billing status explicitly in the verification report.
### Tests with hardcoded model IDs
`bun run lint` does **not** run tests. A few tests assert specific model IDs and can break or need updating when you touch a hosted or flagship model:
- `apps/sim/providers/utils.test.ts` — asserts membership of `getHostedModels()` / `shouldBillModelUsage()`
- `apps/sim/providers/index.test.ts` and serializer tests — reference concrete model IDs
```bash
rg "<new-model-id>|getHostedModels|shouldBillModelUsage" apps/sim/providers/*.test.ts
```
If anything matches, run the affected provider tests and update assertions as needed.
### New API behavior is NOT data-driven
The Consumption Matrix (Step 2) tells you which capability *flags* are honored by existing provider code. But if the new model needs **net-new** request handling that the provider doesn't implement yet — a new beta header (e.g. Anthropic's `anthropic-beta` structured-outputs header in `anthropic/index.ts`), a new thinking/reasoning encoding, a Responses-API quirk — you must edit `apps/sim/providers/<provider>/core.ts` / `index.ts`. Setting a flag whose behavior isn't implemented is a silent no-op.
### Wrong family entirely?
- **Embedding or rerank model** → it does NOT go in the `models[]` array. Use `EMBEDDING_MODEL_PRICING` / `RERANK_MODEL_PRICING` in `models.ts` instead.
- **Brand-new provider** (not just a new model under an existing one) → much larger surface: add the id to `ProviderId` in `providers/types.ts`, a registry entry in `providers/registry.ts`, a provider implementation under `providers/<id>/`, an icon in `components/icons.tsx`, and the `PROVIDER_DEFINITIONS` block. That is beyond this skill — tell the user.
## Step 5: Write, lint
```bash
bun run lint
```
Lint must pass before reporting done. **If lint fails:** read the error, fix the syntax/typing issue in the entry you just wrote (do not delete the entry — it's the work product), re-run lint, and note the fix in a "Lint adjustments" line in the verification report. Never report done with lint failing.
## Step 6: Verification report (mandatory format)
End with this exact structure:
```markdown
### Verification — <model-id>
| Field | Value | Source URL | Status |
|---|---|---|---|
| `id` | `grok-4.3` | https://docs.x.ai/... | ✓ verified |
| `contextWindow` | 1,000,000 | https://docs.x.ai/... + https://openrouter.ai/... | ✓ verified (2 sources agree) |
| `input` | $1.25/M | https://docs.x.ai/... | ✓ verified |
| `cachedInput` | $0.20/M | https://cloudprice.net/... | ⚠️ single source |
| `output` | $2.50/M | https://docs.x.ai/... + https://openrouter.ai/... | ✓ verified |
| `capabilities.temperature` | `{ min: 0, max: 1 }` | matches sibling entries | — pattern-match only |
| `capabilities.reasoningEffort` | NOT SET | provider docs say API rejects it for this model | ✓ correctly omitted |
| `releaseDate` | 2026-04-30 | https://docs.x.ai/... announcement | ✓ verified |
| hosted/billing | BYOK-only (xai not in `getHostedModels`) | `providers/models.ts` | — confirmed intent |
**Disagreements**
- _none_ OR _OpenRouter says X, provider docs say Y — used Y per provider rule_
**Unverified fields**
- _none_ OR _<field>: could not find authoritative source — left as <X> based on sibling pattern; please confirm_
```
If any row is ⚠️ single-source or "unverified," **state it plainly to the user and ask whether to proceed**. Do not silently merge.
## What to do if you cannot find a source
Omitting a field is **not the same as verifying it**. Any field you cannot confirm from a live fetch must be **both** omitted from the entry **and** listed as ❓ UNVERIFIED in the report's "Unverified fields" section, with the URLs you attempted. Then ask the user to confirm before merging.
- Pricing missing → do NOT guess. Omit `cachedInput`. Mark ❓ UNVERIFIED. Ask the user for the price or the docs URL.
- Context window missing → do NOT guess. Ask the user; mark ❓ UNVERIFIED.
- Release date missing → omit the field; mark ❓ UNVERIFIED in the report.
- Capability uncertain → omit the flag (safer than setting a dead/wrong one); mark ❓ UNVERIFIED so the user knows you didn't confirm it either way.
## Anti-patterns this skill exists to prevent
- ❌ Trusting a marketing email (xAI's grok-4.3 email claimed "3 reasoning efforts" but the API rejects `reasoning_effort` — verified by official docs only)
- ❌ Setting `nativeStructuredOutputs: true` on xai/openai/google (dead — only anthropic/fireworks/openrouter consume it)
- ❌ Setting `thinking` on non-Anthropic/non-Gemini providers
- ❌ Setting `verbosity` on anything other than OpenAI gpt-5.x
- ❌ Copying `pricing.updatedAt` from a sibling instead of using today's date
- ❌ Inventing a `cachedInput` price by dividing input by 4 (varies by provider — find an explicit number)
- ❌ Stamping `recommended: true` on the new model without removing it from the previous flagship
- ❌ Adding a BYOK-only model under `openai`/`anthropic`/`google` (silently enrolls it in hosted billing via `getHostedModels()`)
- ❌ Reporting "done" after only `bun run lint` when you touched a hosted (openai/anthropic/google) or flagship model with assertions in `providers/utils.test.ts`
- ❌ Reporting "done" with any UNVERIFIED row in the table
+321
View File
@@ -0,0 +1,321 @@
---
description: Create tool configurations for a Sim integration by reading API docs
argument-hint: <service-name> [api-docs-url]
---
# Add Tools Skill
You are an expert at creating tool configurations for Sim integrations. Your job is to read API documentation and create properly structured tool files.
## Your Task
When the user asks you to create tools for a service:
1. Use Context7 or WebFetch to read the service's API documentation
2. Create the tools directory structure
3. Generate properly typed tool configurations
## Directory Structure
Create files in `apps/sim/tools/{service}/`:
```
tools/{service}/
├── index.ts # Barrel export
├── types.ts # Parameter & response types
└── {action}.ts # Individual tool files (one per operation)
```
## Tool Configuration Structure
Every tool MUST follow this exact structure:
```typescript
import type { {ServiceName}{Action}Params } from '@/tools/{service}/types'
import type { ToolConfig } from '@/tools/types'
interface {ServiceName}{Action}Response {
success: boolean
output: {
// Define output structure here
}
}
export const {serviceName}{Action}Tool: ToolConfig<
{ServiceName}{Action}Params,
{ServiceName}{Action}Response
> = {
id: '{service}_{action}', // snake_case, matches tool name
name: '{Service} {Action}', // Human readable
description: 'Brief description', // One sentence
version: '1.0.0',
// OAuth config (if service uses OAuth)
oauth: {
required: true,
provider: '{service}', // Must match OAuth provider ID
},
params: {
// Hidden params (system-injected, only use hidden for oauth accessToken)
accessToken: {
type: 'string',
required: true,
visibility: 'hidden',
description: 'OAuth access token',
},
// User-only params (credentials, api key, IDs user must provide)
someId: {
type: 'string',
required: true,
visibility: 'user-only',
description: 'The ID of the resource',
},
// User-or-LLM params (everything else, can be provided by user OR computed by LLM)
query: {
type: 'string',
required: false, // Use false for optional
visibility: 'user-or-llm',
description: 'Search query',
},
},
request: {
url: (params) => `https://api.service.com/v1/resource/${params.id}`,
method: 'POST',
headers: (params) => ({
Authorization: `Bearer ${params.accessToken}`,
'Content-Type': 'application/json',
}),
body: (params) => ({
// Request body - only for POST/PUT/PATCH
// Trim ID fields to prevent copy-paste whitespace errors:
// userId: params.userId?.trim(),
}),
},
transformResponse: async (response: Response) => {
const data = await response.json()
return {
success: true,
output: {
// Map API response to output
// Use ?? null for nullable fields
// Use ?? [] for optional arrays
},
}
},
outputs: {
// Define each output field
},
}
```
## Critical Rules for Parameters
### Visibility Options
- `'hidden'` - System-injected (OAuth tokens, internal params). User never sees.
- `'user-only'` - User must provide (credentials, api keys, account-specific IDs)
- `'user-or-llm'` - User provides OR LLM can compute (search queries, content, filters, most fall into this category)
### Parameter Types
- `'string'` - Text values
- `'number'` - Numeric values
- `'boolean'` - True/false
- `'json'` - Complex objects (NOT 'object', use 'json')
- `'file'` - Single file
- `'file[]'` - Multiple files
### Required vs Optional
- Always explicitly set `required: true` or `required: false`
- Optional params should have `required: false`
## Critical Rules for Outputs
### Output Types
- `'string'`, `'number'`, `'boolean'` - Primitives
- `'json'` - Complex objects (use this, NOT 'object')
- `'array'` - Arrays with `items` property
- `'object'` - Objects with `properties` property
### Optional Outputs
Add `optional: true` for fields that may not exist in the response:
```typescript
closedAt: {
type: 'string',
description: 'When the issue was closed',
optional: true,
},
```
### Typed JSON Outputs
When using `type: 'json'` and you know the object shape in advance, **always define the inner structure** using `properties` so downstream consumers know what fields are available:
```typescript
// BAD: Opaque json with no info about what's inside
metadata: {
type: 'json',
description: 'Response metadata',
},
// GOOD: Define the known properties
metadata: {
type: 'json',
description: 'Response metadata',
properties: {
id: { type: 'string', description: 'Unique ID' },
status: { type: 'string', description: 'Current status' },
count: { type: 'number', description: 'Total count' },
},
},
```
For arrays of objects, define the item structure:
```typescript
items: {
type: 'array',
description: 'List of items',
items: {
type: 'object',
properties: {
id: { type: 'string', description: 'Item ID' },
name: { type: 'string', description: 'Item name' },
},
},
},
```
Only use bare `type: 'json'` without `properties` when the shape is truly dynamic or unknown.
## Critical Rules for transformResponse
### Handle Nullable Fields
ALWAYS use `?? null` for fields that may be undefined:
```typescript
transformResponse: async (response: Response) => {
const data = await response.json()
return {
success: true,
output: {
id: data.id,
title: data.title,
body: data.body ?? null, // May be undefined
assignee: data.assignee ?? null, // May be undefined
labels: data.labels ?? [], // Default to empty array
closedAt: data.closed_at ?? null, // May be undefined
},
}
}
```
### Never Output Raw JSON Dumps
DON'T do this:
```typescript
output: {
data: data, // BAD - raw JSON dump
}
```
DO this instead - extract meaningful fields:
```typescript
output: {
id: data.id,
name: data.name,
status: data.status,
metadata: {
createdAt: data.created_at,
updatedAt: data.updated_at,
},
}
```
## Types File Pattern
Create `types.ts` with interfaces for all params and responses:
```typescript
import type { ToolResponse } from '@/tools/types'
// Parameter interfaces
export interface {Service}{Action}Params {
accessToken: string
requiredField: string
optionalField?: string
}
// Response interfaces (extend ToolResponse)
export interface {Service}{Action}Response extends ToolResponse {
output: {
field1: string
field2: number
optionalField?: string | null
}
}
```
## Index.ts Barrel Export Pattern
```typescript
// Export all tools
export { serviceTool1 } from './{action1}'
export { serviceTool2 } from './{action2}'
// Export types
export * from './types'
```
## Registering Tools
After creating tools, remind the user to:
1. Import tools in `apps/sim/tools/registry.ts`
2. Add to the `tools` object with snake_case keys:
```typescript
import { serviceActionTool } from '@/tools/{service}'
export const tools = {
// ... existing tools ...
{service}_{action}: serviceActionTool,
}
```
## V2 Tool Pattern
If creating V2 tools (API-aligned outputs), use `_v2` suffix:
- Tool ID: `{service}_{action}_v2`
- Variable name: `{action}V2Tool`
- Version: `'2.0.0'`
- Outputs: Flat, API-aligned (no content/metadata wrapper)
## Naming Convention
All tool IDs MUST use `snake_case`: `{service}_{action}` (e.g., `x_create_tweet`, `slack_send_message`). Never use camelCase or PascalCase for tool IDs.
## Checklist Before Finishing
- [ ] All tool IDs use snake_case
- [ ] All params have explicit `required: true` or `required: false`
- [ ] All params have appropriate `visibility`
- [ ] All nullable response fields use `?? null`
- [ ] All optional outputs have `optional: true`
- [ ] No raw JSON dumps in outputs
- [ ] Types file has all interfaces
- [ ] Index.ts exports all tools
## Final Validation (Required)
After creating all tools, you MUST validate every tool before finishing:
1. **Read every tool file** you created — do not skip any
2. **Cross-reference with the API docs** to verify:
- All required params are marked `required: true`
- All optional params are marked `required: false`
- Param types match the API (string, number, boolean, json)
- Request URL, method, headers, and body match the API spec
- `transformResponse` extracts the correct fields from the API response
- All output fields match what the API actually returns
- No fields are missing from outputs that the API provides
- No extra fields are defined in outputs that the API doesn't return
3. **Verify consistency** across tools:
- Shared types in `types.ts` match all tools that use them
- Tool IDs in the barrel export match the tool file definitions
- Error handling is consistent (error checks, meaningful messages)
+497
View File
@@ -0,0 +1,497 @@
---
description: Create webhook or polling triggers for a Sim integration
argument-hint: <service-name>
---
# Add Trigger
You are an expert at creating webhook and polling triggers for Sim. You understand the trigger system, the generic `buildTriggerSubBlocks` helper, polling infrastructure, and how triggers connect to blocks.
## Your Task
1. Research what webhook events the service supports — if the service lacks reliable webhooks, use polling
2. Create the trigger files using the generic builder (webhook) or manual config (polling)
3. Create a provider handler (webhook) or polling handler (polling)
4. Register triggers and connect them to the block
## Directory Structure
```
apps/sim/triggers/{service}/
├── index.ts # Barrel exports
├── utils.ts # Service-specific helpers (options, instructions, extra fields, outputs)
├── {event_a}.ts # Primary trigger (includes dropdown)
├── {event_b}.ts # Secondary trigger (no dropdown)
└── webhook.ts # Generic webhook trigger (optional, for "all events")
apps/sim/lib/webhooks/
├── provider-subscription-utils.ts # Shared subscription helpers (getProviderConfig, getNotificationUrl)
├── providers/
│ ├── {service}.ts # Provider handler (auth, formatInput, matchEvent, subscriptions)
│ ├── types.ts # WebhookProviderHandler interface
│ ├── utils.ts # Shared helpers (createHmacVerifier, verifyTokenAuth, skipByEventTypes)
│ └── registry.ts # Handler map + default handler
```
## Step 1: Create `utils.ts`
This file contains all service-specific helpers used by triggers.
```typescript
import type { SubBlockConfig } from '@/blocks/types'
import type { TriggerOutput } from '@/triggers/types'
export const {service}TriggerOptions = [
{ label: 'Event A', id: '{service}_event_a' },
{ label: 'Event B', id: '{service}_event_b' },
]
export function {service}SetupInstructions(eventType: string): string {
const instructions = [
'Copy the <strong>Webhook URL</strong> above',
'Go to <strong>{Service} Settings > Webhooks</strong>',
`Select the <strong>${eventType}</strong> event type`,
'Paste the webhook URL and save',
'Click "Save" above to activate your trigger',
]
return instructions
.map((instruction, index) =>
`<div class="mb-3"><strong>${index + 1}.</strong> ${instruction}</div>`
)
.join('')
}
export function build{Service}ExtraFields(triggerId: string): SubBlockConfig[] {
return [
{
id: 'projectId',
title: 'Project ID (Optional)',
type: 'short-input',
placeholder: 'Leave empty for all projects',
mode: 'trigger',
condition: { field: 'selectedTriggerId', value: triggerId },
},
]
}
export function build{Service}Outputs(): Record<string, TriggerOutput> {
return {
eventType: { type: 'string', description: 'The type of event' },
resourceId: { type: 'string', description: 'ID of the affected resource' },
resource: {
id: { type: 'string', description: 'Resource ID' },
name: { type: 'string', description: 'Resource name' },
},
}
}
```
## Step 2: Create Trigger Files
**Primary trigger** — MUST include `includeDropdown: true`:
```typescript
import { {Service}Icon } from '@/components/icons'
import { buildTriggerSubBlocks } from '@/triggers'
import { build{Service}ExtraFields, build{Service}Outputs, {service}SetupInstructions, {service}TriggerOptions } from '@/triggers/{service}/utils'
import type { TriggerConfig } from '@/triggers/types'
export const {service}EventATrigger: TriggerConfig = {
id: '{service}_event_a',
name: '{Service} Event A',
provider: '{service}',
description: 'Trigger workflow when Event A occurs',
version: '1.0.0',
icon: {Service}Icon,
subBlocks: buildTriggerSubBlocks({
triggerId: '{service}_event_a',
triggerOptions: {service}TriggerOptions,
includeDropdown: true,
setupInstructions: {service}SetupInstructions('Event A'),
extraFields: build{Service}ExtraFields('{service}_event_a'),
}),
outputs: build{Service}Outputs(),
webhook: { method: 'POST', headers: { 'Content-Type': 'application/json' } },
}
```
**Secondary triggers** — NO `includeDropdown` (it's already in the primary):
```typescript
export const {service}EventBTrigger: TriggerConfig = {
// Same as above but: id: '{service}_event_b', no includeDropdown
}
```
## Step 3: Register and Wire
### `apps/sim/triggers/{service}/index.ts`
```typescript
export { {service}EventATrigger } from './event_a'
export { {service}EventBTrigger } from './event_b'
```
### `apps/sim/triggers/registry.ts`
```typescript
import { {service}EventATrigger, {service}EventBTrigger } from '@/triggers/{service}'
export const TRIGGER_REGISTRY: TriggerRegistry = {
// ... existing ...
{service}_event_a: {service}EventATrigger,
{service}_event_b: {service}EventBTrigger,
}
```
### Block file (`apps/sim/blocks/blocks/{service}.ts`)
Wire triggers into the block so the trigger UI appears and `generate-docs.ts` discovers them. Two changes are needed:
1. **Spread trigger subBlocks** at the end of the block's `subBlocks` array
2. **Add `triggers` property** after `outputs` with `enabled: true` and `available: [...]`
```typescript
import { getTrigger } from '@/triggers'
export const {Service}Block: BlockConfig = {
// ...
subBlocks: [
// Regular tool subBlocks first...
...getTrigger('{service}_event_a').subBlocks,
...getTrigger('{service}_event_b').subBlocks,
],
// ... tools, inputs, outputs ...
triggers: {
enabled: true,
available: ['{service}_event_a', '{service}_event_b'],
},
}
```
**Versioned blocks (V1 + V2):** Many integrations have a hidden V1 block and a visible V2 block. Where you add the trigger wiring depends on how V2 inherits from V1:
- **V2 uses `...V1Block` spread** (e.g., Google Calendar): Add trigger to V1 — V2 inherits both `subBlocks` and `triggers` automatically.
- **V2 defines its own `subBlocks`** (e.g., Google Sheets): Add trigger to V2 (the visible block). V1 is hidden and doesn't need it.
- **Single block, no V2** (e.g., Google Drive): Add trigger directly.
`generate-docs.ts` deduplicates by base type (first match wins). If V1 is processed first without triggers, the V2 triggers won't appear in `integrations.json`. Always verify by checking the output after running the script.
## Provider Handler
All provider-specific webhook logic lives in a single handler file: `apps/sim/lib/webhooks/providers/{service}.ts`.
### When to Create a Handler
| Behavior | Method | Examples |
|---|---|---|
| HMAC signature auth | `verifyAuth` via `createHmacVerifier` | Ashby, Jira, Linear, Typeform |
| Custom token auth | `verifyAuth` via `verifyTokenAuth` | Generic, Google Forms |
| Event filtering | `matchEvent` | GitHub, Jira, Attio, HubSpot |
| Idempotency dedup | `extractIdempotencyId` | Slack, Stripe, Linear, Jira |
| Custom input formatting | `formatInput` | Slack, Teams, Attio, Ashby |
| Auto webhook creation | `createSubscription` | Ashby, Grain, Calendly, Airtable |
| Auto webhook deletion | `deleteSubscription` | Ashby, Grain, Calendly, Airtable |
| Challenge/verification | `handleChallenge` | Slack, WhatsApp, Teams |
| Custom success response | `formatSuccessResponse` | Slack, Twilio Voice, Teams |
If none apply, you don't need a handler. The default handler provides bearer token auth.
### Example Handler
```typescript
import crypto from 'crypto'
import { createLogger } from '@sim/logger'
import { safeCompare } from '@/lib/core/security/encryption'
import type { EventMatchContext, FormatInputContext, FormatInputResult, WebhookProviderHandler } from '@/lib/webhooks/providers/types'
import { createHmacVerifier } from '@/lib/webhooks/providers/utils'
const logger = createLogger('WebhookProvider:{Service}')
function validate{Service}Signature(secret: string, signature: string, body: string): boolean {
if (!secret || !signature || !body) return false
const computed = crypto.createHmac('sha256', secret).update(body, 'utf8').digest('hex')
return safeCompare(computed, signature)
}
export const {service}Handler: WebhookProviderHandler = {
verifyAuth: createHmacVerifier({
configKey: 'webhookSecret',
headerName: 'X-{Service}-Signature',
validateFn: validate{Service}Signature,
providerLabel: '{Service}',
}),
async matchEvent({ body, requestId, providerConfig }: EventMatchContext) {
const triggerId = providerConfig.triggerId as string | undefined
if (triggerId && triggerId !== '{service}_webhook') {
const { is{Service}EventMatch } = await import('@/triggers/{service}/utils')
if (!is{Service}EventMatch(triggerId, body as Record<string, unknown>)) return false
}
return true
},
async formatInput({ body }: FormatInputContext): Promise<FormatInputResult> {
const b = body as Record<string, unknown>
return {
input: {
eventType: b.type,
resourceId: (b.data as Record<string, unknown>)?.id || '',
resource: b.data,
},
}
},
extractIdempotencyId(body: unknown) {
const obj = body as Record<string, unknown>
return obj.id && obj.type ? `${obj.type}:${obj.id}` : null
},
}
```
### Register the Handler
In `apps/sim/lib/webhooks/providers/registry.ts`:
```typescript
import { {service}Handler } from '@/lib/webhooks/providers/{service}'
const PROVIDER_HANDLERS: Record<string, WebhookProviderHandler> = {
// ... existing (alphabetical) ...
{service}: {service}Handler,
}
```
## Output Alignment (Critical)
There are two sources of truth that **MUST be aligned**:
1. **Trigger `outputs`** — schema defining what fields SHOULD be available (UI tag dropdown)
2. **`formatInput` on the handler** — implementation that transforms raw payload into actual data
If they differ: the tag dropdown shows fields that don't exist, or actual data has fields users can't discover.
**Rules for `formatInput`:**
- Return `{ input: { ... } }` where inner keys match trigger `outputs` exactly
- Return `{ input: ..., skip: { message: '...' } }` to skip execution
- No wrapper objects or duplication
- Use `null` for missing optional data
## Automatic Webhook Registration
If the service API supports programmatic webhook creation, implement `createSubscription` and `deleteSubscription` on the handler. The orchestration layer calls these automatically — **no code touches `route.ts`, `provider-subscriptions.ts`, or `deploy.ts`**.
```typescript
import { getNotificationUrl, getProviderConfig } from '@/lib/webhooks/provider-subscription-utils'
import type { DeleteSubscriptionContext, SubscriptionContext, SubscriptionResult } from '@/lib/webhooks/providers/types'
export const {service}Handler: WebhookProviderHandler = {
async createSubscription(ctx: SubscriptionContext): Promise<SubscriptionResult | undefined> {
const config = getProviderConfig(ctx.webhook)
const apiKey = config.apiKey as string
if (!apiKey) throw new Error('{Service} API Key is required.')
const res = await fetch('https://api.{service}.com/webhooks', {
method: 'POST',
headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json' },
body: JSON.stringify({ url: getNotificationUrl(ctx.webhook) }),
})
if (!res.ok) throw new Error(`{Service} error: ${res.status}`)
const { id } = (await res.json()) as { id: string }
return { providerConfigUpdates: { externalId: id } }
},
async deleteSubscription(ctx: DeleteSubscriptionContext): Promise<void> {
const config = getProviderConfig(ctx.webhook)
const { apiKey, externalId } = config as { apiKey?: string; externalId?: string }
if (!apiKey || !externalId) return
await fetch(`https://api.{service}.com/webhooks/${externalId}`, {
method: 'DELETE',
headers: { Authorization: `Bearer ${apiKey}` },
}).catch(() => {})
},
}
```
**Key points:**
- Throw from `createSubscription` — orchestration rolls back the DB webhook
- Never throw from `deleteSubscription` — log non-fatally
- Return `{ providerConfigUpdates: { externalId } }` — orchestration merges into `providerConfig`
- Add `apiKey` field to `build{Service}ExtraFields` with `password: true`
## Trigger Outputs Schema
Trigger outputs use the same schema as block outputs (NOT tool outputs).
**Supported:** `type` + `description` for leaf fields, nested objects for complex data.
**NOT supported:** `optional: true`, `items` (those are tool-output-only features).
```typescript
export function buildOutputs(): Record<string, TriggerOutput> {
return {
eventType: { type: 'string', description: 'Event type' },
timestamp: { type: 'string', description: 'When it occurred' },
payload: { type: 'json', description: 'Full event payload' },
resource: {
id: { type: 'string', description: 'Resource ID' },
name: { type: 'string', description: 'Resource name' },
},
}
}
```
## Polling Triggers
Use polling when the service lacks reliable webhooks (e.g., Google Sheets, Google Drive, Google Calendar, Gmail, RSS, IMAP). Polling triggers do NOT use `buildTriggerSubBlocks` — they define subBlocks manually.
### Directory Structure
```
apps/sim/triggers/{service}/
├── index.ts # Barrel export
└── poller.ts # TriggerConfig with polling: true
apps/sim/lib/webhooks/polling/
└── {service}.ts # PollingProviderHandler implementation
```
### Polling Handler (`apps/sim/lib/webhooks/polling/{service}.ts`)
```typescript
import { pollingIdempotency } from '@/lib/core/idempotency/service'
import type { PollingProviderHandler, PollWebhookContext } from '@/lib/webhooks/polling/types'
import { markWebhookFailed, markWebhookSuccess, resolveOAuthCredential, updateWebhookProviderConfig } from '@/lib/webhooks/polling/utils'
import { processPolledWebhookEvent } from '@/lib/webhooks/processor'
export const {service}PollingHandler: PollingProviderHandler = {
provider: '{service}',
label: '{Service}',
async pollWebhook(ctx: PollWebhookContext): Promise<'success' | 'failure'> {
const { webhookData, workflowData, requestId, logger } = ctx
const webhookId = webhookData.id
try {
// For OAuth services:
const accessToken = await resolveOAuthCredential(webhookData, '{service}', requestId)
const config = webhookData.providerConfig as unknown as {Service}WebhookConfig
// First poll: seed state, emit nothing
if (!config.lastCheckedTimestamp) {
await updateWebhookProviderConfig(webhookId, { lastCheckedTimestamp: new Date().toISOString() }, logger)
await markWebhookSuccess(webhookId, logger)
return 'success'
}
// Fetch changes since last poll, process with idempotency
// ...
await markWebhookSuccess(webhookId, logger)
return 'success'
} catch (error) {
logger.error(`[${requestId}] Error processing {service} webhook ${webhookId}:`, error)
await markWebhookFailed(webhookId, logger)
return 'failure'
}
},
}
```
**Key patterns:**
- First poll seeds state and emits nothing (avoids flooding with existing data)
- Use `pollingIdempotency.executeWithIdempotency(provider, key, callback)` for dedup
- Use `processPolledWebhookEvent(webhookData, workflowData, payload, requestId)` to fire the workflow
- Use `updateWebhookProviderConfig(webhookId, partialConfig, logger)` for read-merge-write on state
- Use the latest server-side timestamp from API responses (not wall clock) to avoid clock skew
### Trigger Config (`apps/sim/triggers/{service}/poller.ts`)
```typescript
import { {Service}Icon } from '@/components/icons'
import type { TriggerConfig } from '@/triggers/types'
export const {service}PollingTrigger: TriggerConfig = {
id: '{service}_poller',
name: '{Service} Trigger',
provider: '{service}',
description: 'Triggers when ...',
version: '1.0.0',
icon: {Service}Icon,
polling: true, // REQUIRED — routes to polling infrastructure
subBlocks: [
{ id: 'triggerCredentials', type: 'oauth-input', title: 'Credentials', serviceId: '{service}', requiredScopes: [], required: true, mode: 'trigger' },
// ... service-specific config fields (dropdowns, inputs, switches) ...
{ id: 'triggerInstructions', type: 'text', title: 'Setup Instructions', hideFromPreview: true, mode: 'trigger', defaultValue: '...' },
],
outputs: {
// Must match the payload shape from processPolledWebhookEvent
},
}
```
### Registration (3 places)
1. **`apps/sim/triggers/constants.ts`** — add provider to `POLLING_PROVIDERS` Set
2. **`apps/sim/lib/webhooks/polling/registry.ts`** — import handler, add to `POLLING_HANDLERS`
3. **`apps/sim/triggers/registry.ts`** — import trigger config, add to `TRIGGER_REGISTRY`
### Helm Cron Job
Add to `helm/sim/values.yaml` under the existing polling cron jobs:
```yaml
{service}WebhookPoll:
schedule: "*/1 * * * *"
concurrencyPolicy: Forbid
url: "http://sim:3000/api/webhooks/poll/{service}"
```
### Reference Implementations
- Simple: `apps/sim/lib/webhooks/polling/rss.ts` + `apps/sim/triggers/rss/poller.ts`
- Complex (OAuth, attachments): `apps/sim/lib/webhooks/polling/gmail.ts` + `apps/sim/triggers/gmail/poller.ts`
- Cursor-based (changes API): `apps/sim/lib/webhooks/polling/google-drive.ts`
- Timestamp-based: `apps/sim/lib/webhooks/polling/google-calendar.ts`
## Checklist
### Trigger Definition
- [ ] Created `utils.ts` with options, instructions, extra fields, and output builders
- [ ] Primary trigger has `includeDropdown: true`; secondary triggers do NOT
- [ ] All triggers use `buildTriggerSubBlocks` helper
- [ ] Created `index.ts` barrel export
### Registration
- [ ] All triggers in `triggers/registry.ts``TRIGGER_REGISTRY`
- [ ] Block has `triggers.enabled: true` and lists all trigger IDs in `triggers.available`
- [ ] Block spreads all trigger subBlocks: `...getTrigger('id').subBlocks`
### Provider Handler (if needed)
- [ ] Handler file at `apps/sim/lib/webhooks/providers/{service}.ts`
- [ ] Registered in `providers/registry.ts` (alphabetical)
- [ ] Signature validator is a private function inside the handler file
- [ ] `formatInput` output keys match trigger `outputs` exactly
- [ ] Event matching uses dynamic `await import()` for trigger utils
### Auto Registration (if supported)
- [ ] `createSubscription` and `deleteSubscription` on the handler
- [ ] NO changes to `route.ts`, `provider-subscriptions.ts`, or `deploy.ts`
- [ ] API key field uses `password: true`
### Polling Trigger (if applicable)
- [ ] Handler implements `PollingProviderHandler` at `lib/webhooks/polling/{service}.ts`
- [ ] Trigger config has `polling: true` and defines subBlocks manually (no `buildTriggerSubBlocks`)
- [ ] Provider string matches across: trigger config, handler, `POLLING_PROVIDERS`, polling registry
- [ ] First poll seeds state and emits nothing
- [ ] Added provider to `POLLING_PROVIDERS` in `triggers/constants.ts`
- [ ] Added handler to `POLLING_HANDLERS` in `lib/webhooks/polling/registry.ts`
- [ ] Added cron job to `helm/sim/values.yaml`
- [ ] Payload shape matches trigger `outputs` schema
### Testing
- [ ] `bun run type-check` passes
- [ ] Manually verify output keys match trigger `outputs` keys
- [ ] Trigger UI shows correctly in the block
+26
View File
@@ -0,0 +1,26 @@
---
description: Run all code quality skills in sequence — effects, memo, callbacks, state, React Query, emcn design review, and url-state
argument-hint: [scope] [fix=true|false]
---
# Cleanup
Arguments:
- scope: what to review (default: your current changes). Examples: "diff to main", "PR #123", "src/components/", "whole codebase"
- fix: whether to apply fixes (default: true). Set to false to only propose changes.
User arguments: $ARGUMENTS
## Steps
Run each of these skills in order on the specified scope, passing through the scope and fix arguments. After each skill completes, move to the next. Do not skip any.
1. `/you-might-not-need-an-effect $ARGUMENTS`
2. `/you-might-not-need-a-memo $ARGUMENTS`
3. `/you-might-not-need-a-callback $ARGUMENTS`
4. `/you-might-not-need-state $ARGUMENTS`
5. `/react-query-best-practices $ARGUMENTS`
6. `/emcn-design-review $ARGUMENTS`
7. `/you-might-not-need-url-state $ARGUMENTS`
After all skills have run, output a summary of what was found and fixed (or proposed) across all seven passes.
+12
View File
@@ -0,0 +1,12 @@
---
description: Spawn task agents to explore a given area of interest in the codebase
argument-hint: <area-of-interest>
---
Based on the given area of interest, please:
1. Dig around the codebase in terms of that given area of interest, gather general information such as keywords and architecture overview.
2. Spawn off n=10 (unless specified otherwise) task agents to dig deeper into the codebase in terms of that given area of interest, some of them should be out of the box for variance.
3. Once the task agents are done, use the information to do what the user wants.
If user is in plan mode, use the information to create the plan.
+84
View File
@@ -0,0 +1,84 @@
---
description: Review UI code for alignment with the emcn design system — components, tokens, patterns, and conventions
argument-hint: [scope] [fix=true|false]
---
# EMCN Design Review
Arguments:
- scope: what to review (default: your current changes). Examples: "diff to main", "PR #123", "src/components/", "whole codebase"
- fix: whether to apply fixes (default: true). Set to false to only propose changes.
User arguments: $ARGUMENTS
## Context
This codebase uses **emcn**, a custom component library built on Radix UI primitives with CVA variants and CSS variable design tokens. All UI must use emcn components and tokens.
## Steps
1. Read the emcn public barrel at `apps/sim/components/emcn/index.ts` (re-exports components, Calendar, Table*, and icons) to know what's available; for the full icon set read `apps/sim/components/emcn/icons/index.ts`
2. Read `apps/sim/app/_styles/globals.css` for CSS variable tokens
3. Analyze the specified scope against every rule below
4. If fix=true, apply the fixes. If fix=false, propose the fixes without applying.
---
## Imports
- Import from `@/components/emcn` barrel, never subpaths
- Icons from `@/components/emcn/icons` or `lucide-react`
- Use `cn` from `@/lib/core/utils/cn` for conditional classes
## Design Tokens
Use CSS variable pattern (`text-[var(--text-primary)]`), never Tailwind semantics (`text-muted-foreground`) or hardcoded colors (`text-gray-500`, `#333`).
**Text**: `--text-primary`, `--text-secondary`, `--text-tertiary`, `--text-muted`, `--text-body` (canonical value text), `--text-icon`, `--text-placeholder`, `--text-subtle`, `--text-inverse`, `--text-error`
**Surfaces**: `--bg`, `--surface-1` through `--surface-7`, `--surface-hover`, `--surface-active`
**Borders**: `--border`, `--border-1`, `--border-muted`
**Brand/accent**: `--brand-secondary`, `--brand-accent`
**Z-Index**: `--z-dropdown` (100), `--z-modal` (200), `--z-popover` (300), `--z-tooltip` (400), `--z-toast` (500)
**Shadows**: `shadow-subtle`, `shadow-medium`, `shadow-overlay`, `shadow-card`
**Badges**: `--badge-*` semantic families (success/error/gray/blue/purple/orange/amber/teal/cyan/pink, each with `-bg`/`-text`)
## Buttons
Intent-to-variant mapping (read the actual `buttonVariants` in `apps/sim/components/emcn/components/button/button.tsx` for the full variant set — it exposes more than listed here):
| Action | Variant |
|--------|---------|
| Toolbar, icon-only | `ghost` |
| Create, save, submit | `primary` |
| Cancel, close | `default` |
| Delete, remove | `destructive` |
| Selected state | `active` |
| Toggle | `outline` |
## Delete/Remove Confirmations
`ChipModal` `size='sm'`, title "Delete/Remove {ItemType}", destructive confirm button, plain Cancel (follow the chip footer layout in `.claude/rules/emcn-components.md`). Use `text-[var(--text-error)]` for irreversible warnings.
## Toast
`toast.success()`, `toast.error()`, `toast()` from `@/components/emcn`. Never custom notification UI.
## Badges
`red`=error/failed, `gray-secondary`=metadata/roles, `type`=type annotations, `green`=success/active, `gray`=neutral, `amber`=processing, `orange`=paused, `blue`=info. Use `dot` prop for status indicators.
## Icons
Default: `size-[14px]`. Color: `text-[var(--text-icon)]`. Scale: 14px > 16px > 12px > 20px. Use the `size-*` shorthand — flag `h-[Npx] w-[Npx]` and `h-N w-N` pairs as refactor targets.
## Anti-patterns to flag
- Raw `<button>`/`<input>`, or legacy `Input`/`Textarea`/`Modal`, instead of the canonical chip components (`ChipInput`/`ChipTextarea`/`ChipModal`)
- Hand-rolled field rows inside a `ChipModalBody` instead of `ChipModalField`
- Hardcoded colors (`text-gray-*`, `#hex`, `rgb()`)
- Tailwind semantics (`text-muted-foreground`) instead of CSS variables
- Template literal className instead of `cn()`
- Inline styles for colors/static values (dynamic values OK)
- Importing from emcn subpaths instead of barrel
- Arbitrary z-index instead of tokens
- Wrong button variant for action type
@@ -0,0 +1,54 @@
---
description: Audit React Query usage for best practices — key factories, staleTime, mutations, and server state ownership
argument-hint: [scope] [fix=true|false]
---
# React Query Best Practices
Arguments:
- scope: what to analyze (default: your current changes). Examples: "diff to main", "PR #123", "src/hooks/queries/", "whole codebase"
- fix: whether to apply fixes (default: true). Set to false to only propose changes.
User arguments: $ARGUMENTS
## Context
This codebase uses React Query (TanStack Query) as the single source of truth for all server state. All query hooks live in `hooks/queries/`. Zustand is used only for client-only UI state. Server data must never be duplicated into useState or Zustand outside of mutation callbacks that coordinate cross-store state.
## References
Read these before analyzing:
1. https://tkdodo.eu/blog/practical-react-query — foundational defaults, custom hooks, avoiding local state copies
2. https://tkdodo.eu/blog/effective-react-query-keys — key factory pattern, hierarchical keys, fuzzy invalidation
3. https://tkdodo.eu/blog/react-query-as-a-state-manager — React Query IS your server state manager
## Rules to enforce
### Query key factories
- Every file in `hooks/queries/` must have a hierarchical key factory with an `all` root key
- Keys must include intermediate plural keys (`lists`, `details`) for prefix invalidation
- Key factories are colocated with their query hooks, not in a global keys file
### Query hooks
- Every `queryFn` must forward `signal` for request cancellation
- Every query must have an explicit `staleTime` (default 0 is almost never correct), assigned from a named exported constant — never an inline numeric literal. A server-side prefetch hydrating the same query key must import and reuse that constant instead of restating the number
- `keepPreviousData` / `placeholderData` only on variable-key queries (where params change), never on static keys
- Use `enabled` to prevent queries from running without required params
### Mutations
- Use `onSettled` (not `onSuccess`) for cache reconciliation — it fires on both success and error
- For optimistic updates: save previous data in `onMutate`, roll back in `onError`
- Use targeted invalidation (`entityKeys.lists()`) not broad (`entityKeys.all`) when possible
- Don't include mutation objects in `useCallback` deps — `.mutate()` is stable
### Server state ownership
- Never copy query data into useState. Use query data directly in components.
- Never copy query data into Zustand stores (exception: mutation callbacks that coordinate cross-store state like temp ID replacement)
- The query cache is not a local state manager — `setQueryData` is for optimistic updates only
- Forms are the one deliberate exception: copy server data into local form state with `staleTime: Infinity`
## Steps
1. Read the references above to understand the guidelines
2. Analyze the specified scope against the rules listed above
3. If fix=true, apply the fixes. If fix=false, propose the fixes without applying.
+84
View File
@@ -0,0 +1,84 @@
---
description: Commit, push, and open a PR to staging in one shot
argument-hint: [optional context or scope notes]
---
# Ship Command
You help ship code by creating commits, pushing to the remote branch, and creating PRs in the user's voice.
## Your Task
When the user runs `/ship`:
1. **Check git status** - See what files have changed
2. **Generate a commit message** following this format: `type(scope): description`
- Types: `fix`, `feat`, `improvement`, `chore`
- Scope: short identifier (e.g., `undo-redo`, `api`, `ui`)
- Keep it concise
3. **Run pre-ship checks** from the repo root before staging:
- `bun run lint` to fix formatting issues
- `bun run check:api-validation:strict` to catch boundary contract failures before CI
4. **Stage and commit** the changes with the generated message
5. **Push to origin** using the current branch name
6. **Create a PR** to staging with a description in the user's voice
## Commit Message Format
Based on the repo's commit history:
```
fix(scope): description for bug fixes
feat(scope): description for new features
improvement(scope): description for enhancements
chore(scope): description for maintenance
```
## PR Description Format
Use this exact template in the user's voice (concise, bullet points):
```markdown
## Summary
- bullet point describing what changed
- another bullet point if needed
## Type of Change
- [x] Bug fix (or appropriate type)
## Testing
Tested manually (or describe testing)
## Checklist
- [x] Code follows project style guidelines
- [x] Self-reviewed my changes
- [ ] Tests added/updated and passing
- [x] No new warnings introduced
- [x] I confirm that I have read and agree to the terms outlined in the [Contributor License Agreement (CLA)](./CONTRIBUTING.md#contributor-license-agreement-cla)
```
## PR Creation Command
Use this command structure:
```bash
gh pr create --base staging --title "COMMIT_MESSAGE" --body "PR_BODY"
```
## Important Notes
- Always confirm the commit message and PR description with the user before executing
- The PR should be created against `staging` branch
- Keep descriptions concise and in active voice
- Match the user's previous PR style: direct, no fluff, bullet points
- **DO NOT add "Co-Authored-By" lines to commits** - keep commit messages clean
## User's Voice Characteristics (based on previous PRs)
- Short, direct bullet points
- No unnecessary explanation
- "Tested manually" is acceptable for testing section; include lint and boundary validation results when run
- Checkboxes filled in appropriately
- No screenshots section unless UI changes
+337
View File
@@ -0,0 +1,337 @@
---
description: Validate an existing knowledge base connector against its service's API docs
argument-hint: <service-name> [api-docs-url]
---
# Validate Connector Skill
You are an expert auditor for Sim knowledge base connectors. Your job is to thoroughly validate that an existing connector is correct, complete, and follows all conventions.
## Your Task
When the user asks you to validate a connector:
1. Read the service's API documentation (via Context7 or WebFetch)
2. Read the connector implementation, OAuth config, and registry entries
3. Cross-reference everything against the API docs and Sim conventions
4. Report all issues found, grouped by severity (critical, warning, suggestion)
5. Fix all issues after reporting them
## Step 1: Gather All Files
Read **every** file for the connector — do not skip any:
```
apps/sim/connectors/{service}/meta.ts # ConnectorMeta — client-safe metadata (icon, name, auth, configFields, tagDefinitions)
apps/sim/connectors/{service}/{service}.ts # Connector implementation — spreads the meta + runtime functions
apps/sim/connectors/{service}/index.ts # Barrel export
apps/sim/connectors/registry.server.ts # Server-only full registry entry (CONNECTOR_REGISTRY; full connector)
apps/sim/connectors/registry.ts # Client-safe meta registry entry (CONNECTOR_META_REGISTRY)
apps/sim/connectors/types.ts # ConnectorMeta / ConnectorConfig interfaces, ExternalDocument, etc.
apps/sim/connectors/utils.ts # Shared utilities (computeContentHash, htmlToPlainText, etc.)
apps/sim/lib/oauth/oauth.ts # OAUTH_PROVIDERS — single source of truth for scopes
apps/sim/lib/oauth/utils.ts # getCanonicalScopesForProvider, getScopesForService, SCOPE_DESCRIPTIONS
apps/sim/lib/oauth/types.ts # OAuthService union type
apps/sim/components/icons.tsx # Icon definition for the service
```
If the connector uses selectors, also read:
```
apps/sim/hooks/selectors/registry.ts # Selector key definitions
apps/sim/hooks/selectors/types.ts # SelectorKey union type
apps/sim/lib/workflows/subblocks/context.ts # SELECTOR_CONTEXT_FIELDS
```
## Step 2: Pull API Documentation
Fetch the official API docs for the service. This is the **source of truth** for:
- Endpoint URLs, HTTP methods, and auth headers
- Required vs optional parameters
- Parameter types and allowed values
- Response shapes and field names
- Pagination patterns (cursor, offset, next token)
- Rate limits and error formats
- OAuth scopes and their meanings
Use Context7 (resolve-library-id → query-docs) or WebFetch to retrieve documentation. If both fail, note which claims are based on training knowledge vs verified docs.
## Step 3: Validate API Endpoints
For **every** API call in the connector (`listDocuments`, `getDocument`, `validateConfig`, and any helper functions), verify against the API docs:
### URLs and Methods
- [ ] Base URL is correct for the service's API version
- [ ] Endpoint paths match the API docs exactly
- [ ] HTTP method is correct (GET, POST, PUT, PATCH, DELETE)
- [ ] Path parameters are correctly interpolated and URI-encoded where needed
- [ ] Query parameters use correct names and formats per the API docs
### Headers
- [ ] Authorization header uses the correct format:
- OAuth: `Authorization: Bearer ${accessToken}`
- API Key: correct header name per the service's docs
- [ ] `Content-Type` is set for POST/PUT/PATCH requests
- [ ] Any service-specific headers are present (e.g., `Notion-Version`, `Dropbox-API-Arg`)
- [ ] No headers are sent that the API doesn't support or silently ignores
### Request Bodies
- [ ] POST/PUT body fields match API parameter names exactly
- [ ] Required fields are always sent
- [ ] Optional fields are conditionally included (not sent as `null` or empty unless the API expects that)
- [ ] Field value types match API expectations (string vs number vs boolean)
### Input Sanitization
- [ ] User-controlled values interpolated into query strings are properly escaped:
- OData `$filter`: single quotes escaped with `''` (e.g., `externalId.replace(/'/g, "''")`)
- SOQL: single quotes escaped with `\'`
- GraphQL variables: passed as variables, not interpolated into query strings
- URL path segments: `encodeURIComponent()` applied
- [ ] URL-type config fields (e.g., `siteUrl`, `instanceUrl`) are normalized:
- Strip `https://` / `http://` prefix if the API expects bare domains
- Strip trailing `/`
- Apply `.trim()` before validation
### Response Parsing
- [ ] Response structure is correctly traversed (e.g., `data.results` vs `data.items` vs `data`)
- [ ] Field names extracted match what the API actually returns
- [ ] Nullable fields are handled with `?? null` or `|| undefined`
- [ ] Error responses are checked before accessing data fields
## Step 4: Validate OAuth Scopes (if OAuth connector)
Scopes must be correctly declared and sufficient for all API calls the connector makes.
### Connector requiredScopes
- [ ] `requiredScopes` in the connector's `auth` config lists all scopes needed by the connector
- [ ] Each scope in `requiredScopes` is a real, valid scope recognized by the service's API
- [ ] No invalid, deprecated, or made-up scopes are listed
- [ ] No unnecessary excess scopes beyond what the connector actually needs
### Scope Subset Validation (CRITICAL)
- [ ] Every scope in `requiredScopes` exists in the OAuth provider's `scopes` array in `lib/oauth/oauth.ts`
- [ ] Find the provider in `OAUTH_PROVIDERS[providerGroup].services[serviceId].scopes`
- [ ] Verify: `requiredScopes``OAUTH_PROVIDERS scopes` (every required scope is present in the provider config)
- [ ] If a required scope is NOT in the provider config, flag as **critical** — the connector will fail at runtime
### Scope Sufficiency
For each API endpoint the connector calls:
- [ ] Identify which scopes are required per the API docs
- [ ] Verify those scopes are included in the connector's `requiredScopes`
- [ ] If the connector calls endpoints requiring scopes not in `requiredScopes`, flag as **warning**
### Token Refresh Config
- [ ] Check the `getOAuthTokenRefreshConfig` function in `lib/oauth/oauth.ts` for this provider
- [ ] `useBasicAuth` matches the service's token exchange requirements
- [ ] `supportsRefreshTokenRotation` matches whether the service issues rotating refresh tokens
- [ ] Token endpoint URL is correct
## Step 5: Validate Pagination
### listDocuments Pagination
- [ ] Cursor/pagination parameter name matches the API docs
- [ ] Response pagination field is correctly extracted (e.g., `next_cursor`, `nextPageToken`, `@odata.nextLink`, `offset`)
- [ ] `hasMore` is correctly determined from the response
- [ ] `nextCursor` is correctly passed back for the next page
- [ ] `maxItems` / `maxRecords` cap is correctly applied across pages using `syncContext.totalDocsFetched`
- [ ] Page size is within the API's allowed range (not exceeding max page size)
- [ ] Last page precision: when a `maxItems` cap exists, the final page request uses `Math.min(PAGE_SIZE, remaining)` to avoid fetching more records than needed
- [ ] No off-by-one errors in pagination tracking
- [ ] The connector does NOT hit known API pagination limits silently (e.g., HubSpot search 10k cap)
### Deletion-Reconciliation Safety (`listingCapped`) — CRITICAL
The sync engine hard-deletes any stored document absent from a full listing. Audit every path where `listDocuments` can return less than the full source set:
- [ ] `syncContext.listingCapped = true` is set when a `maxItems`-style cap truncates the listing while more documents exist
- [ ] `listingCapped` is set when a transient per-item error drops a still-existing document from the listing
- [ ] `listingCapped` is NOT set when the source is genuinely exhausted (deleted documents must reconcile) or for intentional scope filters (date cutoffs)
This is the most common connector bug class — verify it explicitly against `sync-engine.ts`'s reconciliation gate.
### Pagination State Across Pages
- [ ] `syncContext` is used to cache state across pages (user names, field maps, instance URLs, portal IDs, etc.)
- [ ] Cached state in `syncContext` is correctly initialized on first page and reused on subsequent pages
## Step 6: Validate Data Transformation
### ExternalDocument Construction
- [ ] `externalId` is a stable, unique identifier from the source API
- [ ] `title` is extracted from the correct field and has a sensible fallback (e.g., `'Untitled'`)
- [ ] `content` is plain text — HTML content is stripped using `htmlToPlainText` from `@/connectors/utils`
- [ ] `mimeType` is `'text/plain'`
- [ ] `contentHash` is metadata-based for contentDeferred connectors (a string template like `service:${id}:${changeIndicator}`, identical between the `listDocuments` stub and `getDocument`); content-based via `computeContentHash` from `@/connectors/utils` ONLY when `listDocuments` returns full content inline
- [ ] `sourceUrl` is a valid, complete URL back to the original resource (not relative)
- [ ] `metadata` contains all fields referenced by `mapTags` and `tagDefinitions`
### Content Extraction
- [ ] Rich text / HTML fields are converted to plain text before indexing
- [ ] Important content is not silently dropped (e.g., nested blocks, table cells, code blocks)
- [ ] Content is not silently truncated without logging a warning
- [ ] Empty/blank documents are properly filtered out
- [ ] Size checks use `Buffer.byteLength(text, 'utf8')` not `text.length` when comparing against byte-based limits (e.g., `MAX_FILE_SIZE` in bytes)
## Step 7: Validate Tag Definitions and mapTags
### tagDefinitions
- [ ] Each `tagDefinition` has an `id`, `displayName`, and `fieldType`
- [ ] `fieldType` matches the actual data type: `'text'` for strings, `'number'` for numbers, `'date'` for dates, `'boolean'` for booleans
- [ ] Every `id` in `tagDefinitions` is returned by `mapTags`
- [ ] No `tagDefinition` references a field that `mapTags` never produces
### mapTags
- [ ] Return keys match `tagDefinition` `id` values exactly
- [ ] Date values are properly parsed using `parseTagDate` from `@/connectors/utils`
- [ ] Array values are properly joined using `joinTagArray` from `@/connectors/utils`
- [ ] Number values are validated (not `NaN`)
- [ ] Metadata field names accessed in `mapTags` match what `listDocuments`/`getDocument` store in `metadata`
## Step 8: Validate Config Fields and Validation
### configFields
- [ ] Every field has `id`, `title`, `type`
- [ ] `required` is set explicitly (not omitted)
- [ ] Dropdown fields have `options` with `label` and `id` for each option
- [ ] Selector fields follow the canonical pair pattern:
- A `type: 'selector'` field with `selectorKey`, `canonicalParamId`, `mode: 'basic'`
- A `type: 'short-input'` field with the same `canonicalParamId`, `mode: 'advanced'`
- `required` is identical on both fields in the pair
- [ ] `selectorKey` values exist in the selector registry
- [ ] `dependsOn` references selector field `id` values, not `canonicalParamId`
### validateConfig
- [ ] Validates all required fields are present before making API calls
- [ ] Validates optional numeric fields (checks `Number.isNaN`, positive values)
- [ ] Makes a lightweight API call to verify access (e.g., fetch 1 record, get profile)
- [ ] Uses `VALIDATE_RETRY_OPTIONS` for retry budget
- [ ] Returns `{ valid: true }` on success
- [ ] Returns `{ valid: false, error: 'descriptive message' }` on failure
- [ ] Catches exceptions and returns user-friendly error messages
- [ ] Does NOT make expensive calls (full data listing, large queries)
## Step 9: Validate getDocument
- [ ] Fetches a single document by `externalId`
- [ ] Returns `null` for 404 / not found (does not throw)
- [ ] Returns the same `ExternalDocument` shape as `listDocuments`
- [ ] Handles all content types that `listDocuments` can produce (e.g., if `listDocuments` returns both pages and blogposts, `getDocument` must handle both — not hardcode one endpoint)
- [ ] Forwards `syncContext` if it needs cached state (user names, field maps, etc.)
- [ ] Error handling is graceful (catches, logs, returns null or throws with context)
- [ ] Does not redundantly re-fetch data already included in the initial API response (e.g., if comments come back with the post, don't fetch them again separately)
## Step 10: Validate General Quality
### fetchWithRetry Usage
- [ ] All external API calls use `fetchWithRetry` from `@/lib/knowledge/documents/utils`
- [ ] No raw `fetch()` calls to external APIs
- [ ] `VALIDATE_RETRY_OPTIONS` used in `validateConfig`
- [ ] If `validateConfig` calls a shared helper (e.g., `linearGraphQL`, `resolveId`), that helper must accept and forward `retryOptions` to `fetchWithRetry`
- [ ] Default retry options used in `listDocuments`/`getDocument`
### API Efficiency
- [ ] APIs that support field selection (e.g., `$select`, `sysparm_fields`, `fields`) should request only the fields the connector needs — in both `listDocuments` AND `getDocument`
- [ ] No redundant API calls: if a helper already fetches data (e.g., site metadata), callers should reuse the result instead of making a second call for the same information
- [ ] Sequential per-item API calls (fetching details for each document in a loop) should be batched with `Promise.all` and a concurrency limit of 3-5
### Error Handling
- [ ] Individual document failures are caught and logged without aborting the sync
- [ ] API error responses include status codes in error messages
- [ ] No unhandled promise rejections in concurrent operations
### Concurrency
- [ ] Concurrent API calls use reasonable batch sizes (3-5 is typical)
- [ ] No unbounded `Promise.all` over large arrays
### Logging
- [ ] Uses `createLogger` from `@sim/logger` (not `console.log`)
- [ ] Logs sync progress at `info` level
- [ ] Logs errors at `warn` or `error` level with context
### Meta / Runtime Split
- [ ] `connectors/{service}/meta.ts` exports `{service}ConnectorMeta: ConnectorMeta` (id, name, description, version, icon, auth, configFields, and any `tagDefinitions` / `supportsIncrementalSync`)
- [ ] `meta.ts` imports ONLY the icon from `@/components/icons`, `ConnectorMeta` (type-only), and pure-data constants — NO server/runtime imports (`@/lib/knowledge/...`, `input-validation.server`, `fetchWithRetry`, etc.); any such import in `meta.ts` is **critical** (breaks the client bundle)
- [ ] `connectors/{service}/{service}.ts` spreads `...{service}ConnectorMeta` as the first property and adds the runtime functions (`listDocuments`, `getDocument`, `validateConfig`, `mapTags?`)
- [ ] Metadata fields (id, name, auth, configFields, etc.) live ONLY in `meta.ts`, not duplicated in `{service}.ts`
### Registry
- [ ] Connector is exported from `connectors/{service}/index.ts`
- [ ] Full connector is registered in `connectors/registry.server.ts` (server-only registry, `CONNECTOR_REGISTRY`)
- [ ] Meta is registered in `connectors/registry.ts` (client-safe registry, `CONNECTOR_META_REGISTRY`), importing `@/connectors/{service}/meta`
- [ ] Both registries use the same key and it matches the connector's `id` field
- [ ] Both registries keep the same alphabetical-by-id ordering
## Step 11: Report and Fix
### Report Format
Group findings by severity:
**Critical** (will cause runtime errors, data loss, or auth failures):
- Wrong API endpoint URL or HTTP method
- Invalid or missing OAuth scopes (not in provider config)
- Incorrect response field mapping (accessing wrong path)
- SOQL/query fields that don't exist on the target object
- Pagination that silently hits undocumented API limits
- Missing error handling that would crash the sync
- `requiredScopes` not a subset of OAuth provider scopes
- Query/filter injection: user-controlled values interpolated into OData `$filter`, SOQL, or query strings without escaping
- Server/runtime import in `meta.ts` (e.g. `@/lib/knowledge/...`, `input-validation.server`, `fetchWithRetry`) — pulls server-only code into the client bundle and breaks the build
- Connector missing from `connectors/registry.ts` (the client-safe meta registry) — or its entry there imports the runtime module instead of `meta.ts` — the knowledge UI can't render it
**Warning** (incorrect behavior, data quality issues, or convention violations):
- HTML content not stripped via `htmlToPlainText`
- `getDocument` not forwarding `syncContext`
- `getDocument` hardcoded to one content type when `listDocuments` returns multiple (e.g., only pages but not blogposts)
- Missing `tagDefinition` for metadata fields returned by `mapTags`
- Incorrect `useBasicAuth` or `supportsRefreshTokenRotation` in token refresh config
- Invalid scope names that the API doesn't recognize (even if silently ignored)
- Private resources excluded from name-based lookup despite scopes being available
- Silent data truncation without logging
- `contentHash` uses the wrong basis: a content-based `computeContentHash` on a contentDeferred connector (breaks the stub/getDocument-identical invariant), or a metadata template when `listDocuments` returns full content inline
- Size checks using `text.length` (character count) instead of `Buffer.byteLength` (byte count) for byte-based limits
- URL-type config fields not normalized (protocol prefix, trailing slashes cause API failures)
- `VALIDATE_RETRY_OPTIONS` not threaded through helper functions called by `validateConfig`
**Suggestion** (minor improvements):
- Missing incremental sync support despite API supporting it
- Overly broad scopes that could be narrowed (not wrong, but could be tighter)
- Source URL format could be more specific
- Missing `orderBy` for deterministic pagination
- Redundant API calls that could be cached in `syncContext`
- Sequential per-item API calls that could be batched with `Promise.all` (concurrency 3-5)
- API supports field selection but connector fetches all fields (e.g., missing `$select`, `sysparm_fields`, `fields`)
- `getDocument` re-fetches data already included in the initial API response (e.g., comments returned with post)
- Last page of pagination requests full `PAGE_SIZE` when fewer records remain (`Math.min(PAGE_SIZE, remaining)`)
### Fix All Issues
After reporting, fix every **critical** and **warning** issue. Apply **suggestions** where they don't add unnecessary complexity.
### Validation Output
After fixing, confirm:
1. `bun run lint` passes
2. TypeScript compiles clean
3. Re-read all modified files to verify fixes are correct
## Checklist Summary
- [ ] Read connector meta.ts, implementation, types, utils, both registries, and OAuth config
- [ ] Pulled and read official API documentation for the service
- [ ] Validated every API endpoint URL, method, headers, and body against API docs
- [ ] Validated input sanitization: no query/filter injection, URL fields normalized
- [ ] Validated OAuth scopes: `requiredScopes` ⊆ OAuth provider `scopes` in `oauth.ts`
- [ ] Validated each scope is real and recognized by the service's API
- [ ] Validated scopes are sufficient for all API endpoints the connector calls
- [ ] Validated token refresh config (`useBasicAuth`, `supportsRefreshTokenRotation`)
- [ ] Validated pagination: cursor names, page sizes, hasMore logic, no silent caps
- [ ] Validated data transformation: plain text extraction, HTML stripping, content hashing
- [ ] Validated tag definitions match mapTags output, correct fieldTypes
- [ ] Validated config fields: canonical pairs, selector keys, required flags
- [ ] Validated validateConfig: lightweight check, error messages, retry options
- [ ] Validated getDocument: null on 404, all content types handled, no redundant re-fetches, syncContext forwarding
- [ ] Validated fetchWithRetry used for all external calls (no raw fetch), VALIDATE_RETRY_OPTIONS threaded through helpers
- [ ] Validated API efficiency: field selection used, no redundant calls, sequential fetches batched
- [ ] Validated error handling: graceful failures, no unhandled rejections
- [ ] Validated logging: createLogger, no console.log
- [ ] Validated meta/runtime split: `meta.ts` holds metadata with no server/runtime imports, `{service}.ts` spreads the meta + adds runtime functions
- [ ] Validated registry: exported from index.ts, full connector in `registry.server.ts`, meta in `registry.ts`, matching keys and alphabetical-by-id ordering in both
- [ ] Reported all issues grouped by severity
- [ ] Fixed all critical and warning issues
- [ ] Ran `bun run lint` after fixes
- [ ] Verified TypeScript compiles clean
+297
View File
@@ -0,0 +1,297 @@
---
description: Validate an existing Sim integration (tools, block, registry) against the service's API docs
argument-hint: <service-name> [api-docs-url]
---
# Validate Integration Skill
You are an expert auditor for Sim integrations. Your job is to thoroughly validate that an existing integration is correct, complete, and follows all conventions.
## Your Task
When the user asks you to validate an integration:
1. Read the service's API documentation (via WebFetch or Context7)
2. Read every tool, the block, and registry entries
3. Cross-reference everything against the API docs and Sim conventions
4. Report all issues found, grouped by severity (critical, warning, suggestion)
5. Fix all issues after reporting them
## Step 1: Gather All Files
Read **every** file for the integration — do not skip any:
```
apps/sim/tools/{service}/ # All tool files, types.ts, index.ts
apps/sim/blocks/blocks/{service}.ts # Block definition
apps/sim/tools/registry.ts # Tool registry entries for this service
apps/sim/blocks/registry-maps.ts # Block + meta registry entry (BLOCK_REGISTRY / BLOCK_META_REGISTRY)
apps/sim/components/icons.tsx # Icon definition
apps/sim/lib/auth/auth.ts # OAuth config — should use getCanonicalScopesForProvider()
apps/sim/lib/oauth/oauth.ts # OAuth provider config — single source of truth for scopes
apps/sim/lib/oauth/utils.ts # Scope utilities, SCOPE_DESCRIPTIONS for modal UI
```
## Step 2: Pull API Documentation
Fetch the official API docs for the service. This is the **source of truth** for:
- Endpoint URLs, HTTP methods, and auth headers
- Required vs optional parameters
- Parameter types and allowed values
- Response shapes and field names
- Pagination patterns (which param name, which response field)
- Rate limits and error formats
## Step 3: Validate Tools
For **every** tool file, check:
### Tool ID and Naming
- [ ] Tool ID uses `snake_case`: `{service}_{action}` (e.g., `x_create_tweet`, `slack_send_message`)
- [ ] Tool `name` is human-readable (e.g., `'X Create Tweet'`)
- [ ] Tool `description` is a concise one-liner describing what it does
- [ ] Tool `version` is set (`'1.0.0'` or `'2.0.0'` for V2)
### Params
- [ ] All required API params are marked `required: true`
- [ ] All optional API params are marked `required: false`
- [ ] Every param has explicit `required: true` or `required: false` — never omitted
- [ ] Param types match the API (`'string'`, `'number'`, `'boolean'`, `'json'`)
- [ ] Visibility is correct:
- `'hidden'` — ONLY for OAuth access tokens and system-injected params
- `'user-only'` — for API keys, credentials, and account-specific IDs the user must provide
- `'user-or-llm'` — for everything else (search queries, content, filters, IDs that could come from other blocks)
- [ ] Every param has a `description` that explains what it does
### Request
- [ ] URL matches the API endpoint exactly (correct base URL, path segments, path params)
- [ ] HTTP method matches the API spec (GET, POST, PUT, PATCH, DELETE)
- [ ] Headers include correct auth pattern:
- OAuth: `Authorization: Bearer ${params.accessToken}`
- API Key: correct header name and format per the service's docs
- [ ] `Content-Type` header is set for POST/PUT/PATCH requests
- [ ] Body sends all required fields and only includes optional fields when provided
- [ ] For GET requests with query params: URL is constructed correctly with query string
- [ ] ID fields in URL paths are `.trim()`-ed to prevent copy-paste whitespace errors
- [ ] Path params use template literals correctly: `` `https://api.service.com/v1/${params.id.trim()}` ``
### Response / transformResponse
- [ ] Correctly parses the API response (`await response.json()`)
- [ ] Extracts the right fields from the response structure (e.g., `data.data` vs `data` vs `data.results`)
- [ ] All nullable fields use `?? null`
- [ ] All optional arrays use `?? []`
- [ ] Error cases are handled: checks for missing/empty data and returns meaningful error
- [ ] Does NOT do raw JSON dumps — extracts meaningful, individual fields
### Outputs
- [ ] All output fields match what the API actually returns
- [ ] No fields are missing that the API provides and users would commonly need
- [ ] No phantom fields defined that the API doesn't return
- [ ] `optional: true` is set on fields that may not exist in all responses
- [ ] When using `type: 'json'` and the shape is known, `properties` defines the inner fields (tool outputs only — block outputs do not support `properties`)
- [ ] When using `type: 'array'`, `items` defines the item structure with `properties` (tool outputs only)
- [ ] Field descriptions are accurate and helpful
### Types (types.ts)
- [ ] Has param interfaces for every tool (e.g., `XCreateTweetParams`)
- [ ] Has response interfaces for every tool (extending `ToolResponse`)
- [ ] Optional params use `?` in the interface (e.g., `replyTo?: string`)
- [ ] Field names in types match actual API field names
- [ ] Shared response types are properly reused (e.g., `XTweetResponse` shared across tweet tools)
### Barrel Export (index.ts)
- [ ] Every tool is exported
- [ ] All types are re-exported (`export * from './types'`)
- [ ] No orphaned exports (tools that don't exist)
### Tool Registry (tools/registry.ts)
- [ ] Every tool is imported and registered
- [ ] Registry keys use snake_case and match tool IDs exactly
- [ ] Entries are in alphabetical order within the file
## Step 4: Validate Block
### Block ↔ Tool Alignment (CRITICAL)
This is the most important validation — the block must be perfectly aligned with every tool it references.
For **each tool** in `tools.access`:
- [ ] The operation dropdown has an option whose ID matches the tool ID (or the `tools.config.tool` function correctly maps to it)
- [ ] Every **required** tool param (except `accessToken`) has a corresponding subBlock input that is:
- Shown when that operation is selected (correct `condition`)
- Marked as `required: true` (or conditionally required)
- [ ] Every **optional** tool param has a corresponding subBlock input (or is intentionally omitted if truly never needed)
- [ ] SubBlock `id` values are unique across the entire block — no duplicates even across different conditions
- [ ] The `tools.config.tool` function returns the correct tool ID for every possible operation value
- [ ] The `tools.config.params` function correctly maps subBlock IDs to tool param names when they differ
### SubBlocks
- [ ] Operation dropdown lists ALL tool operations available in `tools.access`
- [ ] Dropdown option labels are human-readable and descriptive
- [ ] Conditions use correct syntax:
- Single value: `{ field: 'operation', value: 'x_create_tweet' }`
- Multiple values (OR): `{ field: 'operation', value: ['x_create_tweet', 'x_delete_tweet'] }`
- Negation: `{ field: 'operation', value: 'delete', not: true }`
- Compound: `{ field: 'op', value: 'send', and: { field: 'type', value: 'dm' } }`
- [ ] Condition arrays include ALL operations that use that field — none missing
- [ ] `dependsOn` is set for fields that need other values (selectors depending on credential, cascading dropdowns)
- [ ] SubBlock types match tool param types:
- Enum/fixed options → `dropdown`
- Free text → `short-input`
- Long text/content → `long-input`
- True/false → `dropdown` with Yes/No options (not `switch` unless purely UI toggle)
- Credentials → `oauth-input` with correct `serviceId`
- [ ] Dropdown `value: () => 'default'` is set for dropdowns with a sensible default
### Advanced Mode
- [ ] Optional, rarely-used fields are set to `mode: 'advanced'`:
- Pagination tokens / next tokens
- Time range filters (start/end time)
- Sort order / direction options
- Max results / per page limits
- Reply settings / threading options
- Rarely used IDs (reply-to, quote-tweet, etc.)
- Exclude filters
- [ ] **Required** fields are NEVER set to `mode: 'advanced'`
- [ ] Fields that users fill in most of the time are NOT set to `mode: 'advanced'`
### WandConfig
- [ ] Timestamp fields have `wandConfig` with `generationType: 'timestamp'`
- [ ] Comma-separated list fields have `wandConfig` with a descriptive prompt
- [ ] Complex filter/query fields have `wandConfig` with format examples in the prompt
- [ ] All `wandConfig` prompts end with "Return ONLY the [format] - no explanations, no extra text."
- [ ] `wandConfig.placeholder` describes what to type in natural language
### Tools Config
- [ ] `tools.access` lists **every** tool ID the block can use — none missing
- [ ] `tools.config.tool` returns the correct tool ID for each operation
- [ ] Type coercions are in `tools.config.params` (runs at execution time), NOT in `tools.config.tool` (runs at serialization time before variable resolution)
- [ ] `tools.config.params` handles:
- `Number()` conversion for numeric params that come as strings from inputs
- `Boolean` / string-to-boolean conversion for toggle params
- Empty string → `undefined` conversion for optional dropdown values
- Any subBlock ID → tool param name remapping
- [ ] No `Number()`, `JSON.parse()`, or other coercions in `tools.config.tool` — these would destroy dynamic references like `<Block.output>`
### Block Outputs
- [ ] Outputs cover the key fields returned by ALL tools (not just one operation)
- [ ] Output types are correct (`'string'`, `'number'`, `'boolean'`, `'json'`)
- [ ] `type: 'json'` outputs describe inner fields in the description string: `'User profile (id, name, username, bio)'` or `'[{address, status, type}]'` for arrays
- [ ] **Do NOT add a `properties: {...}` field on block outputs.** Block-level `OutputFieldDefinition` (from `@sim/workflow-types/blocks`) only accepts `{ type, description?, condition?, hiddenFromDisplay? }`. Nested `properties` is a tool-level construct (`OutputProperty`) — adding it to a block output will fail TypeScript at build time
- [ ] No opaque `type: 'json'` with vague descriptions like `'Response data'`
- [ ] Outputs that only appear for certain operations use `condition` if supported, or document which operations return them
### Block Metadata
- [ ] `type` is snake_case (e.g., `'x'`, `'cloudflare'`)
- [ ] `name` is human-readable (e.g., `'X'`, `'Cloudflare'`)
- [ ] `description` is a concise one-liner
- [ ] `longDescription` provides detail for docs
- [ ] `docsLink` points to `'https://docs.sim.ai/integrations/{service}'`
- [ ] `category` is `'tools'`
- [ ] `bgColor` uses the service's brand color hex
- [ ] `icon` references the correct icon component from `@/components/icons`
- [ ] `authMode` is set correctly (`AuthMode.OAuth` or `AuthMode.ApiKey`)
- [ ] Block + meta are registered in `blocks/registry-maps.ts` (`BLOCK_REGISTRY` / `BLOCK_META_REGISTRY`) alphabetically
### BlockMeta
- [ ] `{Service}BlockMeta` is exported in the same file as the block
- [ ] Has at least 7 templates, each with `icon`, `title`, `prompt`, `modules`, `category`, and `tags`
- [ ] Prompts describe concrete use cases, not generic descriptions of what the service does
- [ ] `alsoIntegrations` is set on any template whose prompt references another service
- [ ] `skills` present (35 mainstream, 23 niche), each grounded in `tools.access` — flag any skill implying an unsupported action
- [ ] **Each skill is real, not hallucinated** — web-search and confirm it maps to a popular use case attested online (vendor use-case pages, official docs describing the workflow, reputable "top automations" articles); rewrite/remove any you cannot source
### Block Inputs
- [ ] `inputs` section lists all subBlock params that the block accepts
- [ ] Input types match the subBlock types
- [ ] When using `canonicalParamId`, inputs list the canonical ID (not the raw subBlock IDs)
## Step 5: Validate OAuth Scopes (if OAuth service)
Scopes are centralized — the single source of truth is `OAUTH_PROVIDERS` in `lib/oauth/oauth.ts`.
- [ ] Scopes defined in `lib/oauth/oauth.ts` under `OAUTH_PROVIDERS[provider].services[service].scopes`
- [ ] `auth.ts` uses `getCanonicalScopesForProvider(providerId)` — NOT a hardcoded array
- [ ] Block `requiredScopes` uses `getScopesForService(serviceId)` — NOT a hardcoded array
- [ ] No hardcoded scope arrays in `auth.ts` or block files (should all use utility functions)
- [ ] Each scope has a human-readable description in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`
- [ ] No excess scopes that aren't needed by any tool
## Step 6: Validate Pagination Consistency
If any tools support pagination:
- [ ] Pagination param names match the API docs (e.g., `pagination_token` vs `next_token` vs `cursor`)
- [ ] Different API endpoints that use different pagination param names have separate subBlocks in the block
- [ ] Pagination response fields (`nextToken`, `cursor`, etc.) are included in tool outputs
- [ ] Pagination subBlocks are set to `mode: 'advanced'`
## Step 7: Validate Error Handling
- [ ] `transformResponse` checks for error conditions before accessing data
- [ ] Error responses include meaningful messages (not just generic "failed")
- [ ] HTTP error status codes are handled (check `response.ok` or status codes)
## Step 8: Report and Fix
### Report Format
Group findings by severity:
**Critical** (will cause runtime errors or incorrect behavior):
- Wrong endpoint URL or HTTP method
- Missing required params or wrong `required` flag
- Incorrect response field mapping (accessing wrong path in response)
- Missing error handling that would cause crashes
- Tool ID mismatch between tool file, registry, and block `tools.access`
- OAuth scopes missing in `auth.ts` that tools need
- `tools.config.tool` returning wrong tool ID for an operation
- Type coercions in `tools.config.tool` instead of `tools.config.params`
**Warning** (follows conventions incorrectly or has usability issues):
- Optional field not set to `mode: 'advanced'`
- Missing `wandConfig` on timestamp/complex fields
- Wrong `visibility` on params (e.g., `'hidden'` instead of `'user-or-llm'`)
- Missing `optional: true` on nullable outputs
- Opaque `type: 'json'` without property descriptions
- Missing `.trim()` on ID fields in request URLs
- Missing `?? null` on nullable response fields
- Block condition array missing an operation that uses that field
- Hardcoded scope arrays instead of using `getScopesForService()` / `getCanonicalScopesForProvider()`
- Missing scope description in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`
**Suggestion** (minor improvements):
- Better description text
- Inconsistent naming across tools
- Missing `longDescription` or `docsLink`
- Pagination fields that could benefit from `wandConfig`
### Fix All Issues
After reporting, fix every **critical** and **warning** issue. Apply **suggestions** where they don't add unnecessary complexity.
### Validation Output
After fixing, confirm:
1. `bun run lint` passes with no fixes needed
2. TypeScript compiles clean (no type errors)
3. Re-read all modified files to verify fixes are correct
## Checklist Summary
- [ ] Read ALL tool files, block, types, index, and registries
- [ ] Pulled and read official API documentation
- [ ] Validated every tool's ID, params, request, response, outputs, and types against API docs
- [ ] Validated block ↔ tool alignment (every tool param has a subBlock, every condition is correct)
- [ ] Validated advanced mode on optional/rarely-used fields
- [ ] Validated wandConfig on timestamps and complex inputs
- [ ] Validated tools.config mapping, tool selector, and type coercions
- [ ] Validated block outputs match what tools return, with typed JSON where possible
- [ ] Validated OAuth scopes use centralized utilities (getScopesForService, getCanonicalScopesForProvider) — no hardcoded arrays
- [ ] Validated scope descriptions exist in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` for all scopes
- [ ] Validated pagination consistency across tools and block
- [ ] Validated error handling (error checks, meaningful messages)
- [ ] Validated registry entries (tools and block, alphabetical, correct imports)
- [ ] Validated `{Service}BlockMeta` exported with at least 7 templates
- [ ] Reported all issues grouped by severity
- [ ] Fixed all critical and warning issues
- [ ] Ran `bun run lint` after fixes
- [ ] Verified TypeScript compiles clean
+170
View File
@@ -0,0 +1,170 @@
---
description: Validate a model entry (or every model in a provider) in apps/sim/providers/models.ts against the provider's live API docs (no hallucination — reports what cannot be verified)
argument-hint: <provider> [model-id]
---
# Validate Model Skill
You audit one or more model entries in `apps/sim/providers/models.ts` against the provider's official live API docs. **Hallucinated pricing and capabilities are the #1 failure mode in this file.** Every numeric and capability claim must be re-derived from a live web fetch in this session — not from memory, not from training data, not from the user's marketing email.
## Hard rules (do not skip)
1. **Live-fetch or report unverified.** Each field must be backed by a live WebFetch in this session. If you cannot reach an authoritative URL for a field, mark it **UNVERIFIED** in the report — do not silently confirm it from memory.
2. **Cite every fact.** Every value in the report must show the source URL it was checked against. No URL → mark UNVERIFIED.
3. **Two-source rule for pricing.** Cross-check input/output/cached against at least one secondary source (OpenRouter, Artificial Analysis, CloudPrice). If sources disagree, the provider's own docs win — flag the disagreement.
4. **Inspect provider implementation before flagging capability mismatches.** A capability flag in `models.ts` is dead unless the provider's code under `apps/sim/providers/{provider}/` consumes it (see Consumption Matrix below). Setting a flag the provider ignores is a warning, not a critical.
5. **Never auto-fix without printing the diff.** Show the user the proposed diff before applying. Get confirmation.
## Your Task
When invoked as `/validate-model <provider> [model-id]`:
1. Read the target entries from `models.ts`
2. Live-fetch the provider's official models, pricing, and capability/reasoning pages + at least one secondary source for pricing
3. Inspect the provider implementation to know which flags are actually consumed
4. Run the checklist below per model
5. Report findings (critical / warning / suggestion / unverified) with every cell linked to its source URL
6. Offer to fix; on confirm, edit `models.ts` in a single pass and re-lint
If `model-id` is omitted, validate every model in the provider.
## Step 1: Read entries from `models.ts`
Capture per model: `id`, full `pricing`, full `capabilities`, `contextWindow`, `releaseDate`, `recommended`, `speedOptimized`, `deprecated`.
## Step 2: Live-fetch authoritative sources
Use the canonical provider URL table in the `add-model` skill (`.claude/commands/add-model.md`, or its mirror `.agents/skills/add-model/SKILL.md`), Step 1, as the single source of truth — fetch the models index, pricing, and reasoning/parameter caveats pages listed there for the target provider. If you update one table, update the other in the same change.
Secondary cross-check (use at least one): OpenRouter, Artificial Analysis, CloudPrice.
If a fetch fails (404, timeout, paywall), record the URL attempted and mark dependent fields UNVERIFIED.
## Step 3: Build the consumption map for this provider
Re-grep before trusting the snapshot below:
```bash
rg "reasoningEffort|reasoning_effort" apps/sim/providers/<provider>/
rg "verbosity" apps/sim/providers/<provider>/
rg "request\.thinking|thinking:" apps/sim/providers/<provider>/
rg "supportsNativeStructuredOutputs|nativeStructuredOutputs" apps/sim/providers/<provider>/
```
Snapshot (verify before relying):
| Capability | Consumed by |
|---|---|
| `reasoningEffort` | `openai/core.ts`, `azure-openai`, `anthropic/core.ts` (mapped via thinking), `gemini/core.ts` |
| `verbosity` | `openai/core.ts`, `azure-openai/index.ts` |
| `thinking` | `anthropic/core.ts`, `gemini/core.ts` |
| `nativeStructuredOutputs` | `anthropic/core.ts`, `fireworks/index.ts`, `openrouter/index.ts` |
| `computerUse` | `anthropic/core.ts` |
| `temperature` | All providers (passthrough) |
A flag set in `models.ts` but not in the consumption list for this provider = **warning: dead flag**.
## Step 4: Run the checklist
For each model, evaluate every row. Statuses: ✓ matches docs, ✗ disagrees, ⚠️ single-source, ❓ UNVERIFIED (could not fetch).
### Identity
- [ ] `id` exactly matches provider's API model identifier (case, dots, dashes, prefix for resellers)
- [ ] `releaseDate` matches launch announcement
- [ ] `deprecated: true` set if provider has announced retirement (or removed from active list)
### Pricing (per 1M tokens, USD)
- [ ] `pricing.input` matches provider pricing page
- [ ] `pricing.output` matches provider pricing page
- [ ] `pricing.cachedInput` matches provider's documented cached/prompt-cache rate (or is correctly omitted if no caching offered)
- [ ] `pricing.updatedAt` is recent — warn if older than 60 days
### Context & output limits
- [ ] `contextWindow` matches docs (in tokens)
- [ ] `capabilities.maxOutputTokens` matches documented output cap (or is correctly omitted if "no output limit")
### Capabilities (each must be DOCUMENTED-AS-SUPPORTED **and** CONSUMED-BY-PROVIDER-CODE)
- [ ] `temperature` — provider accepts it for this model (reasoning-always-on models often reject)
- [ ] `reasoningEffort.values` — list matches docs; **omitted** for always-reasoning models that reject the parameter (e.g., grok-4.3, where xAI docs explicitly state `reasoning_effort` is not supported). Verify per model — some always-reasoning models (e.g., OpenAI's o-series) DO accept `reasoning_effort` and should keep the flag.
- [ ] `verbosity.values` — only on OpenAI gpt-5.x family; values match docs
- [ ] `thinking.levels` + `thinking.default` — only on Anthropic/Gemini; values match docs
- [ ] `nativeStructuredOutputs` — only on anthropic/fireworks/openrouter; provider must document Structured Outputs / JSON-mode for this model
- [ ] `toolUsageControl` — provider supports `tool_choice` semantics
- [ ] `computerUse` — provider implements computer-use loop AND model is a computer-use SKU
- [ ] `deepResearch` — only on actual deep-research SKUs
- [ ] `memory: false` — only when the model genuinely cannot maintain conversation history
### Flags
- [ ] `recommended: true` — at most one or two per provider; should be current flagship
- [ ] `speedOptimized: true` — only on smallest/fastest tier (nano / flash-lite / haiku class)
### Hosting / billing
- [ ] If the model is under `openai`/`anthropic`/`google`, it is automatically in `getHostedModels()` → served with Sim's rotating key and billed via `shouldBillModelUsage()`. Confirm that is the intent (a BYOK-only model parked under one of these providers is a billing bug — warning).
- [ ] If the model is hosted, the deployment is expected to have its `{PREFIX}_COUNT` / `{PREFIX}_1..N` env vars set (ops concern; note if it looks unset for a model claiming hosted support).
## Step 5: Report (mandatory format)
For each model, emit a table with one row per checklist item. Every row that claims ✓ must have a URL.
```markdown
### Validation — <model-id>
| Field | Repo | Live docs | Source URL | Status |
|---|---|---|---|---|
| `input` | $1.25/M | $1.25/M | https://docs.x.ai/... | ✓ |
| `cachedInput` | $0.50/M | $0.20/M | https://cloudprice.net/... | ✗ stale (price cut not picked up) |
| `reasoningEffort` | low/medium/high | rejected by API | https://docs.x.ai/.../reasoning | ✗ inert — selecting silently no-ops |
| `contextWindow` | 1,000,000 | 1,000,000 | https://docs.x.ai/... + https://openrouter.ai/... | ✓ (2 sources) |
| `releaseDate` | 2026-04-30 | not found in scraped pages | _attempted: docs.x.ai, x.ai/news_ | ❓ UNVERIFIED |
**Findings**
- 🔴 critical — `cachedInput` is wrong: docs say $0.20/M, repo has $0.50/M
- 🟡 warning — `reasoningEffort` is set but provider rejects it for this model (xAI docs explicitly: "reasoning_effort is not supported by grok-4.3")
- 🔵 suggestion — `pricing.updatedAt` is 90 days old; refresh
- ❓ unverified — `releaseDate` could not be confirmed from any fetched page; ask user
**Disagreements between sources**
- _none_ OR _OpenRouter says $X, provider docs say $Y — went with provider docs_
```
End each multi-model run with a summary count: `N models checked · X critical · Y warnings · Z suggestions · W unverified`.
## Step 6: Offer to fix
After reporting, ask: *"Want me to fix the critical and warning items? I'll print the diff first."* On yes:
1. Print the proposed diff (do not apply yet)
2. Get user confirmation
3. Edit `models.ts` in a single pass
4. Run `bun run lint`
5. Re-run only the failed rows of the checklist on the new state
## Severity definitions
- 🔴 **critical** — wrong number or wrong identifier that misleads users about cost or breaks API calls. Examples: incorrect pricing, wrong model id, wrong context window, capability the API rejects.
- 🟡 **warning** — dead code or internal inconsistency. Examples: capability flag the provider ignores, multiple `recommended: true` per provider, `pricing.updatedAt` >60 days old, missing `deprecated: true` on retired model.
- 🔵 **suggestion** — style/consistency. Examples: field order, missing `speedOptimized` on a clearly smallest-tier model.
-**unverified** — could not fetch an authoritative source for this field. Surface it; never silently confirm.
## Common bugs this skill catches
- Pricing drift after a provider price cut (very common — providers cut quarterly)
- `reasoningEffort` set on always-reasoning models that reject the parameter (grok-4.3, o3-pro pattern)
- `nativeStructuredOutputs` set on providers that don't consume the flag (dead)
- `thinking` set on non-Anthropic/non-Gemini providers
- `verbosity` set on non-gpt-5.x models
- Wrong context window (e.g., 128k claimed vs 200k actual)
- Stale `pricing.updatedAt`
- Multiple `recommended: true` per provider after a flagship swap
- Missing `deprecated: true` on retired models (e.g., the xAI batch retiring May 15, 2026)
## What "I cannot verify this" looks like
If, after fetching the documented sources, a field cannot be confirmed:
- Mark the row ❓ UNVERIFIED with the URL(s) attempted
- Surface it in the **Findings** section with severity ❓
- Do NOT mark the validation as passed
- Ask the user for a docs URL or guidance before changing anything
The skill is allowed to say *"I could not verify the cached input price for grok-4.3 from the official xAI docs in this session — I attempted [URLs] without finding the value. Third-party sources [URL1, URL2] both report $0.20/M. Confirm before I update."* That is correct behavior. Hallucinating a number is not.
+212
View File
@@ -0,0 +1,212 @@
---
description: Validate an existing Sim webhook trigger against provider API docs and repository conventions
argument-hint: <service-name> [api-docs-url]
---
# Validate Trigger
You are an expert auditor for Sim webhook triggers. Your job is to validate that an existing trigger implementation is correct, complete, secure, and aligned across all layers.
## Your Task
1. Read the service's webhook/API documentation (via WebFetch)
2. Read every trigger file, provider handler, and registry entry
3. Cross-reference against the API docs and Sim conventions
4. Report all issues grouped by severity (critical, warning, suggestion)
5. Fix all issues after reporting them
## Step 1: Gather All Files
Read **every** file for the trigger — do not skip any:
```
apps/sim/triggers/{service}/ # All trigger files, utils.ts, index.ts
apps/sim/lib/webhooks/providers/{service}.ts # Provider handler (if exists)
apps/sim/lib/webhooks/providers/registry.ts # Handler registry
apps/sim/triggers/registry.ts # Trigger registry
apps/sim/blocks/blocks/{service}.ts # Block definition (trigger wiring)
```
Also read for reference:
```
apps/sim/lib/webhooks/providers/types.ts # WebhookProviderHandler interface
apps/sim/lib/webhooks/providers/utils.ts # Shared helpers (createHmacVerifier, etc.)
apps/sim/lib/webhooks/provider-subscription-utils.ts # Subscription helpers
apps/sim/lib/webhooks/processor.ts # Central webhook processor
```
## Step 2: Pull API Documentation
Fetch the service's official webhook documentation. This is the **source of truth** for:
- Webhook event types and payload shapes
- Signature/auth verification method (HMAC algorithm, header names, secret format)
- Challenge/verification handshake requirements
- Webhook subscription API (create/delete endpoints, if applicable)
- Retry behavior and delivery guarantees
## Step 3: Validate Trigger Definitions
### utils.ts
- [ ] `{service}TriggerOptions` lists all trigger IDs accurately
- [ ] `{service}SetupInstructions` provides clear, correct steps for the service
- [ ] `build{Service}ExtraFields` includes relevant filter/config fields with correct `condition`
- [ ] Output builders expose all meaningful fields from the webhook payload
- [ ] Output builders do NOT use `optional: true` or `items` (tool-output-only features)
- [ ] Nested output objects correctly model the payload structure
### Trigger Files
- [ ] Exactly one primary trigger has `includeDropdown: true`
- [ ] All secondary triggers do NOT have `includeDropdown`
- [ ] All triggers use `buildTriggerSubBlocks` helper (not hand-rolled subBlocks)
- [ ] Every trigger's `id` matches the convention `{service}_{event_name}`
- [ ] Every trigger's `provider` matches the service name used in the handler registry
- [ ] `index.ts` barrel exports all triggers
### Trigger ↔ Provider Alignment (CRITICAL)
- [ ] Every trigger ID referenced in `matchEvent` logic exists in `{service}TriggerOptions`
- [ ] Event matching logic in the provider correctly maps trigger IDs to service event types
- [ ] Event matching logic in `is{Service}EventMatch` (if exists) correctly identifies events per the API docs
## Step 4: Validate Provider Handler
### Auth Verification
- [ ] `verifyAuth` correctly validates webhook signatures per the service's documentation
- [ ] HMAC algorithm matches (SHA-1, SHA-256, SHA-512)
- [ ] Signature header name matches the API docs exactly
- [ ] Signature format is handled (raw hex, `sha256=` prefix, base64, etc.)
- [ ] Uses `safeCompare` for timing-safe comparison (no `===`)
- [ ] If `webhookSecret` is required, handler rejects when it's missing (fail-closed)
- [ ] Signature is computed over raw body (not parsed JSON)
### Event Matching
- [ ] `matchEvent` returns `boolean` (not `NextResponse` or other values)
- [ ] Challenge/verification events are excluded from matching (e.g., `endpoint.url_validation`)
- [ ] When `triggerId` is a generic webhook ID, all events pass through
- [ ] When `triggerId` is specific, only matching events pass
- [ ] Event matching logic uses dynamic `await import()` for trigger utils (avoids circular deps)
### formatInput (CRITICAL)
- [ ] Every key in the `formatInput` return matches a key in the trigger `outputs` schema
- [ ] Every key in the trigger `outputs` schema is populated by `formatInput`
- [ ] No extra undeclared keys that users can't discover in the UI
- [ ] No wrapper objects (`webhook: { ... }`, `{service}: { ... }`)
- [ ] Nested output paths exist at the correct depth (e.g., `resource.id` actually has `resource: { id: ... }`)
- [ ] `null` is used for missing optional fields (not empty strings or empty objects)
- [ ] Returns `{ input: { ... } }` — not a bare object
### Idempotency
- [ ] `extractIdempotencyId` returns a stable, unique key per delivery
- [ ] Uses provider-specific delivery IDs when available (e.g., `X-Request-Id`, `Linear-Delivery`, `svix-id`)
- [ ] Falls back to content-based ID (e.g., `${type}:${id}`) when no delivery header exists
- [ ] Does NOT include timestamps in the idempotency key (would break dedup on retries)
### Challenge Handling (if applicable)
- [ ] `handleChallenge` correctly implements the service's URL verification handshake
- [ ] Returns the expected response format per the API docs
- [ ] Env-backed secrets are resolved via `resolveEnvVarsInObject` if needed
## Step 5: Validate Automatic Subscription Lifecycle
If the service supports programmatic webhook creation:
### createSubscription
- [ ] Calls the correct API endpoint to create a webhook
- [ ] Sends the correct event types/filters
- [ ] Passes the notification URL from `getNotificationUrl(ctx.webhook)`
- [ ] Returns `{ providerConfigUpdates: { externalId } }` with the external webhook ID
- [ ] Throws on failure (orchestration handles rollback)
- [ ] Provides user-friendly error messages (401 → "Invalid API Key", etc.)
### deleteSubscription
- [ ] Calls the correct API endpoint to delete the webhook
- [ ] Handles 404 gracefully (webhook already deleted)
- [ ] Never throws — catches errors and logs non-fatally
- [ ] Skips gracefully when `apiKey` or `externalId` is missing
### Orchestration Isolation
- [ ] NO provider-specific logic in `route.ts`, `provider-subscriptions.ts`, or `deploy.ts`
- [ ] All subscription logic lives on the handler (`createSubscription`/`deleteSubscription`)
## Step 6: Validate Registration and Block Wiring
### Trigger Registry (`triggers/registry.ts`)
- [ ] All triggers are imported and registered
- [ ] Registry keys match trigger IDs exactly
- [ ] No orphaned entries (triggers that don't exist)
### Provider Handler Registry (`providers/registry.ts`)
- [ ] Handler is imported and registered (if handler exists)
- [ ] Registry key matches the `provider` field on the trigger configs
- [ ] Entries are in alphabetical order
### Block Wiring (`blocks/blocks/{service}.ts`)
- [ ] Block has `triggers.enabled: true`
- [ ] `triggers.available` lists all trigger IDs
- [ ] All trigger subBlocks are spread into `subBlocks`: `...getTrigger('id').subBlocks`
- [ ] No trigger IDs in `triggers.available` that aren't in the registry
- [ ] No trigger subBlocks spread that aren't in `triggers.available`
## Step 7: Validate Security
- [ ] Webhook secrets are never logged (not even at debug level)
- [ ] Auth verification runs before any event processing
- [ ] No secret comparison uses `===` (must use `safeCompare` or `crypto.timingSafeEqual`)
- [ ] Timestamp/replay protection is reasonable (not too tight for retries, not too loose for security)
- [ ] Raw body is used for signature verification (not re-serialized JSON)
## Step 8: Report and Fix
### Report Format
Group findings by severity:
**Critical** (runtime errors, security issues, or data loss):
- Wrong HMAC algorithm or header name
- `formatInput` keys don't match trigger `outputs`
- Missing `verifyAuth` when the service sends signed webhooks
- `matchEvent` returns non-boolean values
- Provider-specific logic leaking into shared orchestration files
- Trigger IDs mismatch between trigger files, registry, and block
- `createSubscription` calling wrong API endpoint
- Auth comparison using `===` instead of `safeCompare`
**Warning** (convention violations or usability issues):
- Missing `extractIdempotencyId` when the service provides delivery IDs
- Timestamps in idempotency keys (breaks dedup on retries)
- Missing challenge handling when the service requires URL verification
- Output schema missing fields that `formatInput` returns (undiscoverable data)
- Overly tight timestamp skew window that rejects legitimate retries
- `matchEvent` not filtering challenge/verification events
- Setup instructions missing important steps
**Suggestion** (minor improvements):
- More specific output field descriptions
- Additional output fields that could be exposed
- Better error messages in `createSubscription`
- Logging improvements
### Fix All Issues
After reporting, fix every **critical** and **warning** issue. Apply **suggestions** where they don't add unnecessary complexity.
### Validation Output
After fixing, confirm:
1. `bun run type-check` passes
2. Re-read all modified files to verify fixes are correct
3. Provider handler tests pass (if they exist): `bun test {service}`
## Checklist Summary
- [ ] Read all trigger files, provider handler, types, registries, and block
- [ ] Pulled and read official webhook/API documentation
- [ ] Validated trigger definitions: options, instructions, extra fields, outputs
- [ ] Validated primary/secondary trigger distinction (`includeDropdown`)
- [ ] Validated provider handler: auth, matchEvent, formatInput, idempotency
- [ ] Validated output alignment: every `outputs` key ↔ every `formatInput` key
- [ ] Validated subscription lifecycle: createSubscription, deleteSubscription, no shared-file edits
- [ ] Validated registration: trigger registry, handler registry, block wiring
- [ ] Validated security: safe comparison, no secret logging, replay protection
- [ ] Reported all issues grouped by severity
- [ ] Fixed all critical and warning issues
- [ ] `bun run type-check` passes after fixes
@@ -0,0 +1,52 @@
---
description: Analyze and fix useCallback anti-patterns in your code
argument-hint: [scope] [fix=true|false]
---
# You Might Not Need a Callback
Arguments:
- scope: what to analyze (default: your current changes). Examples: "diff to main", "PR #123", "src/components/", "whole codebase"
- fix: whether to apply fixes (default: true). Set to false to only propose changes.
User arguments: $ARGUMENTS
## References
Read before analyzing:
1. https://react.dev/reference/react/useCallback — official docs on when useCallback is actually needed
## The one rule that matters
`useCallback` is only useful when **something observes the reference**. Ask: does anything care if this function gets a new identity on re-render?
Observers that care about reference stability:
- A `useEffect` that lists the function in its deps array
- A `useMemo` that lists the function in its deps array
- Another `useCallback` that lists the function in its deps array
- A child component wrapped in `React.memo` that receives the function as a prop
If none of those apply — if the function is only called inline, or passed to a non-memoized child, or assigned to a native element event — the reference is unobserved and `useCallback` adds overhead with zero benefit.
## Anti-patterns to detect
1. **No observer tracks the reference**: The function is only called inline in the same component, or passed to a non-memoized child, or used as a native element handler (`<button onClick={fn}>`). Nothing re-runs or bails out based on reference identity. Remove `useCallback`.
2. **useCallback with deps that change every render**: If a dep is a plain object/array created inline, or state that changes on every interaction, memoization buys nothing — the function gets a new identity anyway.
3. **useCallback on handlers passed only to native elements**: `<button onClick={fn}>` — React never does reference equality on native element props. No benefit.
4. **useCallback wrapping functions that return new objects/arrays**: Stable function identity, unstable return value — memoization is at the wrong level. Use `useMemo` on the return value instead, or restructure.
5. **useCallback with empty deps when deps are needed**: Stale closure — reads initial values forever. This is a correctness bug, not just a performance issue.
6. **Pairing useCallback + React.memo on trivially cheap renders**: If the child renders in < 1ms and re-renders rarely, the memo infrastructure costs more than it saves.
## Patterns that ARE correct — do not flag
- `useCallback` whose result is in a `useEffect` dep array — prevents the effect from re-running on every render
- `useCallback` whose result is in a `useMemo` dep array — prevents the memo from recomputing on every render
- `useCallback` whose result is a dep of another `useCallback` — stabilises a callback chain
- `useCallback` passed to a `React.memo`-wrapped child — the whole point of the pattern
- This codebase's ref pattern: `useRef` + callback with empty deps that reads the ref inside — correct, do not flag
## Steps
1. Read the reference above
2. Analyze the specified scope for the anti-patterns listed above
3. If fix=true, apply the fixes. If fix=false, propose the fixes without applying.
@@ -0,0 +1,33 @@
---
description: Analyze and fix useMemo/React.memo anti-patterns in your code
argument-hint: [scope] [fix=true|false]
---
# You Might Not Need a Memo
Arguments:
- scope: what to analyze (default: your current changes). Examples: "diff to main", "PR #123", "src/components/", "whole codebase"
- fix: whether to apply fixes (default: true). Set to false to only propose changes.
User arguments: $ARGUMENTS
## References
Read before analyzing:
1. https://overreacted.io/before-you-memo/ — two techniques to avoid memo entirely
## Anti-patterns to detect
1. **State can be moved down instead of memoizing**: Move state into a smaller child so the slow component stops re-rendering without memo.
2. **Children can be lifted up**: Extract stateful part, pass expensive subtree as `children` — children as props don't re-render when parent state changes.
3. **useMemo on cheap computations**: Small array filters, string concat, arithmetic don't need memoization.
4. **useMemo with constantly-changing deps**: Deps change every render = useMemo does nothing.
5. **useMemo to stabilize props for non-memoized children**: If the child isn't wrapped in React.memo, stable references don't matter.
6. **React.memo on components that always receive new props**: Fix the parent instead.
7. **useMemo for derived state**: Just compute inline during render.
## Steps
1. Read the reference above
2. Analyze the specified scope for the anti-patterns listed above
3. If fix=true, apply the fixes. If fix=false, propose the fixes without applying.
@@ -0,0 +1,17 @@
---
description: Analyze and fix useEffect anti-patterns in your code
argument-hint: [scope] [fix=true|false]
---
# You Might Not Need an Effect
Arguments:
- scope: what to analyze (default: your current changes). Examples: "diff to main", "PR #123", "src/components/", "whole codebase"
- fix: whether to apply fixes (default: true). Set to false to only propose changes.
User arguments: $ARGUMENTS
Steps:
1. Read https://react.dev/learn/you-might-not-need-an-effect to understand the guidelines
2. Analyze the specified scope for useEffect anti-patterns
3. If fix=true, apply the fixes. If fix=false, propose the fixes without applying.
@@ -0,0 +1,38 @@
---
description: Analyze and fix unnecessary useState, derived state, and server-state-in-local-state anti-patterns
argument-hint: [scope] [fix=true|false]
---
# You Might Not Need State
Arguments:
- scope: what to analyze (default: your current changes). Examples: "diff to main", "PR #123", "src/components/", "whole codebase"
- fix: whether to apply fixes (default: true). Set to false to only propose changes.
User arguments: $ARGUMENTS
## Context
This codebase uses React Query for all server state and Zustand for client-only global state. useState should only be used for ephemeral UI concerns (open/closed, hover, local form input). Server data should never be copied into useState or Zustand — React Query is the single source of truth.
## References
Read these before analyzing:
1. https://react.dev/learn/choosing-the-state-structure — 5 principles for structuring state
2. https://tkdodo.eu/blog/dont-over-use-state — never store derived/computed values in state
3. https://tkdodo.eu/blog/putting-props-to-use-state — never mirror props into state via useEffect
## Anti-patterns to detect
1. **Derived state stored in useState**: If a value can be computed from props, other state, or query data, compute it inline during render instead of storing it in state.
2. **Server state copied into useState**: Never `useState` + `useEffect` to sync React Query data into local state. Use query data directly. The only exception is forms where users edit server data.
3. **Props mirrored into state**: Never `useState(prop)` + `useEffect(() => setState(prop))`. Use the prop directly, or use a key to reset component state.
4. **Chained useEffect state updates**: Never chain Effects that set state to trigger other Effects. Calculate all derived values in the event handler or inline during render.
5. **Storing objects when an ID suffices**: Store `selectedId` not a copy of the selected object. Derive the object: `items.find(i => i.id === selectedId)`.
6. **State that duplicates Zustand or React Query**: If the data already lives in a store or query cache, don't create a parallel useState.
## Steps
1. Read the references above to understand the guidelines
2. Analyze the specified scope for the anti-patterns listed above
3. If fix=true, apply the fixes. If fix=false, propose the fixes without applying.
@@ -0,0 +1,45 @@
---
description: Analyze and fix URL/query-param state anti-patterns — manual useSearchParams reads, hand-built query mutations, view-state trapped in useState, and objects in the URL
argument-hint: [scope] [fix=true|false]
---
# You Might Not Need URL State
Arguments:
- scope: what to analyze (default: your current changes). Examples: "diff to main", "PR #123", "app/workspace/[workspaceId]/tables/", "whole codebase"
- fix: whether to apply fixes (default: true). Set to false to only propose changes.
User arguments: $ARGUMENTS
## Context
Shareable client view-state (active tab/panel, filters, search query, sort, pagination, selected-entity id, an open "view" modal/drawer that is a destination) lives in the URL via [`nuqs`](https://nuqs.dev) — driven by a co-located `search-params.ts`, never read via `useSearchParams().get(...)` and never mutated by hand-built query strings. Remote data stays in React Query; high-frequency / large / ephemeral / socket-synced state stays in Zustand; purely local UI stays in `useState`.
`.claude/rules/sim-url-state.md` is the source of truth — read it first.
## References
Read these before analyzing:
1. `.claude/rules/sim-url-state.md` — the decision framework, conventions, debounced-input pattern, sort convention, selected-entity deep-link pattern, and the workflow-editor carve-out
2. https://nuqs.dev/docs/parsers — parsers (`parseAsString`/`parseAsInteger`/`parseAsBoolean`/`parseAsStringLiteral`/`parseAsArrayOf`/`createParser`)
3. https://nuqs.dev/docs/options — `withDefault`, `history`, `shallow`, `clearOnDefault`
4. https://nuqs.dev/docs/server-side — `createSearchParamsCache` for server reads
## Anti-patterns to detect
1. **Manual param reads for state**: `useSearchParams().get(...)` or `new URLSearchParams(window.location.search)` used to *read* view-state. Replace with `useQueryState`/`useQueryStates` bound to a `search-params.ts`. (Read-once auth/invite/redirect tokens — `token`, `callbackUrl`, `redirect`, `error`, `invite_flow`, `code` — are NOT view-state; leave them on `useSearchParams`.)
2. **Hand-built query mutation**: constructing a query string + `router.replace`/`router.push` to change a param on the current path. Use a nuqs setter. (A `router.push` that changes the route *path* is fine; an outbound `new URLSearchParams` building an `href`/`window.open`/download/API URL is fine.)
3. **`window.history.replaceState`/`pushState`** to mutate a param.
4. **URL state duplicated into a store/useState + synced with an effect** (or a `popstate` listener). The URL is the single source of truth; derive from it, don't mirror it.
5. **Objects in the URL**: serializing a `TableDefinition`/`SkillDefinition`/etc. Store the id and derive the object from the loaded list (`items.find(i => i.id === id)`).
6. **High-frequency / large state in the URL**: cursor, pan/zoom, un-debounced keystrokes, big JSON blobs. Debounce text search (local `useState` mirror + reconcile effect); keep canvas/presence/resize state in Zustand.
7. **Shareable view-state trapped in `useState`**: a tab/filter/sort/pagination/selected-entity that should be a link but lives in local state. Migrate it to the URL.
8. **Missing Suspense boundary**: a component newly calling `useQueryState`/`useQueryStates` whose page entry has no `<Suspense>` wrapper (Next.js requires it for `useSearchParams`). Add one with a real-chrome fallback.
9. **`import { z }` for param validation in client code**: use nuqs parsers instead.
## Steps
1. Read `.claude/rules/sim-url-state.md` and the nuqs docs above to understand the guidelines
2. Analyze the specified scope for the anti-patterns listed above
3. For each finding, decide the correct home using the decision table — do not force URL state onto ephemeral/high-frequency/socket-synced state
4. If fix=true, apply the fixes (co-locate a `search-params.ts`, wire `useQueryState(s)`, add the Suspense boundary, delete the replaced state + sync effects). If fix=false, propose the fixes without applying.
+73
View File
@@ -0,0 +1,73 @@
# Sim — Language & Positioning
When editing user-facing copy (landing pages, docs, metadata, marketing), follow these rules.
## Identity
Sim is the **AI workspace** where teams build and run AI agents. Not a workflow tool, not an agent framework, not an automation platform.
**Short definition:** Sim is the open-source AI workspace where teams build, deploy, and manage AI agents.
**Full definition:** Sim is the open-source AI workspace where teams build, deploy, and manage AI agents. Connect 1,000+ integrations and every major LLM to create agents that automate real work — visually, conversationally, or with code.
## Audience
**Primary:** Teams building AI agents for their organization — IT, operations, and technical teams who need governance, security, lifecycle management, and collaboration.
**Secondary:** Individual builders and developers who care about speed, flexibility, and open source.
## Required Language
| Concept | Use | Never use |
|---------|-----|-----------|
| The product | "AI workspace" | "workflow tool", "automation platform", "agent framework" |
| Building | "build agents", "create agents" | "create workflows" (unless describing the workflow module specifically) |
| Visual builder | "workflow builder" or "visual builder" | "canvas", "graph editor" |
| The agent | "Sim" — you talk to Sim | "Mothership", "copilot", "AI assistant" |
| The chat surface | "Chat" (capitalized, the module) | "Mothership", "copilot" |
| Deployment | "deploy", "ship" | "publish", "activate" |
| Audience | "teams", "builders" | "users", "customers" (in marketing copy) |
| What agents do | "automate real work" | "automate tasks", "automate workflows" |
| Our advantage | "open-source AI workspace" | "open-source platform" |
## Tone
- **Direct.** Short sentences. Active voice. Lead with what it does.
- **Concrete.** Name specific things — "Slack bots, compliance agents, data pipelines" — not abstractions.
- **Confident, not loud.** No exclamation marks or superlatives.
- **Simple.** If a 16-year-old can't understand the sentence, rewrite it.
## Claim Hierarchy
When describing Sim, always lead with the most differentiated claim:
1. **What it is:** "The AI workspace for teams"
2. **What you do:** "Build, deploy, and manage AI agents"
3. **How:** "Visually, conversationally, or with code"
4. **Scale:** "1,000+ integrations, every major LLM"
5. **Trust:** "Open source. SOC2. Trusted by 100,000+ builders."
## Module Descriptions
| Module | One-liner |
|--------|-----------|
| **Chat** | Your AI command center. Talk to Sim — build and manage everything in natural language. |
| **Workflows** | The visual builder. Connect blocks, models, and integrations into agent logic. |
| **Knowledge Base** | Your agents' memory. Upload docs, sync sources, build vector databases. |
| **Tables** | A database, built in. Store, query, and wire structured data into agent runs. |
| **Files** | Upload, create, and share. One store for your team and every agent. |
| **Logs** | Full visibility, every run. Trace execution block by block. |
## What We Never Say
- Never call Sim "just a workflow tool"
- Never compare only on integration count — we win on AI-native capabilities
- Never use "no-code" as the primary descriptor — say "visually, conversationally, or with code"
- Never promise unshipped features
- Never use jargon ("RAG", "vector database", "MCP") without plain-English explanation on public pages
- Avoid "agentic workforce" as a primary term — use "AI agents"
- Never say "Mothership" or "copilot" — the agent is "Sim", the surface is "Chat" (in run logs the trigger reads "Sim agent")
## Vision
Sim becomes the default environment where teams build AI agents — not a tool you visit for one task, but a workspace you live in. Workflows are one module; Chat is another. The workspace is the constant; the interface adapts.
+45
View File
@@ -0,0 +1,45 @@
---
paths:
- "apps/sim/components/emcn/**"
---
# EMCN Components
Import components, `cn`, and tokens from the `@sim/emcn` barrel; icons come from the `@sim/emcn/icons` subpath, and CSS modules from their file path. Never deep-import other component subpaths. The **chip family** is the platform's primary chrome — always reach for it over the legacy primitives it is progressively replacing (`Input``ChipInput`, `Textarea``ChipTextarea`, `Modal``ChipModal`, `Select`/`Combobox``ChipSelect`/`ChipCombobox`/`ChipDropdown`, `Switch``ChipSwitch`, date field→`ChipDatePicker`). For context/action menus the canonical control is `DropdownMenu` — the standard menu (not a chip, and never a hand-rolled popover).
## Chip chrome — single source of truth
Never hand-roll the chip pill from raw class strings (they go stale). Compose from the canonical sources:
- **Surface, typography + content tokens:** `chip/chip-chrome.ts``chipFilledSurfaceTokens`, `chipFieldSurfaceClass`, `chipFieldTextClass` (text fields and the dropdown search box build on these), plus the chip-content chrome `chipContentGap`, `chipGeometryClass`, `chipContentIconClass`, `chipContentLabelClass`, and `cellIconNodeClass` (non-chip surfaces that must visually match chip content, e.g. resource table cells). All are re-exported from the `@sim/emcn` barrel — no subpath import needed.
- **Pill geometry:** `chip/chip.tsx``chipVariants` (30px tall, `rounded-lg`, `px-2`, icon↔text `gap-1.5`). Every pill-shaped trigger (`ChipDropdown`, `ChipSelect`, `ChipSwitch`) reuses it for visual parity.
Canonical look: normal font-weight (never `font-medium`/`font-semibold`), value text `--text-body`, icons `--text-icon` at `size-[14px]`, placeholder `--text-muted`, `transition-colors`, **no focus ring** (the caret marks focus). Filled surface is `--surface-5` light / `--surface-4` dark with a `--border-1` border.
The menu surface intentionally diverges from the pill: `dropdown-menu.tsx` items use `text-small` and `gap-2` (a menu convention, not the chip pill). Keep them distinct.
## Component catalogue
- **`Chip` / `ChipLink`** — the pill button (`<button>` / Next `<Link>`). Variants: `ghost`, `filled`, `primary`, `destructive`, `border-shadow`. `leftIcon`/`rightIcon`, `active`, `fullWidth`, `flush`.
- **`ChipInput`** — single-line text field. `icon`, `endAdornment`, `error`, `inputClassName` (inner `<input>`); `className` styles the chrome wrapper.
- **`ChipCopyInput`** — the canonical view-only field: a read-only `ChipInput` at full opacity with a trailing copy-to-clipboard button. View-only is a display mode, not a disabled state — reach for it (or `ChipModalField type='copy'`) over a `disabled` (greyed) input for values the user cannot edit.
- **`ChipTextarea`** — multi-line sibling. `error`, `resizable` (off by default), `viewOnly` (read-only at full opacity with the default cursor — the multi-line counterpart of `ChipCopyInput`).
- **`ChipDropdown`** — pill that opens a menu. Single OR multi-select via the discriminated `multiple` prop (one component, not two). Owns its trailing chevron — no `rightIcon`.
- **`ChipSelect` / `ChipCombobox`** — `Combobox`-backed pickers with search, groups, multi-select; for richer lists than `ChipDropdown`.
- **`ChipModal` + `ChipModalField`** — declarative compact modal. The field's `type` (`input` | `email` | `textarea` | `dropdown` | `copy` | `file` | `emails` | `custom`) picks the control and **owns all chrome** — consumers describe intent, never pass `variant`/`className`/`id` to the inner control. `custom` is the escape hatch. **Every body field MUST be a `ChipModalField`** — never hand-roll a field row (raw `<div>` + hand-rolled `<p>`/`<label>` title + bare `ChipInput`/`ChipTextarea`). `ChipModalBody` applies `px-2` + `gap-4`; `ChipModalField` adds another `px-2`, so each field lands at effective `px-4`, exactly matching the `px-4` header/footer — a hand-rolled row skips that gutter and sits misaligned at `px-2`. For controls the field doesn't cover (`ChipCombobox`, `ChipSelect`, `DatePicker`, `TimePicker`, `ButtonGroup`, arbitrary JSX), use `type='custom'` with a `title` — it still applies the gutter and renders the canonical `Label`.
- **`ChipSwitch`** — segmented pill control (built from `chipVariants`).
- **`ChipTag`** — 20px inline tag/badge (`mono`/`gray`/`invite`), not a pill trigger.
- **`ChipDatePicker`** — chip-styled date field.
- **`ChipTimePicker`** — minute-granular time sibling of `ChipDatePicker`, a `ChipInput` that leniently parses typed input (`9:47`, `947`, `2:05pm`, `14:30`), commits on Enter/blur, and re-renders the canonical `9:47 AM` label.
- **`DropdownMenu`** — the canonical context/action menu (Radix-backed). Not a chip, but the standard menu for command/action lists; reach for it instead of a hand-rolled popover. Its surface intentionally diverges from the chip pill (`text-small`, `gap-2`) — keep them distinct. For a pill that opens a value picker, use `ChipDropdown`/`ChipSelect` instead.
## Authoring principles
- **One source of truth for shared chrome.** Compose from `chip-chrome.ts` / `chipVariants`; never duplicate the chrome string.
- **`cn()` for a single state toggle, CVA for genuine multiple variants.** A lone `error` boolean is `cn()`, not a CVA variant.
- **Discriminated-union props for modes** (e.g. `multiple`, the modal field `type`) instead of near-duplicate components.
- **Delete legacy variants after migration** — don't leave dead paths (this paradigm removed `Input variant='chip'` and `ChipMultiSelect`).
- **Verify CSS vars exist.** An undefined var resolves to `currentColor` (caused a real black-border bug). Align to the canonical tokens: normal weight, `--text-body`, `--text-icon`.
- Use Radix UI primitives for accessibility. Export the component and its `variants` (when using CVA). Document with TSDoc + a usage example.
Color tokens and icon-size conventions are canonical in `.claude/rules/sim-styling.md` — follow it rather than restating.
+67
View File
@@ -0,0 +1,67 @@
# Global Standards
## Logging
Import `createLogger` from `@sim/logger`. Use `logger.info`, `logger.warn`, `logger.error` instead of `console.log`. Inside API routes wrapped with `withRouteHandler`, loggers automatically include the request ID.
## API Route Handlers
All API route handlers must be wrapped with `withRouteHandler` from `@/lib/core/utils/with-route-handler`. Never export a bare `async function GET/POST/...` — always use `export const METHOD = withRouteHandler(...)`.
## Comments
Use TSDoc for documentation. No `====` separators. No non-TSDoc comments.
## Styling
Never update global styles. Keep all styling local to components.
## ID Generation
Never use `crypto.randomUUID()`, `nanoid`, or the `uuid` package directly. Use the utilities from `@sim/utils/id`:
- `generateId()` — UUID v4, use by default
- `generateShortId(size?)` — short URL-safe ID (default 21 chars), for compact identifiers
Both use `crypto.getRandomValues()` under the hood and work in all contexts including non-secure (HTTP) browsers.
```typescript
// ✗ Bad
import { nanoid } from 'nanoid'
import { v4 as uuidv4 } from 'uuid'
const id = crypto.randomUUID()
// ✓ Good
import { generateId, generateShortId } from '@sim/utils/id'
const uuid = generateId()
const shortId = generateShortId()
const tiny = generateShortId(8)
```
## Common Utilities
Use shared helpers from `@sim/utils` instead of writing inline implementations:
- `sleep(ms)` from `@sim/utils/helpers` — async delay. Never write `new Promise(resolve => setTimeout(resolve, ms))`
- `toError(value)` from `@sim/utils/errors` — normalize unknown caught values to `Error`. Never write `e instanceof Error ? e : new Error(String(e))`
- `getErrorMessage(value, fallback?)` from `@sim/utils/errors` — extract error message string. Never write `e instanceof Error ? e.message : 'fallback'`
- `structuredClone(value)` — built-in deep clone, no import needed. Never write `JSON.parse(JSON.stringify(obj))`
- `omit(obj, keys)` from `@sim/utils/object` — remove keys from object
- `filterUndefined(obj)` from `@sim/utils/object` — strip undefined-valued keys. Never write `Object.fromEntries(Object.entries(obj).filter(([, v]) => v !== undefined))`
- `truncate(str, maxLength, suffix?)` from `@sim/utils/string` — safe string truncation with ellipsis
- `backoffWithJitter(attempt, retryAfterMs, options?)` from `@sim/utils/retry` — exponential backoff with jitter
- `parseRetryAfter(header)` from `@sim/utils/retry` — parse HTTP `Retry-After` header to milliseconds
```typescript
// ✗ Bad
await new Promise(resolve => setTimeout(resolve, 1000))
const msg = error instanceof Error ? error.message : 'Unknown error'
const clone = JSON.parse(JSON.stringify(obj))
const filtered = Object.fromEntries(Object.entries(obj).filter(([, v]) => v !== undefined))
// ✓ Good
import { sleep } from '@sim/utils/helpers'
import { getErrorMessage, toError } from '@sim/utils/errors'
import { filterUndefined } from '@sim/utils/object'
await sleep(1000)
const msg = getErrorMessage(error, 'Unknown error')
const clone = structuredClone(obj)
const filtered = filterUndefined(obj)
```
## Package Manager
Use `bun` and `bunx`, not `npm` and `npx`.
+26
View File
@@ -0,0 +1,26 @@
---
paths:
- "apps/sim/app/(landing)/**/*.tsx"
---
# Landing Page — SEO / GEO
## SEO
- One `<h1>` per page, in Hero only — never add another.
- Strict heading hierarchy: H1 (Hero) → H2 (section titles) → H3 (feature names).
- Every section: `<section id="…" aria-labelledby="…-heading">`.
- Decorative/animated elements: `aria-hidden="true"`.
- All internal routes use Next.js `<Link>` (crawlable). External links get `rel="noopener noreferrer"`.
- Navbar is a Server Component (no `'use client'`) for immediate crawlability. Logo `<Image>` has `priority` (LCP element).
- Navbar `<nav>` carries `SiteNavigationElement` schema.org markup.
- Feature lists must stay in sync with `WebApplication.featureList` in `structured-data.tsx`.
## GEO (Generative Engine Optimisation)
- **Answer-first pattern**: each section's H2 + subtitle should directly answer a user question (e.g. "What is Sim?", "How fast can I deploy?").
- **Atomic answer blocks**: each feature / template card should be independently extractable by an AI summariser.
- **Entity consistency**: always write "Sim" by name — never "the platform" or "our tool".
- **Keyword density**: first 150 visible chars of Hero must name "Sim", "AI workspace", "AI agents".
- **sr-only summaries**: Hero and Templates each have a `<p className="sr-only">` (~50 words) as an atomic product/catalog summary for AI citation.
- **Specific numbers**: prefer concrete figures ("1,000+ integrations", "15+ AI providers") over vague claims.
+71
View File
@@ -0,0 +1,71 @@
---
paths:
- "apps/sim/**"
---
# Sim App Architecture
## Core Principles
1. **Single Responsibility**: Each component, hook, store has one clear purpose
2. **Composition Over Complexity**: Break down complex logic into smaller pieces
3. **Type Safety First**: TypeScript interfaces for all props, state, return types
4. **Predictable State**: Zustand for global state, useState for UI-only concerns
## Root-Level Structure
```
apps/
├── sim/ # this app (Next.js: UI + API routes + workflow editor)
│ ├── app/ # Next.js app router (pages, API routes)
│ ├── blocks/ # Block definitions and registry
│ ├── components/ # Shared UI (emcn/, ui/)
│ ├── executor/ # Workflow execution engine
│ ├── hooks/ # Shared hooks (queries/, selectors/)
│ ├── lib/ # App-wide utilities
│ ├── providers/ # LLM provider integrations
│ ├── stores/ # Zustand stores
│ ├── tools/ # Tool definitions
│ └── triggers/ # Trigger definitions
└── realtime/ # Bun Socket.IO server (collaborative canvas)
packages/ # @sim/* — audit, auth, db, logger, realtime-protocol,
# security, tsconfig, utils, platform-authz,
# workflow-persistence, workflow-types
```
## Package Boundaries
- `apps/* → packages/*` only. Packages never import from `apps/*`.
- `apps/realtime` avoids Next.js, React, the block/tool registry, provider SDKs, and the executor; never add `@/lib/webhooks/providers/*`, `@/executor/*`, `@/blocks/*`, or `@/tools/*` imports to any package it consumes. CI enforces this via `scripts/check-monorepo-boundaries.ts` and `scripts/check-realtime-prune-graph.ts`.
## The `'use client'` server boundary
Every export of a `'use client'` module becomes a *client reference* on the server — server-evaluated code (RSC pages/layouts, `prefetch.ts`, route handlers, block definitions, triggers) can only *render* it as a component or pass it as a prop, never *call* it (doing so throws at runtime, e.g. `tableKeys.list is not a function`; `next build` does not catch it). Keep server-importable query primitives (key factories, fetchers, mappers, constants) in non-`'use client'` modules — see `.claude/rules/sim-queries.md`. Enforced by `scripts/check-client-boundary-imports.ts`.
## Feature Organization
Features live under `app/workspace/[workspaceId]/`:
```
feature/
├── components/ # Feature components
├── hooks/ # Feature-scoped hooks
├── utils/ # Feature-scoped utilities (2+ consumers)
├── feature.tsx # Main component
└── page.tsx # Next.js page entry
```
## Naming Conventions
- **Components**: PascalCase (`WorkflowList`)
- **Hooks**: `use` prefix (`useWorkflowOperations`)
- **Files**: kebab-case (`workflow-list.tsx`)
- **Stores**: `stores/feature/store.ts`
- **Constants**: SCREAMING_SNAKE_CASE
- **Interfaces**: PascalCase with suffix (`WorkflowListProps`)
## Utils Rules
- **Never create `utils.ts` for single consumer** - inline it
- **Create `utils.ts` when** 2+ files need the same helper
- **Check existing sources** before duplicating (`lib/` has many utilities)
- **Location**: `lib/` (app-wide) → `feature/utils/` (feature-scoped) → inline (single-use)
+33
View File
@@ -0,0 +1,33 @@
---
paths:
- "apps/sim/**/*.tsx"
---
# Component Patterns
Component authoring rules — structure order (refs → external hooks → store hooks → custom hooks → state → useMemo → useCallback → useEffect → return), required props interface, and extraction thresholds (50+ lines / 2+ files vs keep inline < 10 lines) — live in CLAUDE.md > Components.
`.tsx`-specific deltas not covered there:
- `'use client'` only when using React hooks or browser-only APIs.
- Prefer semantic HTML (`aside`, `nav`, `article`).
- Optional-chain callbacks: `onAction?.(id)`.
## List-render performance
When rendering or sorting a list of rows against a lookup collection (members, folders, tags), keep the per-row work O(1):
- **Precompute a lookup `Map` once**, never `array.find(...)` per row. Build `const byId = useMemo(() => { const m = new Map<string, T>(); for (const x of items ?? []) m.set(x.id, x); return m }, [items])` and read `byId.get(id)` in the sort comparator, `.map(...)`, and cell builders. A `.find` inside a sort comparator is O(n²·log n) — the worst offender. Depend memos on the derived `Map`, not the raw array.
- **Sort with `[...array].sort(cmp)` in client code — NOT `array.toSorted(cmp)`.** SWC (Next's compiler) transforms syntax but does not polyfill prototype methods, and the repo sets no core-js/browserslist target, so `toSorted`/`toReversed`/`toSpliced`/`Array.prototype.with` ship as-is and throw `TypeError` on Safari <16 / iOS 15 (they landed in Safari 16). `tsconfig` `lib: ES2023` only affects type-checking, never the runtime. The `[...arr].sort()` spread copy is the intended cost — it keeps the sort non-mutating without an unpolyfilled builtin. These methods are only safe in server-only modules (no `'use client'`, executed on Node ≥20). react-doctor's `js-tosorted-immutable` is therefore a won't-fix in client components.
- **Partition in a single pass** — when splitting one collection into several (`fileIds`/`folderIds`), do one `for…of` pushing into each bucket and return `{ a, b }` from a single `useMemo`, not two memos that each `map→filter→map` the same source twice.
## react-doctor (`npx react-doctor`) — apply the wins, skip the false positives
react-doctor diagnostics are hypotheses, not verdicts — confirm against the code before acting, and preserve behavior. Known repo-specific false positives to NOT "fix":
- `no-barrel-import` — barrel imports are the repo convention (see sim-imports.md, "Barrel Exports"). Keep them.
- `js-tosorted-immutable` — in `'use client'` code, keep `[...arr].sort(cmp)`; `toSorted` is unpolyfilled and crashes Safari <16 / iOS 15 (see "List-render performance" above). Only apply it in server-only modules.
- `rerender-state-only-in-handlers` / "state set but never rendered" — a false positive when the `useState` is consumed by a `useEffect`/`useLayoutEffect` dependency (the effect must re-run on change). Only convert to a ref when nothing reads the value reactively.
- `no-render-in-render` — a helper *called inline* (`{renderRow()}`) is reconciled by position and does **not** remount, so extracting it to a component is usually pure churn and can regress behavior (prop-drilling many closures, focus/scroll loss on the inner `<input>`). Apply it only when the helper is genuinely a *component defined during render*, or when the move is mechanical (a stateless, ref-free helper whose closures become a small, explicit prop set).
- `async-await-in-loop` on an upload/progress loop where sequential execution is intentional (per-item progress, server backpressure) — leave it.
- Broad refactors (`prefer-useReducer` for many `useState`, `no-giant-component` splits) — out of scope for a perf pass; note, don't churn.
+74
View File
@@ -0,0 +1,74 @@
---
paths:
- "apps/sim/**/use-*.ts"
- "apps/sim/**/hooks/**/*.ts"
---
# Hook Patterns
## Structure
For server data, use a React Query hook from `hooks/queries/` — do NOT `useState` + `fetch` here (see `.claude/rules/sim-queries.md`). This pattern is for UI/orchestration hooks that hold UI-only state and wrap callbacks.
```typescript
interface UseFeatureProps {
id: string
onSelect?: (item: Item) => void
}
export function useFeature({ id, onSelect }: UseFeatureProps) {
// 1. Refs for stable dependencies
const idRef = useRef(id)
const onSelectRef = useRef(onSelect)
// 2. UI-only state (never server data)
const [isOpen, setIsOpen] = useState(false)
// 3. Sync refs
useEffect(() => {
idRef.current = id
onSelectRef.current = onSelect
}, [id, onSelect])
// 4. Operations (useCallback with empty deps when using refs)
const select = useCallback((item: Item) => {
onSelectRef.current?.(item)
setIsOpen(false)
}, [])
return { isOpen, setIsOpen, select }
}
```
## Rules
1. Single responsibility per hook
2. Props interface required
3. Refs for stable callback dependencies
4. Wrap returned functions in useCallback
5. Server data goes through React Query (`hooks/queries/`), never `useState` + `fetch`
6. Keep only UI/orchestration state in these hooks
## State shape
Never mirror a prop into state with `useState(prop)` + a syncing `useEffect` — a prop change clobbers in-progress local edits. Use the prop directly, reset via a remount `key`, or — when you must seed local state from a prop only on a transition (e.g. a modal opening) — adjust it **during render** with a `prev`-value tracker held in `useState`:
```typescript
const [prevOpen, setPrevOpen] = useState(open)
if (prevOpen !== open) {
setPrevOpen(open)
if (open) setName(initialName) // closed → open only
}
```
React re-renders immediately with the corrected state without committing the stale value. Rules: the `if (prev !== current)` guard is mandatory (an unconditional `setState` in render loops forever), the tracker is set **inside** the guard, and you may only set the currently-rendering component's state this way. Hold the tracker in `useState`, **not a `useRef`** — React forbids reading/writing `ref.current` during render (react.dev, useRef → "Do not write _or read_ `ref.current` during rendering"; the `react-hooks` `refs` lint flags it), and a `useState` tracker is concurrent-safe where a mutated ref is not (a discarded render rolls state back, not a ref).
**The tracker's initial value decides mount behavior — choose it deliberately.** The example seeds `useState(open)` because the modal mounts closed, so the first render's guard is `false` and nothing resets on mount (correct — `name` is already at its initial value). When the effect you're replacing did real work **on mount** — opening a panel because a prop already matches, seeding editable state from an already-present value, or a component that can mount in the active state — seed a **sentinel** the live value can't equal (e.g. `useState<T | null>(null)`), otherwise the guard is `false` on the first render and that mount action is silently dropped. Place the block before any early `return`.
> Some existing components use a `useRef` prev-tracker (`if (prevRef.current !== x) { prevRef.current = x; … }`). It works but reads/writes a ref during render — prefer the `useState` form above for new code.
Only convert a `useState` to a `useRef` when the value is **never read during render/JSX and is never a hook dependency** — a value in a `useEffect`/`useMemo`/`useCallback` dep array must re-run the hook on change, so it stays state (see also `rerender-state-only-in-handlers` in `sim-components.md`). Convert only set-only values read solely inside handlers or effect bodies (e.g. a prompt-history index, a pending-upload URL). If a ref feeds render, mutating it won't re-render and the UI goes stale.
Model mutually-exclusive flags as ONE `status` enum, not several contradictory booleans. `isLoading`/`isVerified`/`isInvalidOtp` describing one machine collapse to `status: 'idle' | 'verifying' | 'verified' | 'error'` (+ `errorMessage`); derive any boolean a consumer still needs (`status === 'error'`).
Derive busy/success from the mutation object — never duplicate `mutation.isPending`/`mutation.isSuccess` into local `useState`. Read them directly (`mutation.isSuccess`) and reset with `mutation.reset()`. A distinct phase the mutation doesn't cover — e.g. a pre-submit captcha/Turnstile gate that runs before `mutate()` — is not a duplicate; keep that flag.
+76
View File
@@ -0,0 +1,76 @@
---
paths:
- "apps/sim/**/*.ts"
- "apps/sim/**/*.tsx"
---
# Import Patterns
## Absolute Imports
**Always use absolute imports.** Never use relative imports.
```typescript
// ✓ Good
import { useWorkflowStore } from '@/stores/workflows/store'
import { Button } from '@/components/ui/button'
// ✗ Bad
import { useWorkflowStore } from '../../../stores/workflows/store'
```
## Barrel Exports
Use barrel exports (`index.ts`) when a folder has 3+ exports. Import from barrel, not individual files.
```typescript
// ✓ Good
import { Dashboard, Sidebar } from '@/app/workspace/[workspaceId]/logs/components'
// ✗ Bad
import { Dashboard } from '@/app/workspace/[workspaceId]/logs/components/dashboard/dashboard'
```
## Code-splitting through barrels
When you `lazy(() => import(...))` a component to keep it out of a route's initial bundle, import the **deep module path** (`./components/foo/foo`), never the barrel — and **delete the now-dead barrel re-export** of that component. This app has no `"sideEffects": false` in `apps/sim/package.json`, so when any sibling still imports that barrel, webpack can conservatively keep the barrel's re-export edge to the heavy module. A leftover `export { Foo } from './foo'` line can therefore drag `Foo` (and its transitive deps) back into the initial chunk and silently defeat the split. Removing the dead re-export is the guaranteed fix; verify with a production bundle diff, not by eyeballing the `lazy()` call.
```typescript
// ✓ Good — deep lazy import + no barrel edge left behind
const MothershipView = lazy(() =>
import('./components/mothership-view/mothership-view').then((m) => ({ default: m.MothershipView }))
)
// (and remove `export { MothershipView } from './mothership-view'` from components/index.ts)
```
Wrap the lazy component in a **local `<Suspense>`** so its suspension resolves at the nearest boundary instead of bubbling to the page-level fallback (which would flash the whole route). `React.lazy(memo(forwardRef(...)))` forwards a DOM `ref` correctly in React 19 — but during the fallback window `ref.current` is `null`, so every consumer must guard it (`if (!el) return` / `el?.`).
## No Re-exports
Do not re-export from non-barrel files. Import directly from the source.
```typescript
// ✓ Good - import from where it's declared
import { CORE_TRIGGER_TYPES } from '@/stores/logs/filters/types'
// ✗ Bad - re-exporting in utils.ts then importing from there
import { CORE_TRIGGER_TYPES } from '@/app/workspace/.../utils'
```
## Import Order
1. React/core libraries
2. External libraries
3. UI components (`@sim/emcn`, `@/components/ui`)
4. Utilities (`@/lib/...`)
5. Stores (`@/stores/...`)
6. Feature imports
7. CSS imports
## Type Imports
Use `type` keyword for type-only imports:
```typescript
import type { WorkflowLog } from '@/stores/logs/types'
```
+19
View File
@@ -0,0 +1,19 @@
---
paths:
- "apps/sim/tools/**"
- "apps/sim/blocks/**"
- "apps/sim/triggers/**"
---
# Adding Integrations
Build order: **Tools** (`tools/{service}/`) → **Block** (`blocks/blocks/{service}.ts`) → **Icon** (`components/icons.tsx`) → optional **Trigger** (`triggers/{service}/`). Always look up the service's API docs first.
The full authoring instructions — tool/block/icon/trigger scaffolding, SubBlock property tables, `condition`/`required`/`dependsOn`/`mode`/`canonicalParamId` syntax, the `normalizeFileInput`/file-handling helper table, and checklists — live in the skills: `/add-integration` (end-to-end), `/add-tools`, `/add-block`, `/add-trigger`.
## Hard rules (don't get these wrong)
- Tool IDs are `snake_case` (`service_action`). Register tools in `tools/registry.ts`, blocks in `blocks/registry-maps.ts` (the `BLOCK_REGISTRY` config map + `BLOCK_META_REGISTRY` catalog-meta map, alphabetically — `blocks/registry.ts` holds only the accessor functions), triggers in `triggers/registry.ts`.
- Type coercions (`Number()`, etc.) belong in `tools.config.params` (runs at execution, after variable resolution) — never in `tools.config.tool` (runs at serialization; coercing there destroys dynamic `<Block.output>` references).
- `canonicalParamId` must NOT match any subblock's `id`, must be unique per operation/condition context, and all subblocks in a canonical group must share the same `required` status. The `inputs` section and the params function reference canonical IDs, not raw subblock IDs.
- Blocks must also set the catalog/UI metadata fields `integrationType`, `tags`, `authMode`, `docsLink`, and export a `{Service}BlockMeta` — see the `/add-block` skill's BlockMeta section for details.
+160
View File
@@ -0,0 +1,160 @@
---
paths:
- "apps/sim/hooks/queries/**/*.ts"
---
# React Query Patterns
All React Query hooks live in `hooks/queries/`. All server state must go through React Query — never use `useState` + `fetch` in components for data fetching or mutations.
For *client* view-state that belongs in a shareable link (tabs, filters, search, pagination, selected entity id), use URL query params via nuqs — see `.claude/rules/sim-url-state.md`. React Query owns remote data; nuqs owns shareable client view-state.
## Query Key Factory
Every query file defines a hierarchical keys factory with an `all` root key and intermediate plural keys for prefix-level invalidation:
```typescript
export const entityKeys = {
all: ['entity'] as const,
lists: () => [...entityKeys.all, 'list'] as const,
list: (workspaceId?: string) => [...entityKeys.lists(), workspaceId ?? ''] as const,
details: () => [...entityKeys.all, 'detail'] as const,
detail: (id?: string) => [...entityKeys.details(), id ?? ''] as const,
}
```
Never use inline query keys — always use the factory.
**Every identifier the `queryFn` forwards into the fetch MUST appear in the `queryKey`.** (Query-machinery identifiers — `signal`, `pageParam` — are exempt; they aren't fetch-scoping args.) If the fetch is scoped by `workspaceId`, `cursor`, `limit`, an org id, etc., those values must be part of the key — otherwise distinct fetch args share one cache entry (a cross-tenant / per-param cache collision). The lone exception is a globally-unique id used as the key while a second fetch arg is only an authz scope that cannot collide; annotate those with `// rq-lint-allow: <reason>`. Enforced by the `key-fetch-arg-drift` check in `scripts/check-react-query-patterns.ts`.
## Server-importable query primitives must NOT live in a `'use client'` module
Next.js rewrites **every** export of a `'use client'` module into a *client reference* in the server bundle. Server-evaluated code — RSC `page.tsx`/`layout.tsx`, `prefetch.ts`, route handlers, **block definitions**, triggers/workers — can only *render* such an export as a component or pass it as a prop; **calling** one throws at runtime (`Attempted to call X from the server but X is on the client` — for an object export it surfaces as `X.list is not a function`). `next build` does **not** catch this — only SSR/runtime does.
So any **query-key factory, standalone `requestJson` fetcher, mapper, or constant** that a server module imports must live in a **non-`'use client'`** module:
- key factories → `hooks/queries/utils/<entity>-keys.ts` (see `folder-keys.ts`, `table-keys.ts`, `credential-keys.ts`)
- standalone fetchers/mappers → `hooks/queries/utils/fetch-*.ts` / `*-list-query.ts` (see `fetch-workflow-envelope.ts`, `fetch-workspace-credentials.ts`)
The `'use client'` hook module then imports these back for its hooks. **Never** define a server-imported factory/fetcher directly in a `'use client'` hooks file — it crashes SSR (this caused the tables-page crash). Enforced for prefetch/route/trigger/block files by `scripts/check-client-boundary-imports.ts` (`bun run check:client-boundary`, run in CI). Escape hatch for a genuinely browser-only path: `// client-boundary-allow: <reason>` on the line above the import.
## File Structure
```typescript
// 1. Query keys factory
// 2. Types (if needed)
// 3. Private fetch functions (accept signal parameter)
// 4. Exported hooks
```
## Query Hook
- Every `queryFn` must destructure and forward `signal` for request cancellation
- Every query must have an explicit `staleTime`, assigned from a named exported constant (`ENTITY_LIST_STALE_TIME`), never an inline numeric literal. A server-side prefetch (`prefetch.ts`) hydrating the same query key must import and reuse that constant, not restate the number — this is what keeps a prefetched cache entry from going stale out of sync with the client hook that reads it
- Use `keepPreviousData` only on variable-key queries (where params change), never on static keys
- Same-origin JSON calls must go through `requestJson(contract, ...)` from `@/lib/api/client/request` against the contract in `@/lib/api/contracts/**`
```typescript
import { requestJson } from '@/lib/api/client/request'
import { listEntitiesContract, type EntityList } from '@/lib/api/contracts/entities'
export const ENTITY_LIST_STALE_TIME = 60 * 1000
async function fetchEntities(workspaceId: string, signal?: AbortSignal): Promise<EntityList> {
const data = await requestJson(listEntitiesContract, {
query: { workspaceId },
signal,
})
return data.entities
}
export function useEntityList(workspaceId?: string, options?: { enabled?: boolean }) {
return useQuery({
queryKey: entityKeys.list(workspaceId),
queryFn: ({ signal }) => fetchEntities(workspaceId as string, signal),
enabled: Boolean(workspaceId) && (options?.enabled ?? true),
staleTime: ENTITY_LIST_STALE_TIME,
placeholderData: keepPreviousData, // OK: workspaceId varies
})
}
```
## Mutation Hook
- Use targeted invalidation (`entityKeys.lists()`) not broad (`entityKeys.all`) when possible
- Invalidation must cover all affected query key prefixes (lists, details, related views)
- Use `onSuccess` invalidation for plain mutations; use `onSettled` for optimistic mutations so the cache is reconciled on both success and error (see Optimistic Updates below)
- `mutationFn` calls go through `requestJson(contract, { body, signal })` from `@/lib/api/client/request` — same boundary rule as queries
```typescript
export function useCreateEntity() {
const queryClient = useQueryClient()
return useMutation({
mutationFn: (body: CreateEntityBody) => requestJson(createEntityContract, { body }),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: entityKeys.lists() })
},
})
}
```
## Optimistic Updates
For optimistic mutations, use `onSettled` (not `onSuccess`) for cache reconciliation — `onSettled` fires on both success and error, ensuring the cache is always reconciled with the server.
```typescript
export function useUpdateEntity() {
const queryClient = useQueryClient()
return useMutation({
mutationFn: async (variables) => { /* ... */ },
onMutate: async (variables) => {
await queryClient.cancelQueries({ queryKey: entityKeys.detail(variables.id) })
const previous = queryClient.getQueryData(entityKeys.detail(variables.id))
queryClient.setQueryData(entityKeys.detail(variables.id), /* optimistic value */)
return { previous }
},
onError: (_err, variables, context) => {
queryClient.setQueryData(entityKeys.detail(variables.id), context?.previous)
},
onSettled: (_data, _error, variables) => {
queryClient.invalidateQueries({ queryKey: entityKeys.lists() })
queryClient.invalidateQueries({ queryKey: entityKeys.detail(variables.id) })
},
})
}
```
For optimistic mutations syncing with Zustand, use `createOptimisticMutationHandlers` from `@/hooks/queries/utils/optimistic-mutation`.
## useCallback Dependencies
Never include mutation objects (e.g., `createEntity`) in `useCallback` dependency arrays — the mutation object is not referentially stable and changes on every state update. The `.mutate()` and `.mutateAsync()` functions are stable in TanStack Query v5.
```typescript
// ✗ Bad — causes unnecessary recreations
const handler = useCallback(() => {
createEntity.mutate(data)
}, [createEntity]) // unstable reference
// ✓ Good — omit from deps, mutate is stable
const handler = useCallback(() => {
createEntity.mutate(data)
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [data])
```
## Boundary Types
- Hooks import named type aliases from `@/lib/api/contracts/**` (e.g., `import { listEntitiesContract, type EntityList } from '@/lib/api/contracts/entities'`). Never write `z.input<...>` / `z.output<...>` in hooks, and never `import { z } from 'zod'` in client code.
- Raw `fetch` is allowed only for documented exceptions — multipart uploads, binary downloads, streaming responses, signed-URL flows, OAuth redirects, external origins. Each such raw `fetch(` inside `apps/sim/hooks/queries/**` or `apps/sim/hooks/selectors/**` — and any same-origin `/api/...` fetch elsewhere under `apps/sim/**` outside an API route handler — must be preceded by a `// boundary-raw-fetch: <reason>` annotation (reason non-empty; up to three preceding comment lines tolerated). Enforced by `scripts/check-api-validation-contracts.ts` (`bun run check:api-validation` / `:strict`).
## Naming
- **Keys**: `entityKeys`
- **Query hooks**: `useEntity`, `useEntityList`
- **Mutation hooks**: `useCreateEntity`, `useUpdateEntity`, `useDeleteEntity`
- **Fetch functions**: `fetchEntity`, `fetchEntities` (private)
## Enforcement
`scripts/check-react-query-patterns.ts` (`bun run check:react-query`, run in CI) statically enforces these conventions: every `useQuery`/`useInfiniteQuery`/`useSuspenseQuery` declares an explicit `staleTime`, inline `queryFn`s destructure `signal`, `queryKey`s reference a colocated factory rather than an inline literal, every `*Keys` factory in `hooks/queries/**` exposes an `all` root key, and every identifier the `queryFn` forwards into the fetch also appears in the `queryKey` (`key-fetch-arg-drift`). `hooks/queries/**` is a zero-tolerance zone; the rest of `apps/sim/**` is ratcheted against `scripts/check-react-query-patterns.baseline.json`. For a genuine exception, put `// rq-lint-allow: <reason>` on the line directly above the flagged construct.
+95
View File
@@ -0,0 +1,95 @@
# React & Render Performance
Behavior-preserving performance idioms for components, hooks, and hot render paths. These are safe defaults — apply them freely. For the render-causing *effect/state* anti-patterns (derived state in effects, effect chains, state synced to a prop), use the dedicated skills: `/you-might-not-need-an-effect`, `/you-might-not-need-state`, `/you-might-not-need-a-memo`, `/you-might-not-need-a-callback`. Those refactors change render timing — verify them against the running UI, never mass-apply blind.
## Lazy-init refs that hold objects
`useRef(new Map())` / `useRef(new Set())` / `useRef({...})` allocates a fresh object on **every render** and throws it away — only the first is ever kept. Lazy-init instead so the allocation happens once.
```typescript
// ✗ Bad — allocates a new Map each render, discards all but the first
const cacheRef = useRef<Map<string, string>>(new Map())
// ✓ Good — allocated once, stable identity thereafter
const cacheRef = useRef<Map<string, string> | null>(null)
cacheRef.current ??= new Map()
```
Read `cacheRef.current` directly inside effects/handlers — refs are stable and never belong in a dependency array. A cheap primitive (`useRef(0)`, `useRef('')`, `useRef(null)`) needs no lazy init.
## Hoist static values and closure-free functions to module scope
A value or function declared inside a component is rebuilt every render. If it captures **nothing** from component scope (no props/state/refs), move it above the component at module scope. This skips the per-render allocation and keeps a stable identity so memoized children don't re-render.
```typescript
// ✗ Bad — rebuilt every render, new identity each time
function Toolbar({ mode }: ToolbarProps) {
const TITLES = { create: 'Add', edit: 'Configure' } as const
const handleWheel = (e: React.WheelEvent) => e.currentTarget.scrollBy(e.deltaX, e.deltaY)
// ...
}
// ✓ Good — allocated once at module load
const TITLES = { create: 'Add', edit: 'Configure' } as const
function handleWheel(e: React.WheelEvent) {
e.currentTarget.scrollBy(e.deltaX, e.deltaY)
}
function Toolbar({ mode }: ToolbarProps) { /* ... */ }
```
A closure-free function that IS wired through a ref sink or intentionally kept for stable identity may stay inline — hoisting a one-line `preventDefault` handler is churn, not a win. Hoist when it removes a real per-render allocation or unblocks child memoization.
## Pre-index with Map/Set for repeated lookups
`array.find()` / `array.includes()` / `array.indexOf()` scan the whole list each call. Inside a loop or a hot render path over a non-trivial list, that is O(n·m). Build a `Map` (for lookup-by-key) or `Set` (for membership) **once before** the loop, then look up in O(1).
```typescript
// ✗ Bad — find() re-scans outputs for every column
for (const child of columns) {
const output = group.outputs.find((o) => o.columnName === getColumnId(child))
}
// ✓ Good — index once, then O(1) lookups
const outputByName = new Map<string, Output>()
for (const o of group.outputs) {
if (!outputByName.has(o.columnName)) outputByName.set(o.columnName, o) // first wins, matches find()
}
for (const child of columns) {
const output = outputByName.get(getColumnId(child))
}
```
Preserve `.find()`'s **first-match** semantics when duplicate keys are possible: `new Map(arr.map(...))` keeps the *last* entry, so guard with `if (!map.has(key))` when replacing a `.find()`. Skip this for tiny, cold arrays (a handful of items in an event handler) where the Map build costs more than it saves.
## Never mutate a shared array in place
The real bug to avoid is `array.sort()` / `array.reverse()` on an array you don't own — sorting a React Query cache array in place corrupts shared state. Always sort a copy:
```typescript
// ✗ Bad — mutates the (possibly shared) source array in place
return items.sort(compare)
// ✓ Good — sorts a throwaway copy, source untouched
return [...items].sort(compare)
```
**Do NOT reach for `toSorted()` / `toReversed()` / `with()` / `toSpliced()` on client render paths.** They are ES2023 *runtime* methods — and a tsconfig `"lib": ["ES2023"]` only makes them **type-check**, it does not make them **run**. Next/SWC compiles syntax but does **not** polyfill prototype methods, and the default browserslist still includes browsers without them (`toSorted` landed in Safari 16 / iOS 16, so any device capped at iOS 15 throws `TypeError: x.toSorted is not a function` and crashes the page). The perf difference vs `[...arr].sort()` is negligible (both allocate one array), so the copy-then-sort form is the correct default everywhere client code runs. Only consider the immutable methods in Node-only code (server routes, scripts) on Node ≥20, where the runtime is known.
## Run independent awaits in parallel
Sequential `await`s that don't consume each other's result serialize latency for nothing — in an async Server Component or a route handler this directly delays the response. Kick them off together with `Promise.all` and destructure.
```typescript
// ✗ Bad — waits for params, then separately waits for searchParams
const { id } = await params
const { kbName } = await searchParams
// ✓ Good — one combined wait
const [{ id }, { kbName }] = await Promise.all([params, searchParams])
```
Only keep awaits sequential when a later call genuinely uses an earlier result, or when the ordering is deliberate (rate-limited batches, retry loops, write-then-read).
## Local feature barrels are the convention — do not "fix" them
Tooling (e.g. react-doctor's `no-barrel-import`) will flag imports from local `index.ts` barrels as a bundle cost. In this repo that is a **false positive**: barrel imports for 3+ export folders are mandated by `.claude/rules/sim-imports.md`. Leave them.
+89
View File
@@ -0,0 +1,89 @@
---
description: Isolated-vm sandbox worker security policy. Hard rules for anything that lives in the worker child process that runs user code.
paths:
- "apps/sim/lib/execution/isolated-vm-worker.cjs"
- "apps/sim/lib/execution/isolated-vm.ts"
- "apps/sim/lib/execution/sandbox/**"
- "apps/sim/sandbox-tasks/**"
---
# Sim Sandbox — Worker Security Policy
The isolated-vm worker child process at
`apps/sim/lib/execution/isolated-vm-worker.cjs` runs untrusted user code inside
V8 isolates. The process itself is a trust boundary. Everything in this rule is
about what must **never** live in that process.
## Hard rules
1. **No app credentials in the worker process**. The worker must not hold, load,
or receive via IPC: database URLs, Redis URLs, AWS keys, Stripe keys,
session-signing keys, encryption keys, OAuth client secrets, internal API
secrets, or any LLM / email / search provider API keys. If you catch yourself
`require`'ing `@/lib/auth`, `@sim/db`, `@/lib/uploads/core/storage-service`,
or anything that imports `env` directly inside the worker, stop and use a
host-side broker instead.
2. **Host-side brokers own all credentialed work**. The worker can only access
resources through `ivm.Reference` / `ivm.Callback` bridges back to the host
process. Today the only broker is `workspaceFileBroker`
(`apps/sim/lib/execution/sandbox/brokers/workspace-file.ts`); adding a new
one requires co-reviewing this file.
3. **Host-side brokers must scope every resource access to a single tenant**.
The `SandboxBrokerContext` always carries `workspaceId`. Any new broker that
accesses storage, DB, or an external API must use `ctx.workspaceId` to scope
the lookup — never accept a raw path, key, or URL from isolate code without
validation.
4. **Nothing that runs in the isolate is trusted, even if we wrote it**. The
task `bootstrap` and `finalize` strings in `apps/sim/sandbox-tasks/` execute
inside the isolate. They must treat `globalThis` as adversarial — no pulling
values from it that might have been mutated by user code. The hardening
script in `executeTask` undefines dangerous globals before user code runs.
## Why
A V8 JIT bug (Chrome ships these roughly monthly) gives an attacker a native
code primitive inside the process that owns whatever that process can reach.
If the worker only holds `isolated-vm` + a single narrow workspace-file broker,
a V8 escape leaks one tenant's files. If the worker holds a Stripe key or a DB
connection, a V8 escape leaks the service.
The original `doc-worker.cjs` vulnerability (CVE-class, 225 production secrets
leaked via `/proc/1/environ`) was the forcing function for this architecture.
Keep the blast radius small.
## Checklist for changes to `isolated-vm-worker.cjs`
Before landing any change that adds a new `require(...)` or `process.send(...)`
payload or `ivm.Reference` wrapper in the worker:
- [ ] Does it load a credential, key, connection string, or secret? If yes,
move it host-side and expose as a broker.
- [ ] Does it import from `@/lib/auth`, `@sim/db`, `@/lib/uploads/core/*`,
`@/lib/core/config/env`, or any module that reads `process.env` of the
main app? If yes, same — move host-side.
- [ ] Does it expose a resource that's workspace-scoped without taking a
`workspaceId`? If yes, re-scope.
- [ ] Did you update the broker limits (`IVM_MAX_BROKER_ARGS_JSON_CHARS`,
`IVM_MAX_BROKER_RESULT_JSON_CHARS`, `IVM_MAX_BROKERS_PER_EXECUTION`) if
the new broker can emit large payloads or fire frequently?
## What the worker *may* hold
- `isolated-vm` module
- Node built-ins: `node:fs` (only for reading the checked-in bundle `.cjs`
files) and `node:path`
- The three prebuilt library bundles under
`apps/sim/lib/execution/sandbox/bundles/*.cjs`
- IPC message handlers for `execute`, `cancel`, `fetchResponse`,
`brokerResponse`
The worker deliberately has **no host-side logger**. All errors and
diagnostics flow through IPC back to the host, which has `@sim/logger`. Do
not add `createLogger` or console-based logging to the worker — it would
require pulling the main app's config / env, which is exactly what this
rule is preventing.
Anything else is suspect.
+228
View File
@@ -0,0 +1,228 @@
---
paths:
- "apps/sim/app/workspace/*/settings/**"
- "apps/sim/ee/**/components/**"
---
# Settings Pages
The Next.js `settings/[section]/layout.tsx` owns all settings page chrome via
`SettingsHeaderShell` — a fixed header bar (a left back chip + right-aligned
action chips), a scroll region, and a centered `max-w-[48rem]` content column led
by a **title + description from navigation metadata**. The chrome stays mounted
across section navigation (it never re-renders or re-lays-out). Each section
renders through the **`SettingsPanel`** registrar
(`@/app/workspace/[workspaceId]/settings/components/settings-panel`), which feeds
the shell its header data and renders only the section body. Sections supply
**data**, never chrome.
Do NOT hand-roll any of these in a settings page — they are owned by the layout
shell (fed through `SettingsPanel`):
- `<div className='flex h-full flex-col bg-[var(--bg)]'>` shell
- the header bar (`flex flex-shrink-0 … px-[16px] pt-[8.5px] pb-[8.5px]`)
- the scroll container (`min-h-0 flex-1 overflow-y-auto px-6 [scrollbar-gutter:stable_both-edges]`)
- the content column (`mx-auto … max-w-[48rem] … gap-7`)
- a title block (`<h1 className='font-medium text-[var(--text-body)] text-lg'>` + `<p className='text-[var(--text-muted)] text-md'>`)
- the page-level search input
## Canonical page shape
```tsx
import { SettingsPanel } from '@/app/workspace/[workspaceId]/settings/components/settings-panel'
return (
<SettingsPanel
actions={[{ text: 'Create', icon: Plus, variant: 'primary', onSelect: onCreate }]}
search={{ value: searchTerm, onChange: setSearchTerm, placeholder: 'Search …' }}
>
{/* body only — sections, lists, forms */}
</SettingsPanel>
)
```
When the page has modal/dialog siblings, wrap them with the panel in a fragment:
```tsx
return (
<>
<SettingsPanel actions={}>{body}</SettingsPanel>
<SomeModal />
</>
)
```
## `SettingsPanel` props
- `actions?: SettingsAction[]` — right-aligned header chips, **data only**:
`{ text, icon?, variant?: 'primary'|'destructive', active?, onSelect, disabled?, tooltip? }`.
The shell renders each as a `Chip` — never pass JSX, a `<div>`, or `className`
(the locked contract: it's structurally impossible to vibe-code a padding
change). Multiple/conditional actions are a plain array
(`[...(canManage ? [{…}] : []), …]`). Labels are **sentence case** (`Add override`,
not `Add Override`). A disabled action that needs to explain itself sets
`tooltip` (the shell renders the hover tooltip, disabled chip included) — never
hand-roll a tooltip-wrapped chip in `aside`. Save/Discard pairs come from the
`saveDiscardActions()` helper (spread it into `actions`). Only a widget that
genuinely cannot be a chip (e.g. one needing hover-prefetch) goes in `aside`.
- `back?: SettingsBackAction` (`{ text, icon?, onSelect }`) — left-aligned back
chip for a **detail sub-view** (e.g. a selected MCP server, a permission group,
a retention policy). Detail sub-views render through `SettingsPanel` like list
pages — they do NOT hand-roll their own shell.
- `aside?: ReactNode` — escape hatch for the rare non-chip header widget. Keep it rare.
- `search?: { value; onChange: (value: string) => void; placeholder?; disabled? }`
renders the canonical search field directly below the title. Pass `setSearchTerm`
straight to `onChange`. Use this for a standalone search; if search shares a row
with other controls (sort, filters, a date picker), render that whole row in
`children` instead and omit the prop.
- `title?` / `description?` — overrides for the nav-driven defaults. **Only** for a
detail sub-view that needs a different heading; normal pages never pass these.
- `scrollContainerRef?: React.Ref<HTMLDivElement>` — forwards a ref to the scroll
region (e.g. programmatic scroll-to-bottom).
## Title + description live in navigation metadata
`apps/sim/app/workspace/[workspaceId]/settings/navigation.ts` is the single source
of truth. Every `NavigationItem` carries a one-line `description`; `SettingsPanel`
resolves both via `getSettingsSectionMeta(section)` and the
`SettingsSectionProvider` the settings shell wraps around the active section.
Adding a new settings page:
1. Add the `SettingsSection` id + a `NavigationItem` (with `label` **and**
`description`) in `navigation.ts`. Keep descriptions verb-first, one line,
~4055 chars, in the product voice (see `.claude/rules/constitution.md`).
2. Render the component inside the shell's `effectiveSection` switch in
`settings/[section]/settings.tsx`.
3. Build the component body inside `<SettingsPanel>` — no shell, no title block.
## Text-scale tokens (no literal pixel sizes)
Settings pages never use a literal `text-[Npx]` class — always the named Tailwind
scale token from `apps/sim/tailwind.config.ts`'s `fontSize` extension (`text-micro`
10px, `text-xs` 11px, `text-caption` 12px, `text-small` 13px, `text-sm` 14px
[Tailwind default, unmodified], `text-base` 15px, `text-md` 16px, `text-lg` 18px
[Tailwind default]). A literal size is either a straight rename to the equivalent
token (if the pixel value matches one exactly) or a sign the page never migrated —
grep `text-\[1[0-8]px\]` under `apps/sim/app/workspace/*/settings/**` and
`apps/sim/ee/**` to find stragglers.
For a two-line list row (title/value on top, a muted subtitle below — a name +
email, a tool name + description, a server name + status), the established
pairing is:
- **Title / row value**: `text-[var(--text-body)] text-sm`
- **Subtitle / muted description**: `text-[var(--text-muted)] text-caption`
This is not a stylistic guess — it is the tokenized form of the literal-pixel
pairing (`text-[14px] text-[var(--text-body)]` / `text-[12px]
text-[var(--text-muted)]`) already used for this exact row shape across
`member-list.tsx`, `api-keys.tsx`, `mcp.tsx`, `billing.tsx`,
`workflow-mcp-servers.tsx`, and others — keep new rows consistent with it rather
than inventing a new size pairing.
For a toggle row (a `Switch` with a title and optional description), use the emcn
`Label` component for the title — never a hand-rolled `<span>` — paired with
`Switch`'s `id`/`Label`'s `htmlFor`:
```tsx
<div className='flex items-center justify-between'>
<div className='flex flex-col gap-1'>
<Label htmlFor='my-toggle'>Enable thing</Label>
<p className='text-[var(--text-muted)] text-caption'>One-line description.</p>
</div>
<Switch id='my-toggle' checked={enabled} onCheckedChange={onToggle} />
</div>
```
`Label`'s own default styling (`font-medium text-[var(--text-primary)]
text-small`) already matches the established title treatment — do not add a
`className` overriding its size/color unless the row genuinely needs something
different.
`--text-primary`/`--text-secondary` and `--text-body`/`--text-muted` are both real,
independently-defined tokens (not interchangeable — they resolve to different
colors) and both see legitimate use across settings pages; this rule only pins
down the **row title/subtitle** shape above, not every text element on every page.
## Other shared settings primitives (do not re-roll these)
- **`SettingsEmptyState`** (`…/components/settings-empty-state`) — the canonical
muted status message. `variant='fill'` (default) centers in the available
height (empty list, or a not-entitled/loading gate); `variant='inline'` sits in
flow (a search "no results"). Never hand-roll
`<div className='flex h-full items-center justify-center text-[var(--text-muted)] text-sm'>`
or `<div className='py-4 text-center …'>`. It owns the `--text-muted` + `text-sm`
tokens, so it also keeps these messages consistent across pages.
- **`RowActionsMenu`** (`…/components/row-actions-menu`) — the trailing `...`
actions menu for a list row. Pass `label` (aria-label) and
`actions: RowAction[]` (`{ label, onSelect, destructive?, disabled? }`); the
component renders the canonical flush `...` trigger + `DropdownMenuContent`.
Conditional items become array spreads: `...(canManage ? [{…}] : [])`. Never
hand-roll the `<DropdownMenu>` + `<MoreHorizontal>` trigger per page.
## Save / Discard + unsaved-changes guard
Any settings surface with editable state uses **one** shared stack — never
hand-roll a Save button, a Discard button, a `beforeunload`, or an "Unsaved
changes" modal:
- **`saveDiscardActions(config)`** (`…/components/save-discard-actions/save-discard-actions`)
— returns the canonical dirty-gated **Discard + Save** `SettingsAction[]` (empty
when not dirty). Spread it into a `SettingsPanel` `actions` array, beside any
sibling actions (a detail view's Delete / Remove override). Config: `dirty`,
`saving`, `onSave`, `onDiscard`, `saveDisabled?`, `saveLabel?`, `savingLabel?`.
- **`useSettingsUnsavedGuard({ isDirty })`** (`…/settings/hooks/use-settings-unsaved-guard`)
— syncs the page's local `isDirty` into the shared `useSettingsDirtyStore` (so
the sidebar's **section-switch** confirm + the centralized `beforeunload` both
apply for free) and returns `{ showUnsavedModal, setShowUnsavedModal, guardBack,
confirmDiscard }` for a detail view's **in-view back** chip.
- **Top-level pages** (whitelabeling, sso): call it **unassigned**
`useSettingsUnsavedGuard({ isDirty: hasChanges })` — they only need the
store-sync; the sidebar/`beforeunload` do the rest.
- **Detail sub-views** (data-retention, access-control group-detail): route the
back chip through `onClick={() => guard.guardBack(closeFn)}` and render the
shared `<UnsavedChangesModal open={guard.showUnsavedModal}
onOpenChange={guard.setShowUnsavedModal} onDiscard={guard.confirmDiscard} />`
(from `@/app/workspace/[workspaceId]/components/credential-detail`). The
in-view header **Discard** chip (via `SaveDiscardActions onDiscard`) is a
*reset to original* — distinct from the back-confirm's discard, which leaves.
- **`useSettingsBeforeUnload`** is mounted **once** in the settings shell
(`settings/[section]/settings.tsx`) — never add a per-page `beforeunload`.
- **Dirty *computation* stays local** (shapes differ: field-compare vs
normalize+stringify) — only how dirty is *consumed* is shared. Derive it (a
`const`/`useMemo`), never store it in `useState`.
- **CRITICAL — rules of hooks:** call `useSettingsUnsavedGuard(...)`
**unconditionally, before every early-return gate** (entitlement / loading /
not-entitled `return <SettingsEmptyState>`). A hook placed after a gate is
skipped on gated renders and crashes.
- The route-based credential detail keeps its own `useUnsavedChangesGuard` (it
guards real `router.push` navigation + browser Back via a history sentinel);
it already shares `UnsavedChangesModal`, so copy stays unified.
## Detail sub-views
A drill-down view reached from a list row (selected MCP server, workflow MCP
server, permission group, retention policy) renders through
`SettingsPanel` like a list page: pass `back={{ text, icon: ArrowLeft, onSelect }}`
for the left back chip, `title` (the entity name), and the header `actions`, then
render the body. Do NOT hand-roll a shell or header bar; a tab bar renders as the
first body child. Gate/early-return states (not-entitled, loading, upgrade
prompts) stay as-is.
The route-based credential detail (`settings/secrets/[credentialId]`) is the lone
exception — it lives outside `[section]` and keeps its own `CredentialDetailLayout`.
## Audit checklist
A settings page is design-system-clean when:
- [ ] Its main return is a `<SettingsPanel>` (or `<>…<SettingsPanel>…</>` with modal siblings) — no hand-rolled shell/header/scroll/column.
- [ ] It renders **no** hand-rolled `<h1>`/description title block — the title comes from nav metadata.
- [ ] Header chips are in `actions`; a standalone search is in the `search` prop.
- [ ] Its `NavigationItem` has an accurate, consistent-length `description`.
- [ ] Detail sub-views and entitlement/loading gates keep their own chrome (intentional).
- [ ] If it has editable state: Save/Discard go through `SaveDiscardActions`, dirty is wired via `useSettingsUnsavedGuard` (called before any early-return gate), and there is **no** hand-rolled Save button / `beforeunload` / "Unsaved changes" modal.
- [ ] No business logic, handlers, or conditional rendering changed by the migration.
- [ ] No literal `text-[Npx]` classes — named scale tokens only (see "Text-scale tokens" above).
- [ ] `tsc`, `biome`, and the page's tests pass.
+71
View File
@@ -0,0 +1,71 @@
---
paths:
- "apps/sim/**/store.ts"
- "apps/sim/**/stores/**/*.ts"
---
# Zustand Store Patterns
Stores live in `stores/`. Complex stores split into `store.ts` + `types.ts`.
## Basic Store
```typescript
import { create } from 'zustand'
import { devtools } from 'zustand/middleware'
import type { FeatureState } from '@/stores/feature/types'
const initialState = { items: [] as Item[], activeId: null as string | null }
export const useFeatureStore = create<FeatureState>()(
devtools(
(set, get) => ({
...initialState,
setItems: (items) => set({ items }),
addItem: (item) => set((state) => ({ items: [...state.items, item] })),
reset: () => set(initialState),
}),
{ name: 'feature-store' }
)
)
```
## Persisted Store
```typescript
import { create } from 'zustand'
import { persist } from 'zustand/middleware'
export const useFeatureStore = create<FeatureState>()(
persist(
(set) => ({
width: 300,
setWidth: (width) => set({ width }),
_hasHydrated: false,
setHasHydrated: (v) => set({ _hasHydrated: v }),
}),
{
name: 'feature-state',
partialize: (state) => ({ width: state.width }),
onRehydrateStorage: () => (state) => state?.setHasHydrated(true),
}
)
)
```
## Rules
1. Use `devtools` middleware (named stores)
2. Use `persist` only when data should survive reload
3. `persist` MUST use `partialize` with an explicit whitelist of the durable fields. Exclude transient flags (`isResizing`, drag/hover state) and `_hasHydrated` from the whitelist, and never spread the whole state (`{ ...state }`) — it leaks actions and transient state into storage
4. `_hasHydrated` pattern for persisted stores needing hydration tracking
5. Immutable updates only
6. `set((state) => ...)` when depending on previous state
7. Provide `reset()` action
## Outside React
```typescript
const items = useFeatureStore.getState().items
useFeatureStore.setState({ items: newItems })
```
+78
View File
@@ -0,0 +1,78 @@
---
paths:
- "apps/sim/**/*.tsx"
- "apps/sim/**/*.css"
---
# Styling Rules
## Tailwind
1. **No inline styles** - Use Tailwind classes
2. **No duplicate dark classes** - Skip `dark:` when value matches light mode
3. **Exact values** - `text-[14px]`, `h-[26px]`
4. **Transitions** - `transition-colors` for interactive states
## Conditional Classes
```typescript
import { cn } from '@sim/emcn'
<div className={cn(
'base-classes',
isActive && 'active-classes',
disabled ? 'opacity-60' : 'hover:bg-accent'
)} />
```
## CSS Variables
For dynamic values (widths, heights) synced with stores:
```typescript
// In store
setWidth: (width) => {
set({ width })
document.documentElement.style.setProperty('--sidebar-width', `${width}px`)
}
// In component
<aside style={{ width: 'var(--sidebar-width)' }} />
```
## Text Scale
Custom font sizes (`apps/sim/tailwind.config.ts`): `text-micro`=10px, `text-xs`=11px, `text-caption`=12px, `text-small`=13px, `text-base`=15px. `text-sm` is Tailwind default 14px. Field titles use `text-small` (13px); hints/errors use `text-caption` (12px).
Icons default `size-[14px]`. Equal h/w → `size-*` (`size-[14px]`, `size-4`), never `h-N w-N`.
## Color Tokens
Value text `--text-body`; muted/placeholder/labels `--text-muted`; icons `--text-icon`; borders `--border-1` (fields) / `--border` (dividers); surfaces `--surface-5` (light) / `--surface-4` (dark); active row `--surface-active`; error `--text-error`. No focus rings on chip surfaces.
## Chip Components (consumer usage)
`ChipInput`, `ChipTextarea`, `ChipModal*` own their full chrome. Consumers describe intent through PROPS; they never re-style the chrome. The canonical chrome lives in `apps/sim/components/emcn/components/chip/chip-chrome.ts` (all tokens are re-exported from the `@sim/emcn` barrel — no subpath import needed) — never hand-roll `rounded-lg`/`border`/`bg-[var(--surface-5)]`/`h-[30px]`/`px-2`/`text-sm`/focus rings.
### Props over className
- **Errors** → `error` prop. Never `className={cn(err && 'border-[var(--text-error)]')}`.
- **Leading icon** → `icon` prop (rendered 14px in `--text-icon`).
- **Trailing buttons** (reveal/copy/fetch) → `endAdornment`.
- **Inner-input styling** (e.g. `font-mono`, number-spinner reset) → `inputClassName` (ChipInput only). See `app/workspace/[workspaceId]/settings/components/billing/components/usage-limit-field/usage-limit-field.tsx:117`.
- **`ChipModalField` controls take NO className.** Pass `title`/`value`/`onChange`/`error`/`hint`/`required`/`flush`. The field owns label, control, and error/hint rendering. See `app/workspace/[workspaceId]/skills/components/skill-modal/skill-modal.tsx:170`.
### What className MAY carry
Layout/sizing ONLY: `flex-1`, `w-full`, `w-[Npx]`, `min-w-0`, `max-w-*`, margins, `truncate`. Example: `<ChipInput icon={Search} className='min-w-0 flex-1' .../>` (`app/workspace/[workspaceId]/integrations/integrations.tsx:257`). NEVER re-specify canonical chrome — the component already applies it.
### Form / chip-modal layout rhythm
- **Field row** = `ChipModalField`: label↔control `gap-[9px]`, field gutter `px-2` (`px-0` when `flush`). Title = `Label` at `text-small` (13px), muted, normal weight; hint/error at `text-caption` (12px).
- **Modal body** (`ChipModalBody`): `gap-4` between fields, padding `px-2 pt-4 pb-4.5`.
- **Header/footer**: horizontal gutter `px-4` (header `pt-3`; footer `px-4 pt-2 pb-2`, tinted bar).
- **Every body field MUST be a `ChipModalField`** — NEVER hand-roll a field row (raw `<div>` + hand-rolled `<p>`/`<label>` title + bare `ChipInput`/`ChipTextarea`). WHY: body `px-2` + field `px-2` = effective `px-4`, exactly matching the `px-4` header/footer. A hand-rolled row skips the field gutter, sits at `px-2`, and is visibly misaligned (this bug shipped in the scheduled-tasks "Create new scheduled task" modal). Inline errors go through the `error` prop, not a hand-rolled `<p>`.
- **Uncovered controls** (`ChipCombobox`, `ChipSelect`, `DatePicker`, `TimePicker`, `ButtonGroup`, arbitrary JSX) → `ChipModalField type='custom'` with a `title`. It still applies the `px-2` gutter and renders the canonical `Label`, so it stays aligned. Never drop such a control into a raw `<div>`, and never add a body-level wrapper `<div>` with a custom `gap-*` that fights `gap-4`.
- **Page section rhythm** (integrations/skills/settings): muted `text-small` label + `mt-[9px] mb-3 h-px bg-[var(--border)]` divider, sections stacked `gap-7`. Reuse `SettingsSection` (`app/workspace/[workspaceId]/settings/components/settings-section/settings-section.tsx`) rather than re-deriving it.
When a standalone labeled field outside a `ChipModal` needs the same look (e.g. `SkillImport`), match the field rhythm by hand: `flex flex-col gap-[9px]`, muted label, `ChipInput`/`ChipTextarea` control, `text-caption` error below.
+21
View File
@@ -0,0 +1,21 @@
---
paths:
- "apps/sim/**/*.ts"
- "apps/sim/**/*.tsx"
---
# TypeScript Rules
1. **No `any`** - Use proper types or `unknown` with type guards
2. **Props interface** - Always define for components
3. **Const assertions** - `as const` for constant objects/arrays
4. **Ref types** - Explicit: `useRef<HTMLDivElement>(null)`
5. **Type imports** - `import type { X }` for type-only imports
```typescript
// ✗ Bad
const handleClick = (e: any) => {}
// ✓ Good
const handleClick = (e: React.MouseEvent<HTMLButtonElement>) => {}
```
+215
View File
@@ -0,0 +1,215 @@
---
paths:
- "apps/sim/app/**/*.tsx"
- "apps/sim/app/**/*.ts"
- "apps/sim/app/**/search-params.ts"
---
# URL / Query-Param State (nuqs)
URL query state is managed with [`nuqs`](https://nuqs.dev). The `NuqsAdapter` is wired once in `apps/sim/app/layout.tsx` — do not add another. This rule is the source of truth for *what* belongs in the URL and *how* to wire it.
## Decision framework — where does this state live?
Pick exactly one home for each piece of state:
- **React Query** → server/remote data. Unchanged; see `.claude/rules/sim-queries.md`.
- **URL params (nuqs)** → client view-state worth putting in a link: active tab/panel, selected entity id, filters, search query, pagination, view mode (list/grid), an open "view" drawer/modal that represents a destination.
- **Zustand** → cross-component client state that must NOT be in the URL: high-frequency, large, ephemeral, or socket-synced (canvas pan/zoom, cursor, drag state, resize widths, unsaved buffers, live collaborative selection).
- **`useState`** → purely local, single-component UI.
Put state in the URL **only** when it is *all* of: shareable, deep-linkable, bookmarkable, survives reload + back/forward — **and** is discrete, low-frequency, and small. If it fails any of those, it does not go in the URL.
### When to use what (decision table)
| Home | Trigger | Example |
| --- | --- | --- |
| **URL (nuqs)** | Client view-state worth a link: tab, filter, search, sort, pagination, selected-entity id, an open "view" modal/drawer that is a destination | `?tab=licenses`, `?category=Communication`, `?page=3`, `?skillId=abc` |
| **React Query** | Server/remote data fetched from an endpoint | `useMcpServers(workspaceId)`, `useSkills(workspaceId)` |
| **Zustand** | Cross-component client state that must NOT be in the URL: high-frequency, large, ephemeral, socket-synced | canvas pan/zoom, live cursor, drag state, resize widths, unsaved buffers |
| **`useState`** | Purely local single-component UI; also the snappy mirror of a debounced URL search | a hover flag, a transient dialog target, the live text of a debounced search box |
## Anti-patterns (forbidden)
- Direct `useSearchParams().get(...)` or `new URLSearchParams(window.location.search)` to **read** state.
- Hand-built query strings + `router.replace`/`router.push` to **mutate** state.
- `window.history.replaceState`/`pushState` to mutate a param.
- Duplicating URL state into a store and syncing it with effects / `popstate` listeners.
- High-frequency or large state in the URL (cursor, pan/zoom, un-debounced keystrokes, big JSON blobs).
- `import { z } from 'zod'` in client code for param validation — use nuqs parsers (`parseAsString`, `parseAsInteger`, `parseAsBoolean`, `parseAsStringLiteral`, `parseAsArrayOf`) or a custom `createParser`.
These reads/mutations are **not** anti-patterns and stay as-is:
- **Outbound URL builders** — `new URLSearchParams({...})` to construct a `href`, a download endpoint, an external WebSocket/API URL, or a `window.open(_, '_blank')` destination.
- **Route navigations** — `router.push('/path/[id]?folderId=x')` that changes the route *path*, not just the current query. A nuqs setter only mutates the query on the current path; cross-path navigation stays on `router`.
- **Read-once auth / redirect signals** — `token`, `callbackUrl`, `redirect`, `error`, `invite_flow`, `upgraded`, `redirect_workflow`, etc. These are navigation signals consumed once (often read-then-strip), not synced view-state. Leave them on `useSearchParams`.
## Per-feature `search-params.ts` — single source of truth
Co-locate a `search-params.ts` next to the feature. Export the parser map (and shared options). Both the client (`useQueryStates`/`useQueryState`) and any server component (`createSearchParamsCache` from `nuqs/server`) import from this one file. Import parsers from `nuqs/server` so the module is safe to import in both client and server contexts.
Conventions:
- `.withDefault(...)` on every parser so reads are non-null.
- Filter / search / toggle / pagination options: `{ history: 'replace', shallow: true, clearOnDefault: true }` — clean URLs, no back-stack churn.
- Navigations that belong in browser history (changing folder, opening a deep-linked entity): `{ history: 'push' }`.
- `shallow: false` **only** when a Server Component / loader must re-read the param.
- Short, stable, **kebab-case** URL keys. Renaming a key is a breaking change to shared links — treat it as one.
- For an opaque/literal value use `parseAsStringLiteral([...] as const)`; for a custom wire format use [`createParser`](https://nuqs.dev/docs/parsers).
- A `createParser` for a value **not** comparable with `===` (arrays, objects, `Date`) **must** define an `eq``clearOnDefault` uses it to detect the default, so without it an empty-array/object default never strips from the URL. Built-in `parseAsArrayOf(...)` already ships its own `eq`; only string/number/boolean custom parsers can omit it. Example (array): `eq: (a, b) => a.length === b.length && a.every((v, i) => v === b[i])`.
### Example — grouped filters (single source of truth)
```typescript
// apps/sim/app/workspace/[workspaceId]/things/search-params.ts
import { parseAsArrayOf, parseAsString, parseAsStringLiteral } from 'nuqs/server'
const VIEW_MODES = ['list', 'grid'] as const
export const thingsParsers = {
search: parseAsString.withDefault(''),
tags: parseAsArrayOf(parseAsString).withDefault([]),
view: parseAsStringLiteral(VIEW_MODES).withDefault('list'),
} as const
/** Clean URLs, no back-stack churn for filter changes. */
export const thingsUrlKeys = {
history: 'replace',
shallow: true,
clearOnDefault: true,
} as const
```
### Client — `useQueryStates` (grouped) / `useQueryState` (single)
```typescript
'use client'
import { useQueryStates } from 'nuqs'
import { thingsParsers, thingsUrlKeys } from '@/app/workspace/[workspaceId]/things/search-params'
export function useThingFilters() {
const [filters, setFilters] = useQueryStates(thingsParsers, thingsUrlKeys)
// filters.search / filters.tags / filters.view are non-null (defaults applied)
// setFilters({ view: 'grid' }) — pass null to clear a single key back to default
return { filters, setFilters }
}
```
For a single param, use `useQueryState(key, parser)`:
```typescript
const [serverId, setServerId] = useQueryState(mcpServerIdParam.key, mcpServerIdParam.parser)
```
### Server — `createSearchParamsCache`
When a Server Component or loader must read a param, build a cache from the **same** parser map:
```typescript
// in a server component / page.tsx
import { createSearchParamsCache } from 'nuqs/server'
import { thingsParsers } from '@/app/workspace/[workspaceId]/things/search-params'
const thingsCache = createSearchParamsCache(thingsParsers)
export default async function Page({ searchParams }: { searchParams: Promise<Record<string, string | string[] | undefined>> }) {
const { search, view } = await thingsCache.parse(await searchParams)
// ...
}
```
If a client param must be re-read server-side after a change, set `shallow: false` on the write.
## Suspense boundary
`useQueryState`/`useQueryStates` read `useSearchParams` internally, so any client component using them must sit under a `<Suspense>` boundary (Next.js requirement). Wrap the page entry with a real-chrome fallback so a suspend never flashes a blank frame — see `apps/sim/app/workspace/[workspaceId]/files/page.tsx`.
## Debounced text inputs
Use nuqs's built-in [`limitUrlUpdates: debounce(ms)`](https://nuqs.dev/docs/options) — never hand-roll a local `useState` mirror + `useDebounce` + a URL write-back effect + a ref-guarded URL→local reconcile effect. The hook's returned value updates instantly (so the input is controlled directly by the nuqs value and stays snappy); only the *URL write* is debounced. Back/forward and deep links flow back natively because the input reads the nuqs value — no reconcile effect needed.
- **Standalone single search param** (`useQueryState`): put `limitUrlUpdates: debounce(300)` in the param's options.
- **Search inside a grouped `useQueryStates`**: keep the group's immediate writes for the discrete filters; pass the option **per call** only on the search setter, never on the whole group:
```typescript
import { debounce } from 'nuqs'
const setSearch = useCallback(
(value: string) => {
const next = value.length > 0 ? value : null
// Immediate update when clearing so the param drops out without lingering.
setFilters({ search: next }, next === null ? undefined : { limitUrlUpdates: debounce(300) })
},
[setFilters]
)
```
- **Keep fetches/filtering debounced.** Where the search value feeds a React Query key or an expensive in-memory filter, derive a debounced value off the instant nuqs value (`const debounced = useDebounce(urlSearch, 300)`) and feed *that* to the query — the instant value is only for the input box. Cheap in-memory filtering over a small static list may read the instant value directly.
- Preserve `.trim()` handling, `clearOnDefault` (empty clears the param), the existing default, and `history: 'replace'`. Import `debounce` from `nuqs` (client) — not `nuqs/server`. See logs (`use-log-filters.ts` grouped, query stays debounced), integrations/recently-deleted (cheap in-memory filter, instant value), and tables (filter stays debounced).
## Sort convention (`sort` + `dir`)
Sortable lists use **two scalar params**, never a serialized `{column,direction}` object:
```typescript
const SORT_COLUMNS = ['name', 'created', 'updated'] as const
const SORT_DIRECTIONS = ['asc', 'desc'] as const
export const thingsParsers = {
sort: parseAsStringLiteral(SORT_COLUMNS).withDefault('updated'),
dir: parseAsStringLiteral(SORT_DIRECTIONS).withDefault('desc'),
} as const
```
Both carry the shared filter options (`{ history: 'replace', clearOnDefault: true }`). The defaults must match the list's existing default sort exactly. If a UI exposes "no active sort" as `null`, derive that in the component (`sort === DEFAULT && dir === DEFAULT ? null : { column, direction }`) — the URL still holds the resolved values. "Clear sort" writes the defaults back (which `clearOnDefault` strips from the URL); never write `null`/garbage columns.
## Dates in the URL (date-only params)
A date-only param (a calendar anchor, a date filter) is stored as `yyyy-MM-dd` — never serialize a full `Date`/timestamp when only the day matters.
**Local vs UTC — pick the parser that matches your date math.** nuqs's built-in `parseAsIsoDate` is **UTC-based** (`serialize` via `toISOString()`, `parse` to UTC midnight). If your `Date` is local-time (e.g. produced by local-time helpers and read by `date-fns` `startOfWeek`/`isSameDay`, which are all local), `parseAsIsoDate` will shift the day by ±1 in any non-UTC timezone on reload/deep-link/back-forward. For local-time date math, use a small local-date `createParser` that serializes/parses on local calendar fields (`getFullYear`/`getMonth`/`getDate` ↔ `new Date(y, m-1, d)`) with an `eq` comparing y/m/d. Only use `parseAsIsoDate` when the value is genuinely UTC/midnight-UTC. See `scheduled-tasks/search-params.ts` (`parseAsLocalDate`).
When the default is **dynamic** (e.g. "today"), make the param **nullable** (omit `.withDefault`) and derive the fallback in the hook (`const anchor = param ?? today`), so a clean URL means the dynamic default and navigating back to it writes `null` (clears the param). See `scheduled-tasks/hooks/use-calendar.ts`.
## Selected-entity deep-link (store the id, derive the object)
To deep-link a row/modal/drawer to one entity, store **only its id** and look the object up in the already-loaded list — never serialize the object into the URL:
```typescript
const [skillId, setSkillId] = useQueryState(skillIdParam.key, {
...skillIdParam.parser,
history: 'push', // opening an entity is a destination; "back" closes it
clearOnDefault: true,
})
// Derive — do not duplicate into useState or sync with an effect:
const editingSkill = skillId ? (skills.find((s) => s.id === skillId) ?? null) : null
```
Open the panel/modal when the id resolves to a loaded entity; closing it calls `setSkillId(null)`. Because this reads `useSearchParams` it needs a **Suspense** boundary on the page (see below). A separate "create new" flow has no id and stays in local `useState`.
## Read-then-strip deep links
For an ephemeral deep-link that pre-opens a modal/drawer and should not linger in the URL (e.g. integrations `?connect=oauth`, knowledge `?addConnector=`), read the param, act on it once behind a `useRef` guard, then clear it: `setParam(null, { history: 'replace', scroll: false })`. See `apps/sim/app/workspace/[workspaceId]/integrations/[block]/integration-block-detail.tsx`.
## Workflow editor carve-out — what must NOT go in the URL
The workflow editor (`apps/sim/app/workspace/[workspaceId]/w/**`) is realtime/socket-synced via `socket-provider.tsx`. Its view-state is intentionally store-backed (Zustand), not URL-backed. Do **not** move the following into the URL:
- **Live cursor** and **broadcast live selection** (presence; emitted over the socket, throttled).
- **Pan / zoom / viewport** (ReactFlow-owned, continuous, not persisted).
- **Drag state** and **resize widths/heights** (panel/terminal/sidebar — high-frequency, persisted as local preferences).
- **Ephemeral diff staging** (`hasActiveDiff`, `baselineWorkflow`, `diffAnalysis`).
Borderline candidates that *look* shareable but currently stay in Zustand because moving them fights existing machinery:
- **Panel `activeTab`** and **`canvasMode`** — persisted local *preferences* wired into an SSR flash-prevention path (`data-panel-active-tab` + `_hasHydrated`). They are layout prefs, not destinations; moving them would unwind the SSR machinery and risk tab-flash on load.
- **`focusedBlockId`** ("look at this block") — the only genuinely shareable candidate, but it is entangled with the persisted editor store and panel-open orchestration. Adding it is a *new feature*, not a migration; ship it deliberately (with runtime verification against a live socket), not as part of a sweep.
Rule of thumb for the editor: if state is socket-coupled, high-frequency, viewport-related, or a persisted resize/preference, it stays in Zustand. When in doubt, leave it and flag it — do not force fragile URL state into the canvas.
## Docs
- Adapters (App Router `NuqsAdapter`): https://nuqs.dev/docs/adapters
- Parsers & options (`parseAsString`/`parseAsInteger`/`parseAsBoolean`/`parseAsStringLiteral`/`parseAsArrayOf`/`createParser`, `withDefault`, `history`, `shallow`, `clearOnDefault`): https://nuqs.dev/docs/parsers and https://nuqs.dev/docs/options
- Server-side reads (`createSearchParamsCache`): https://nuqs.dev/docs/server-side
+79
View File
@@ -0,0 +1,79 @@
---
name: add-settings-page
description: Add a new Sim settings page, or audit existing settings pages for design-system compliance with the shared SettingsPanel layout. Use when creating a settings tab, or when asked to check/clean up settings pages so they match the design system (consistent title, header, search, spacing).
---
# Settings Page (add / audit)
Sim settings pages all render through the shared **`SettingsPanel`** primitive,
which owns the page chrome and renders a nav-driven title + description. The full
convention lives in `.claude/rules/sim-settings-pages.md` — read it first; this
skill is the procedure.
Key paths:
- Layout primitive: `apps/sim/app/workspace/[workspaceId]/settings/components/settings-panel/settings-panel.tsx`
- Nav metadata (titles + descriptions): `apps/sim/app/workspace/[workspaceId]/settings/navigation.ts`
- Section switch + provider: `apps/sim/app/workspace/[workspaceId]/settings/[section]/settings.tsx`
- Pages: `apps/sim/app/workspace/[workspaceId]/settings/components/<name>/<name>.tsx` and EE pages under `apps/sim/ee/<feature>/components/`
## Mode A — Add a new settings page
1. **Navigation.** In `navigation.ts`: add the id to the `SettingsSection` union,
then a `NavigationItem` with `label` AND a one-line `description` (verb-first,
~4055 chars, product voice per `.claude/rules/constitution.md`). Place it in
the right `section` group and set any gating flags (`requiresHosted`,
`requiresEnterprise`, etc.).
2. **Wire the switch.** Add the component to the `effectiveSection` render switch
in `settings/[section]/settings.tsx` (lazy `dynamic(...)` like its siblings).
3. **Build the body inside `SettingsPanel`.** Never hand-roll the shell, header
bar, scroll region, content column, or title block. Put header buttons in
`actions`, a standalone search in `search={{ value, onChange, placeholder }}`,
and the page content as `children`. Modals go beside the panel inside a `<>`.
4. **If the page has editable state**, wire the shared save/discard stack — put
`SaveDiscardActions` (dirty-gated Discard+Save chips) in `actions`, and call
`useSettingsUnsavedGuard({ isDirty })` **before any early-return gate**.
Detail sub-views additionally route the back chip through
`guard.guardBack(closeFn)` and render the shared `UnsavedChangesModal`. Never
hand-roll a Save button, a `beforeunload`, or an "Unsaved changes" modal —
they're centralized. See the "Save / Discard + unsaved-changes guard" section
in `.claude/rules/sim-settings-pages.md`.
5. **Verify:** `cd apps/sim && bunx tsc --noEmit`; `bunx biome check --write <file>`.
## Mode B — Audit existing settings pages
For each page component, confirm the checklist in `.claude/rules/sim-settings-pages.md`:
1. Find hand-rolled shells that should be `SettingsPanel`:
`git grep -n "flex h-full flex-col bg-\[var(--bg)\]" -- 'apps/sim/**/settings/' 'apps/sim/ee/'`
— every match should be either `settings-panel.tsx`, a **detail sub-view**
(has a `<Chip leftIcon={ArrowLeft}>` back button), or an entitlement/loading
**gate** early-return. Anything else is a page that still needs migrating.
2. Find hand-rolled title blocks (should be 0 outside detail views):
`git grep -n "text-\[var(--text-body)\] text-lg" -- 'apps/sim/**/settings/' 'apps/sim/ee/'`
3. Find literal pixel text sizes (should be 0 — see "Text-scale tokens" in
`.claude/rules/sim-settings-pages.md` for the token map and the row
title/subtitle pairing convention):
`git grep -n "text-\[1[0-8]px\]" -- 'apps/sim/**/settings/' 'apps/sim/ee/'`
4. Confirm each page imports `SettingsPanel` and that its `NavigationItem` has an
accurate `description` of consistent length with its peers.
- Editable pages: confirm Save/Discard go through `SaveDiscardActions` and
dirty is wired via `useSettingsUnsavedGuard` (called before early-return
gates) — flag any hand-rolled Save button, `beforeunload`, or unsaved modal.
`git grep -n "beforeunload" -- 'apps/sim/**/settings/' 'apps/sim/ee/'`
should only hit the centralized `use-settings-before-unload.ts`.
5. When migrating a page, change ONLY the structural shell→`SettingsPanel` swap:
move header chips to `actions`, the standalone search to `search`, delete the
`<h1>` title block, replace the three closing `</div>` (column/scroll/shell)
with `</SettingsPanel>`, and keep modal siblings in a `<>` fragment. Do NOT
touch handlers, state, queries, conditional rendering, or detail/gate returns.
Drop per-page `gap-*`/`pt-*` on the content column in favor of the panel default.
6. When fixing literal pixel text sizes, replace ONLY the size class with its
exact-pixel-equivalent named token (e.g. `text-[12px]``text-caption`,
never a different size) — this must render pixel-identical, not restyle the
page. Leave color tokens (`--text-primary` vs `--text-body`, etc.) untouched
unless they're also being changed for an unrelated, deliberate reason.
7. Remove now-unused imports (`ChipInput`/`Search`) ONLY after grepping that
they are not still used elsewhere in the file (e.g. by a detail view).
8. **Verify the whole sweep:** `tsc --noEmit`, `biome check` on every touched
file, and run the affected pages' tests. Diff each file against the base and
confirm the change is purely structural before shipping.
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/design-taste-frontend
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/emil-design-eng
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/make-interfaces-feel-better