simstudioai--sim/.claude/commands/validate-integration.md

---
description: Validate an existing Sim integration (tools, block, registry) against the service's API docs
argument-hint: <service-name> [api-docs-url]
---

# Validate Integration Skill

You are an expert auditor for Sim integrations. Your job is to thoroughly validate that an existing integration is correct, complete, and follows all conventions.

## Your Task

When the user asks you to validate an integration:
1. Read the service's API documentation (via WebFetch or Context7)
2. Read every tool, the block, and registry entries
3. Cross-reference everything against the API docs and Sim conventions
4. Report all issues found, grouped by severity (critical, warning, suggestion)
5. Fix all issues after reporting them

## Step 1: Gather All Files

Read **every** file for the integration — do not skip any:

```
apps/sim/tools/{service}/          # All tool files, types.ts, index.ts
apps/sim/blocks/blocks/{service}.ts # Block definition
apps/sim/tools/registry.ts          # Tool registry entries for this service
apps/sim/blocks/registry-maps.ts    # Block + meta registry entry (BLOCK_REGISTRY / BLOCK_META_REGISTRY)
apps/sim/components/icons.tsx        # Icon definition
apps/sim/lib/auth/auth.ts           # OAuth config — should use getCanonicalScopesForProvider()
apps/sim/lib/oauth/oauth.ts         # OAuth provider config — single source of truth for scopes
apps/sim/lib/oauth/utils.ts               # Scope utilities, SCOPE_DESCRIPTIONS for modal UI
```

## Step 2: Pull API Documentation

Fetch the official API docs for the service. This is the **source of truth** for:
- Endpoint URLs, HTTP methods, and auth headers
- Required vs optional parameters
- Parameter types and allowed values
- Response shapes and field names
- Pagination patterns (which param name, which response field)
- Rate limits and error formats

## Step 3: Validate Tools

For **every** tool file, check:

### Tool ID and Naming
- [ ] Tool ID uses `snake_case`: `{service}_{action}` (e.g., `x_create_tweet`, `slack_send_message`)
- [ ] Tool `name` is human-readable (e.g., `'X Create Tweet'`)
- [ ] Tool `description` is a concise one-liner describing what it does
- [ ] Tool `version` is set (`'1.0.0'` or `'2.0.0'` for V2)

### Params
- [ ] All required API params are marked `required: true`
- [ ] All optional API params are marked `required: false`
- [ ] Every param has explicit `required: true` or `required: false` — never omitted
- [ ] Param types match the API (`'string'`, `'number'`, `'boolean'`, `'json'`)
- [ ] Visibility is correct:
  - `'hidden'` — ONLY for OAuth access tokens and system-injected params
  - `'user-only'` — for API keys, credentials, and account-specific IDs the user must provide
  - `'user-or-llm'` — for everything else (search queries, content, filters, IDs that could come from other blocks)
- [ ] Every param has a `description` that explains what it does

### Request
- [ ] URL matches the API endpoint exactly (correct base URL, path segments, path params)
- [ ] HTTP method matches the API spec (GET, POST, PUT, PATCH, DELETE)
- [ ] Headers include correct auth pattern:
  - OAuth: `Authorization: Bearer ${params.accessToken}`
  - API Key: correct header name and format per the service's docs
- [ ] `Content-Type` header is set for POST/PUT/PATCH requests
- [ ] Body sends all required fields and only includes optional fields when provided
- [ ] For GET requests with query params: URL is constructed correctly with query string
- [ ] ID fields in URL paths are `.trim()`-ed to prevent copy-paste whitespace errors
- [ ] Path params use template literals correctly: `` `https://api.service.com/v1/${params.id.trim()}` ``

### Response / transformResponse
- [ ] Correctly parses the API response (`await response.json()`)
- [ ] Extracts the right fields from the response structure (e.g., `data.data` vs `data` vs `data.results`)
- [ ] All nullable fields use `?? null`
- [ ] All optional arrays use `?? []`
- [ ] Error cases are handled: checks for missing/empty data and returns meaningful error
- [ ] Does NOT do raw JSON dumps — extracts meaningful, individual fields

### Outputs
- [ ] All output fields match what the API actually returns
- [ ] No fields are missing that the API provides and users would commonly need
- [ ] No phantom fields defined that the API doesn't return
- [ ] `optional: true` is set on fields that may not exist in all responses
- [ ] When using `type: 'json'` and the shape is known, `properties` defines the inner fields (tool outputs only — block outputs do not support `properties`)
- [ ] When using `type: 'array'`, `items` defines the item structure with `properties` (tool outputs only)
- [ ] Field descriptions are accurate and helpful

### Types (types.ts)
- [ ] Has param interfaces for every tool (e.g., `XCreateTweetParams`)
- [ ] Has response interfaces for every tool (extending `ToolResponse`)
- [ ] Optional params use `?` in the interface (e.g., `replyTo?: string`)
- [ ] Field names in types match actual API field names
- [ ] Shared response types are properly reused (e.g., `XTweetResponse` shared across tweet tools)

### Barrel Export (index.ts)
- [ ] Every tool is exported
- [ ] All types are re-exported (`export * from './types'`)
- [ ] No orphaned exports (tools that don't exist)

### Tool Registry (tools/registry.ts)
- [ ] Every tool is imported and registered
- [ ] Registry keys use snake_case and match tool IDs exactly
- [ ] Entries are in alphabetical order within the file

## Step 4: Validate Block

### Block ↔ Tool Alignment (CRITICAL)

This is the most important validation — the block must be perfectly aligned with every tool it references.

For **each tool** in `tools.access`:
- [ ] The operation dropdown has an option whose ID matches the tool ID (or the `tools.config.tool` function correctly maps to it)
- [ ] Every **required** tool param (except `accessToken`) has a corresponding subBlock input that is:
  - Shown when that operation is selected (correct `condition`)
  - Marked as `required: true` (or conditionally required)
- [ ] Every **optional** tool param has a corresponding subBlock input (or is intentionally omitted if truly never needed)
- [ ] SubBlock `id` values are unique across the entire block — no duplicates even across different conditions
- [ ] The `tools.config.tool` function returns the correct tool ID for every possible operation value
- [ ] The `tools.config.params` function correctly maps subBlock IDs to tool param names when they differ

### SubBlocks
- [ ] Operation dropdown lists ALL tool operations available in `tools.access`
- [ ] Dropdown option labels are human-readable and descriptive
- [ ] Conditions use correct syntax:
  - Single value: `{ field: 'operation', value: 'x_create_tweet' }`
  - Multiple values (OR): `{ field: 'operation', value: ['x_create_tweet', 'x_delete_tweet'] }`
  - Negation: `{ field: 'operation', value: 'delete', not: true }`
  - Compound: `{ field: 'op', value: 'send', and: { field: 'type', value: 'dm' } }`
- [ ] Condition arrays include ALL operations that use that field — none missing
- [ ] `dependsOn` is set for fields that need other values (selectors depending on credential, cascading dropdowns)
- [ ] SubBlock types match tool param types:
  - Enum/fixed options → `dropdown`
  - Free text → `short-input`
  - Long text/content → `long-input`
  - True/false → `dropdown` with Yes/No options (not `switch` unless purely UI toggle)
  - Credentials → `oauth-input` with correct `serviceId`
- [ ] Dropdown `value: () => 'default'` is set for dropdowns with a sensible default

### Advanced Mode
- [ ] Optional, rarely-used fields are set to `mode: 'advanced'`:
  - Pagination tokens / next tokens
  - Time range filters (start/end time)
  - Sort order / direction options
  - Max results / per page limits
  - Reply settings / threading options
  - Rarely used IDs (reply-to, quote-tweet, etc.)
  - Exclude filters
- [ ] **Required** fields are NEVER set to `mode: 'advanced'`
- [ ] Fields that users fill in most of the time are NOT set to `mode: 'advanced'`

### WandConfig
- [ ] Timestamp fields have `wandConfig` with `generationType: 'timestamp'`
- [ ] Comma-separated list fields have `wandConfig` with a descriptive prompt
- [ ] Complex filter/query fields have `wandConfig` with format examples in the prompt
- [ ] All `wandConfig` prompts end with "Return ONLY the [format] - no explanations, no extra text."
- [ ] `wandConfig.placeholder` describes what to type in natural language

### Tools Config
- [ ] `tools.access` lists **every** tool ID the block can use — none missing
- [ ] `tools.config.tool` returns the correct tool ID for each operation
- [ ] Type coercions are in `tools.config.params` (runs at execution time), NOT in `tools.config.tool` (runs at serialization time before variable resolution)
- [ ] `tools.config.params` handles:
  - `Number()` conversion for numeric params that come as strings from inputs
  - `Boolean` / string-to-boolean conversion for toggle params
  - Empty string → `undefined` conversion for optional dropdown values
  - Any subBlock ID → tool param name remapping
- [ ] No `Number()`, `JSON.parse()`, or other coercions in `tools.config.tool` — these would destroy dynamic references like `<Block.output>`

### Block Outputs
- [ ] Outputs cover the key fields returned by ALL tools (not just one operation)
- [ ] Output types are correct (`'string'`, `'number'`, `'boolean'`, `'json'`)
- [ ] `type: 'json'` outputs describe inner fields in the description string: `'User profile (id, name, username, bio)'` or `'[{address, status, type}]'` for arrays
- [ ] **Do NOT add a `properties: {...}` field on block outputs.** Block-level `OutputFieldDefinition` (from `@sim/workflow-types/blocks`) only accepts `{ type, description?, condition?, hiddenFromDisplay? }`. Nested `properties` is a tool-level construct (`OutputProperty`) — adding it to a block output will fail TypeScript at build time
- [ ] No opaque `type: 'json'` with vague descriptions like `'Response data'`
- [ ] Outputs that only appear for certain operations use `condition` if supported, or document which operations return them

### Block Metadata
- [ ] `type` is snake_case (e.g., `'x'`, `'cloudflare'`)
- [ ] `name` is human-readable (e.g., `'X'`, `'Cloudflare'`)
- [ ] `description` is a concise one-liner
- [ ] `longDescription` provides detail for docs
- [ ] `docsLink` points to `'https://docs.sim.ai/integrations/{service}'`
- [ ] `category` is `'tools'`
- [ ] `bgColor` uses the service's brand color hex
- [ ] `icon` references the correct icon component from `@/components/icons`
- [ ] `authMode` is set correctly (`AuthMode.OAuth` or `AuthMode.ApiKey`)
- [ ] Block + meta are registered in `blocks/registry-maps.ts` (`BLOCK_REGISTRY` / `BLOCK_META_REGISTRY`) alphabetically

### BlockMeta
- [ ] `{Service}BlockMeta` is exported in the same file as the block
- [ ] Has at least 7 templates, each with `icon`, `title`, `prompt`, `modules`, `category`, and `tags`
- [ ] Prompts describe concrete use cases, not generic descriptions of what the service does
- [ ] `alsoIntegrations` is set on any template whose prompt references another service
- [ ] `skills` present (3–5 mainstream, 2–3 niche), each grounded in `tools.access` — flag any skill implying an unsupported action
- [ ] **Each skill is real, not hallucinated** — web-search and confirm it maps to a popular use case attested online (vendor use-case pages, official docs describing the workflow, reputable "top automations" articles); rewrite/remove any you cannot source

### Block Inputs
- [ ] `inputs` section lists all subBlock params that the block accepts
- [ ] Input types match the subBlock types
- [ ] When using `canonicalParamId`, inputs list the canonical ID (not the raw subBlock IDs)

## Step 5: Validate OAuth Scopes (if OAuth service)

Scopes are centralized — the single source of truth is `OAUTH_PROVIDERS` in `lib/oauth/oauth.ts`.

- [ ] Scopes defined in `lib/oauth/oauth.ts` under `OAUTH_PROVIDERS[provider].services[service].scopes`
- [ ] `auth.ts` uses `getCanonicalScopesForProvider(providerId)` — NOT a hardcoded array
- [ ] Block `requiredScopes` uses `getScopesForService(serviceId)` — NOT a hardcoded array
- [ ] No hardcoded scope arrays in `auth.ts` or block files (should all use utility functions)
- [ ] Each scope has a human-readable description in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`
- [ ] No excess scopes that aren't needed by any tool

## Step 6: Validate Pagination Consistency

If any tools support pagination:
- [ ] Pagination param names match the API docs (e.g., `pagination_token` vs `next_token` vs `cursor`)
- [ ] Different API endpoints that use different pagination param names have separate subBlocks in the block
- [ ] Pagination response fields (`nextToken`, `cursor`, etc.) are included in tool outputs
- [ ] Pagination subBlocks are set to `mode: 'advanced'`

## Step 7: Validate Error Handling

- [ ] `transformResponse` checks for error conditions before accessing data
- [ ] Error responses include meaningful messages (not just generic "failed")
- [ ] HTTP error status codes are handled (check `response.ok` or status codes)

## Step 8: Report and Fix

### Report Format

Group findings by severity:

**Critical** (will cause runtime errors or incorrect behavior):
- Wrong endpoint URL or HTTP method
- Missing required params or wrong `required` flag
- Incorrect response field mapping (accessing wrong path in response)
- Missing error handling that would cause crashes
- Tool ID mismatch between tool file, registry, and block `tools.access`
- OAuth scopes missing in `auth.ts` that tools need
- `tools.config.tool` returning wrong tool ID for an operation
- Type coercions in `tools.config.tool` instead of `tools.config.params`

**Warning** (follows conventions incorrectly or has usability issues):
- Optional field not set to `mode: 'advanced'`
- Missing `wandConfig` on timestamp/complex fields
- Wrong `visibility` on params (e.g., `'hidden'` instead of `'user-or-llm'`)
- Missing `optional: true` on nullable outputs
- Opaque `type: 'json'` without property descriptions
- Missing `.trim()` on ID fields in request URLs
- Missing `?? null` on nullable response fields
- Block condition array missing an operation that uses that field
- Hardcoded scope arrays instead of using `getScopesForService()` / `getCanonicalScopesForProvider()`
- Missing scope description in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`

**Suggestion** (minor improvements):
- Better description text
- Inconsistent naming across tools
- Missing `longDescription` or `docsLink`
- Pagination fields that could benefit from `wandConfig`

### Fix All Issues

After reporting, fix every **critical** and **warning** issue. Apply **suggestions** where they don't add unnecessary complexity.

### Validation Output

After fixing, confirm:
1. `bun run lint` passes with no fixes needed
2. TypeScript compiles clean (no type errors)
3. Re-read all modified files to verify fixes are correct

## Checklist Summary

- [ ] Read ALL tool files, block, types, index, and registries
- [ ] Pulled and read official API documentation
- [ ] Validated every tool's ID, params, request, response, outputs, and types against API docs
- [ ] Validated block ↔ tool alignment (every tool param has a subBlock, every condition is correct)
- [ ] Validated advanced mode on optional/rarely-used fields
- [ ] Validated wandConfig on timestamps and complex inputs
- [ ] Validated tools.config mapping, tool selector, and type coercions
- [ ] Validated block outputs match what tools return, with typed JSON where possible
- [ ] Validated OAuth scopes use centralized utilities (getScopesForService, getCanonicalScopesForProvider) — no hardcoded arrays
- [ ] Validated scope descriptions exist in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` for all scopes
- [ ] Validated pagination consistency across tools and block
- [ ] Validated error handling (error checks, meaningful messages)
- [ ] Validated registry entries (tools and block, alphabetical, correct imports)
- [ ] Validated `{Service}BlockMeta` exported with at least 7 templates
- [ ] Reported all issues grouped by severity
- [ ] Fixed all critical and warning issues
- [ ] Ran `bun run lint` after fixes
- [ ] Verified TypeScript compiles clean