d25d482dc2
Publish CLI Package / publish-npm (push) Waiting to run
Publish Python SDK / publish-pypi (push) Waiting to run
Publish TypeScript SDK / publish-npm (push) Waiting to run
CI / Migrate Dev DB (push) Has been skipped
CI / Detect Version (push) Has been cancelled
CI / Migrate DB (push) Has been cancelled
CI / Build Dev ECR (./docker/app.Dockerfile, ECR_APP) (push) Has been cancelled
CI / Build Dev ECR (./docker/db.Dockerfile, ECR_MIGRATIONS) (push) Has been cancelled
CI / Build Dev ECR (./docker/pii.Dockerfile, ECR_PII) (push) Has been cancelled
CI / Build Dev ECR (./docker/realtime.Dockerfile, ECR_REALTIME) (push) Has been cancelled
CI / Deploy Trigger.dev (Dev) (push) Has been cancelled
CI / Build AMD64 (./docker/app.Dockerfile, ECR_APP, ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Build AMD64 (./docker/db.Dockerfile, ECR_MIGRATIONS, ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Build AMD64 (./docker/pii.Dockerfile, ECR_PII, ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Build AMD64 (./docker/realtime.Dockerfile, ECR_REALTIME, ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/app.Dockerfile, ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/db.Dockerfile, ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/pii.Dockerfile, ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/realtime.Dockerfile, ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Check Docs Changes (push) Has been cancelled
CI / Process Docs (push) Has been cancelled
CI / Create GitHub Release (push) Has been cancelled
CI / Test and Build (push) Has been cancelled
819 lines
27 KiB
Markdown
819 lines
27 KiB
Markdown
---
|
|
description: Add a complete integration to Sim (tools, block, icon, registration)
|
|
argument-hint: <service-name> [api-docs-url]
|
|
---
|
|
|
|
# Add Integration Skill
|
|
|
|
You are an expert at adding complete integrations to Sim. This skill orchestrates the full process of adding a new service integration.
|
|
|
|
## Overview
|
|
|
|
Adding an integration involves these steps in order:
|
|
1. **Research** - Read the service's API documentation
|
|
2. **Create Tools** - Build tool configurations for each API operation
|
|
3. **Create Block** - Build the block UI configuration
|
|
4. **Add Icon** - Add the service's brand icon
|
|
5. **Create Triggers** (optional) - If the service supports webhooks
|
|
6. **Register** - Register tools, block, and triggers in their registries
|
|
7. **Generate Docs** - Run the docs generation script
|
|
|
|
## Step 1: Research the API
|
|
|
|
Before writing any code:
|
|
1. Use Context7 to find official documentation: `mcp__context7__resolve-library-id`, then fetch with `mcp__context7__query-docs`
|
|
2. Or use WebFetch to read API docs directly
|
|
3. Identify:
|
|
- Authentication method (OAuth, API Key, both)
|
|
- Available operations (CRUD, search, etc.)
|
|
- Required vs optional parameters
|
|
- Response structures
|
|
|
|
## Step 2: Create Tools
|
|
|
|
### Directory Structure
|
|
```
|
|
apps/sim/tools/{service}/
|
|
├── index.ts # Barrel exports
|
|
├── types.ts # TypeScript interfaces
|
|
├── {action1}.ts # Tool for action 1
|
|
├── {action2}.ts # Tool for action 2
|
|
└── ...
|
|
```
|
|
|
|
### Key Patterns
|
|
|
|
**types.ts:**
|
|
```typescript
|
|
import type { ToolResponse } from '@/tools/types'
|
|
|
|
export interface {Service}{Action}Params {
|
|
accessToken: string // For OAuth services
|
|
// OR
|
|
apiKey: string // For API key services
|
|
|
|
requiredParam: string
|
|
optionalParam?: string
|
|
}
|
|
|
|
export interface {Service}Response extends ToolResponse {
|
|
output: {
|
|
// Define output structure
|
|
}
|
|
}
|
|
```
|
|
|
|
**Tool file pattern:**
|
|
```typescript
|
|
export const {service}{Action}Tool: ToolConfig<Params, Response> = {
|
|
id: '{service}_{action}',
|
|
name: '{Service} {Action}',
|
|
description: '...',
|
|
version: '1.0.0',
|
|
|
|
oauth: { required: true, provider: '{service}' }, // If OAuth
|
|
|
|
params: {
|
|
accessToken: { type: 'string', required: true, visibility: 'hidden', description: '...' },
|
|
// ... other params
|
|
},
|
|
|
|
request: { url, method, headers, body },
|
|
|
|
transformResponse: async (response) => {
|
|
const data = await response.json()
|
|
return {
|
|
success: true,
|
|
output: {
|
|
field: data.field ?? null, // Always handle nullables
|
|
},
|
|
}
|
|
},
|
|
|
|
outputs: { /* ... */ },
|
|
}
|
|
```
|
|
|
|
### Critical Rules
|
|
- `visibility: 'hidden'` for OAuth tokens
|
|
- `visibility: 'user-only'` for API keys and user credentials
|
|
- `visibility: 'user-or-llm'` for operation parameters
|
|
- Always use `?? null` for nullable API response fields
|
|
- Always use `?? []` for optional array fields
|
|
- Set `optional: true` for outputs that may not exist
|
|
- Never output raw JSON dumps - extract meaningful fields
|
|
- When using `type: 'json'` and you know the object shape, define `properties` with the inner fields so downstream consumers know the structure. Only use bare `type: 'json'` when the shape is truly dynamic
|
|
|
|
## Step 3: Create Block
|
|
|
|
### File Location
|
|
`apps/sim/blocks/blocks/{service}.ts`
|
|
|
|
### Block Structure
|
|
```typescript
|
|
import { {Service}Icon } from '@/components/icons'
|
|
import type { BlockConfig } from '@/blocks/types'
|
|
import { AuthMode, IntegrationType } from '@/blocks/types'
|
|
import { getScopesForService } from '@/lib/oauth/utils'
|
|
|
|
export const {Service}Block: BlockConfig = {
|
|
type: '{service}',
|
|
name: '{Service}',
|
|
description: '...',
|
|
longDescription: '...',
|
|
docsLink: 'https://docs.sim.ai/integrations/{service}',
|
|
category: 'tools',
|
|
integrationType: IntegrationType.X, // Primary category (see IntegrationType enum)
|
|
tags: ['oauth', 'api'], // Cross-cutting tags (see IntegrationTag type)
|
|
bgColor: '#HEXCOLOR',
|
|
icon: {Service}Icon,
|
|
authMode: AuthMode.OAuth, // or AuthMode.ApiKey
|
|
|
|
subBlocks: [
|
|
// Operation dropdown
|
|
{
|
|
id: 'operation',
|
|
title: 'Operation',
|
|
type: 'dropdown',
|
|
options: [
|
|
{ label: 'Operation 1', id: 'action1' },
|
|
{ label: 'Operation 2', id: 'action2' },
|
|
],
|
|
value: () => 'action1',
|
|
},
|
|
// Credential field
|
|
{
|
|
id: 'credential',
|
|
title: '{Service} Account',
|
|
type: 'oauth-input',
|
|
serviceId: '{service}',
|
|
requiredScopes: getScopesForService('{service}'),
|
|
required: true,
|
|
},
|
|
// Conditional fields per operation
|
|
// ...
|
|
],
|
|
|
|
tools: {
|
|
access: ['{service}_action1', '{service}_action2'],
|
|
config: {
|
|
tool: (params) => `{service}_${params.operation}`,
|
|
},
|
|
},
|
|
|
|
outputs: { /* ... */ },
|
|
}
|
|
```
|
|
|
|
### Key SubBlock Patterns
|
|
|
|
**Condition-based visibility:**
|
|
```typescript
|
|
{
|
|
id: 'resourceId',
|
|
title: 'Resource ID',
|
|
type: 'short-input',
|
|
condition: { field: 'operation', value: ['read', 'update', 'delete'] },
|
|
required: { field: 'operation', value: ['read', 'update', 'delete'] },
|
|
}
|
|
```
|
|
|
|
**DependsOn for cascading selectors:**
|
|
```typescript
|
|
{
|
|
id: 'project',
|
|
type: 'project-selector',
|
|
dependsOn: ['credential'],
|
|
},
|
|
{
|
|
id: 'issue',
|
|
type: 'file-selector',
|
|
dependsOn: ['credential', 'project'],
|
|
}
|
|
```
|
|
|
|
**Basic/Advanced mode for dual UX:**
|
|
```typescript
|
|
// Basic: Visual selector
|
|
{
|
|
id: 'channel',
|
|
type: 'channel-selector',
|
|
mode: 'basic',
|
|
canonicalParamId: 'channel',
|
|
dependsOn: ['credential'],
|
|
},
|
|
// Advanced: Manual input
|
|
{
|
|
id: 'channelId',
|
|
type: 'short-input',
|
|
mode: 'advanced',
|
|
canonicalParamId: 'channel',
|
|
}
|
|
```
|
|
|
|
**Critical Canonical Param Rules:**
|
|
- `canonicalParamId` must NOT match any subblock's `id` in the block
|
|
- `canonicalParamId` must be unique per operation/condition context
|
|
- Only use `canonicalParamId` to link basic/advanced alternatives for the same logical parameter
|
|
- `mode` only controls UI visibility, NOT serialization. Without `canonicalParamId`, both basic and advanced field values would be sent
|
|
- Every subblock `id` must be unique within the block. Duplicate IDs cause conflicts even with different conditions
|
|
- **Required consistency:** If one subblock in a canonical group has `required: true`, ALL subblocks in that group must have `required: true` (prevents bypassing validation by switching modes)
|
|
- **Inputs section:** Must list canonical param IDs (e.g., `fileId`), NOT raw subblock IDs (e.g., `fileSelector`, `manualFileId`)
|
|
- **Params function:** Must use canonical param IDs, NOT raw subblock IDs (raw IDs are deleted after canonical transformation)
|
|
|
|
### BlockMeta (Required)
|
|
|
|
Export a `{Service}BlockMeta` in the same file as the block — **minimum 7 templates**. See `add-block.md` → "BlockMeta (Required)" for valid `modules` and `category` values and the full pattern.
|
|
|
|
```typescript
|
|
export const {Service}BlockMeta = {
|
|
tags: ['tag1', 'tag2'],
|
|
templates: [
|
|
{
|
|
icon: {Service}Icon,
|
|
title: '{Service} <use-case>',
|
|
prompt: 'Build a workflow that...', // concrete trigger → transformation → output
|
|
modules: ['agent', 'workflows'],
|
|
category: 'operations',
|
|
tags: ['automation'],
|
|
alsoIntegrations: ['slack'], // when the prompt references another service
|
|
},
|
|
// ... at least 6 more
|
|
],
|
|
} as const satisfies BlockMeta
|
|
```
|
|
|
|
## Step 4: Add Icon
|
|
|
|
### File Location
|
|
`apps/sim/components/icons.tsx`
|
|
|
|
### Pattern
|
|
```typescript
|
|
export function {Service}Icon(props: SVGProps<SVGSVGElement>) {
|
|
return (
|
|
<svg
|
|
{...props}
|
|
viewBox="0 0 24 24"
|
|
fill="none"
|
|
xmlns="http://www.w3.org/2000/svg"
|
|
>
|
|
{/* SVG paths from user-provided SVG */}
|
|
</svg>
|
|
)
|
|
}
|
|
```
|
|
|
|
### Getting Icons
|
|
**Do NOT search for icons yourself.** At the end of implementation, ask the user to provide the SVG:
|
|
|
|
```
|
|
I've completed the integration. Before I can add the icon, please provide the SVG for {Service}.
|
|
You can usually find this in the service's brand/press kit page, or copy it from their website.
|
|
|
|
Paste the SVG code here and I'll convert it to a React component.
|
|
```
|
|
|
|
Once the user provides the SVG:
|
|
1. Extract the SVG paths/content
|
|
2. Create a React component that spreads props
|
|
3. Ensure viewBox is preserved from the original SVG
|
|
|
|
### Theme-safety (bare rendering) — REQUIRED
|
|
|
|
The icon renders both inside its colored `bgColor` tile AND "bare" (no tile) on a
|
|
neutral page — e.g. the home **Suggested actions** list — in both light and dark
|
|
mode. A monochrome logo whose paths hardcode a single near-white or near-black
|
|
fill is invisible bare on the matching background (white-on-white in light mode,
|
|
black-on-black in dark mode).
|
|
|
|
Rules when adding the SVG:
|
|
|
|
- **Monochrome logos** (a single white or black mark): draw the shape with
|
|
`fill='currentColor'`, not `fill='#fff'` / `fill='#000000'`. It then inherits
|
|
white inside dark tiles, near-black inside light tiles (via
|
|
`getTileIconColorClass`), and the theme-aware `var(--text-icon)` bare — legible
|
|
everywhere. Do NOT set `iconColor` for these.
|
|
- **Multi-color brand logos** (their own vivid fills): keep the hardcoded fills.
|
|
They read on any background. Only set `iconColor` (a vivid brand hex, never a
|
|
near-black/near-white tile color) if the bare icon should adopt a brand tint.
|
|
- A large white shape with a tiny vivid accent (e.g. a logo where the body is the
|
|
white negative space) still vanishes bare — convert the body to `currentColor`.
|
|
|
|
Verify with `bun run check:bare-icons` (also runs in CI). It flags purely
|
|
monochrome hazards; for partial-accent logos, eyeball the suggested-actions list
|
|
in both light and dark mode.
|
|
|
|
## Step 5: Create Triggers (Optional)
|
|
|
|
If the service supports webhooks, create triggers using the generic `buildTriggerSubBlocks` helper.
|
|
|
|
### Directory Structure
|
|
```
|
|
apps/sim/triggers/{service}/
|
|
├── index.ts # Barrel exports
|
|
├── utils.ts # Trigger options, setup instructions, extra fields
|
|
├── {event_a}.ts # Primary trigger (includes dropdown)
|
|
├── {event_b}.ts # Secondary triggers (no dropdown)
|
|
└── webhook.ts # Generic webhook (optional)
|
|
```
|
|
|
|
### Key Pattern
|
|
|
|
```typescript
|
|
import { buildTriggerSubBlocks } from '@/triggers'
|
|
import { {service}TriggerOptions, {service}SetupInstructions, build{Service}ExtraFields } from './utils'
|
|
|
|
// Primary trigger - includeDropdown: true
|
|
export const {service}EventATrigger: TriggerConfig = {
|
|
id: '{service}_event_a',
|
|
subBlocks: buildTriggerSubBlocks({
|
|
triggerId: '{service}_event_a',
|
|
triggerOptions: {service}TriggerOptions,
|
|
includeDropdown: true, // Only for primary trigger!
|
|
setupInstructions: {service}SetupInstructions('Event A'),
|
|
extraFields: build{Service}ExtraFields('{service}_event_a'),
|
|
}),
|
|
// ...
|
|
}
|
|
|
|
// Secondary triggers - no dropdown
|
|
export const {service}EventBTrigger: TriggerConfig = {
|
|
id: '{service}_event_b',
|
|
subBlocks: buildTriggerSubBlocks({
|
|
triggerId: '{service}_event_b',
|
|
triggerOptions: {service}TriggerOptions,
|
|
// No includeDropdown!
|
|
setupInstructions: {service}SetupInstructions('Event B'),
|
|
extraFields: build{Service}ExtraFields('{service}_event_b'),
|
|
}),
|
|
// ...
|
|
}
|
|
```
|
|
|
|
### Connect to Block
|
|
```typescript
|
|
import { getTrigger } from '@/triggers'
|
|
|
|
export const {Service}Block: BlockConfig = {
|
|
triggers: {
|
|
enabled: true,
|
|
available: ['{service}_event_a', '{service}_event_b'],
|
|
},
|
|
subBlocks: [
|
|
// Tool fields...
|
|
...getTrigger('{service}_event_a').subBlocks,
|
|
...getTrigger('{service}_event_b').subBlocks,
|
|
],
|
|
}
|
|
```
|
|
|
|
See `/add-trigger` skill for complete documentation.
|
|
|
|
## Step 6: Register Everything
|
|
|
|
### Tools Registry (`apps/sim/tools/registry.ts`)
|
|
|
|
```typescript
|
|
// Add import (alphabetically)
|
|
import {
|
|
{service}Action1Tool,
|
|
{service}Action2Tool,
|
|
} from '@/tools/{service}'
|
|
|
|
// Add to tools object (alphabetically)
|
|
export const tools: Record<string, ToolConfig> = {
|
|
// ... existing tools ...
|
|
{service}_action1: {service}Action1Tool,
|
|
{service}_action2: {service}Action2Tool,
|
|
}
|
|
```
|
|
|
|
### Block Registry (`apps/sim/blocks/registry-maps.ts`)
|
|
|
|
The data maps (`BLOCK_REGISTRY` + `BLOCK_META_REGISTRY`) live in `registry-maps.ts`; `registry.ts` holds only the accessor functions. Add the import and an entry to each map alphabetically:
|
|
|
|
```typescript
|
|
// Add import (alphabetically)
|
|
import { {Service}Block, {Service}BlockMeta } from '@/blocks/blocks/{service}'
|
|
|
|
// Add to the config map (alphabetically)
|
|
export const BLOCK_REGISTRY: Record<string, BlockConfig> = {
|
|
// ... existing blocks ...
|
|
{service}: {Service}Block,
|
|
}
|
|
|
|
// Add to the catalog-meta map (alphabetically)
|
|
export const BLOCK_META_REGISTRY: Record<string, BlockMeta> = {
|
|
// ... existing metas ...
|
|
{service}: {Service}BlockMeta,
|
|
}
|
|
```
|
|
|
|
### Trigger Registry (`apps/sim/triggers/registry.ts`) - If triggers exist
|
|
|
|
```typescript
|
|
// Add import (alphabetically)
|
|
import {
|
|
{service}EventATrigger,
|
|
{service}EventBTrigger,
|
|
{service}WebhookTrigger,
|
|
} from '@/triggers/{service}'
|
|
|
|
// Add to TRIGGER_REGISTRY (alphabetically)
|
|
export const TRIGGER_REGISTRY: TriggerRegistry = {
|
|
// ... existing triggers ...
|
|
{service}_event_a: {service}EventATrigger,
|
|
{service}_event_b: {service}EventBTrigger,
|
|
{service}_webhook: {service}WebhookTrigger,
|
|
}
|
|
```
|
|
|
|
## Step 7: Generate Docs
|
|
|
|
Run the documentation generator (from `apps/sim`):
|
|
```bash
|
|
bun run generate-docs
|
|
```
|
|
|
|
This creates `apps/docs/content/docs/en/tools/{service}.mdx`
|
|
|
|
## V2 Integration Pattern
|
|
|
|
If creating V2 versions (API-aligned outputs):
|
|
|
|
1. **V2 Tools** - Add `_v2` suffix, version `2.0.0`, flat outputs
|
|
2. **V2 Block** - Add `_v2` type, use `createVersionedToolSelector`
|
|
3. **V1 Block** - Add `(Legacy)` to name, set `hideFromToolbar: true`
|
|
4. **Registry** - Register both versions
|
|
|
|
```typescript
|
|
// In registry
|
|
{service}: {Service}Block, // V1 (legacy, hidden)
|
|
{service}_v2: {Service}V2Block, // V2 (visible)
|
|
```
|
|
|
|
## Complete Checklist
|
|
|
|
### Tools
|
|
- [ ] Created `tools/{service}/` directory
|
|
- [ ] Created `types.ts` with all interfaces
|
|
- [ ] Created tool file for each operation
|
|
- [ ] All params have correct visibility
|
|
- [ ] All nullable fields use `?? null`
|
|
- [ ] All optional outputs have `optional: true`
|
|
- [ ] Created `index.ts` barrel export
|
|
- [ ] Registered all tools in `tools/registry.ts`
|
|
|
|
### Block
|
|
- [ ] Created `blocks/blocks/{service}.ts`
|
|
- [ ] Set `integrationType` to the correct `IntegrationType` enum value
|
|
- [ ] Set `tags` array with all applicable `IntegrationTag` values
|
|
- [ ] Defined operation dropdown with all operations
|
|
- [ ] Added credential field with `requiredScopes: getScopesForService('{service}')`
|
|
- [ ] Added conditional fields per operation
|
|
- [ ] Set up dependsOn for cascading selectors
|
|
- [ ] Configured tools.access with all tool IDs
|
|
- [ ] Configured tools.config.tool selector
|
|
- [ ] Defined outputs matching tool outputs
|
|
- [ ] Registered block + meta in `blocks/registry-maps.ts` (`BLOCK_REGISTRY` / `BLOCK_META_REGISTRY`)
|
|
- [ ] If triggers: set `triggers.enabled` and `triggers.available`
|
|
- [ ] If triggers: spread trigger subBlocks with `getTrigger()`
|
|
- [ ] Exported `{Service}BlockMeta` with at least 7 templates
|
|
|
|
### OAuth Scopes (if OAuth service)
|
|
- [ ] Defined scopes in `lib/oauth/oauth.ts` under `OAUTH_PROVIDERS`
|
|
- [ ] Added scope descriptions in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`
|
|
- [ ] Used `getCanonicalScopesForProvider()` in `auth.ts` (never hardcode)
|
|
- [ ] Used `getScopesForService()` in block `requiredScopes` (never hardcode)
|
|
|
|
### Icon
|
|
- [ ] Asked user to provide SVG
|
|
- [ ] Added icon to `components/icons.tsx`
|
|
- [ ] Icon spreads props correctly
|
|
- [ ] Monochrome marks use `fill='currentColor'` (not hardcoded white/black) so the icon renders bare in light AND dark mode — verified with `bun run check:bare-icons`
|
|
|
|
### Triggers (if service supports webhooks)
|
|
- [ ] Created `triggers/{service}/` directory
|
|
- [ ] Created `utils.ts` with options, instructions, and extra fields helpers
|
|
- [ ] Primary trigger uses `includeDropdown: true`
|
|
- [ ] Secondary triggers do NOT have `includeDropdown`
|
|
- [ ] All triggers use `buildTriggerSubBlocks` helper
|
|
- [ ] Created `index.ts` barrel export
|
|
- [ ] Registered all triggers in `triggers/registry.ts`
|
|
|
|
### Docs
|
|
- [ ] Ran `bun run scripts/generate-docs.ts`
|
|
- [ ] Verified docs file created
|
|
|
|
### Final Validation (Required)
|
|
- [ ] Read every tool file and cross-referenced inputs/outputs against the API docs
|
|
- [ ] Verified block subBlocks cover all required tool params with correct conditions
|
|
- [ ] Verified block outputs match what the tools actually return
|
|
- [ ] Verified `tools.config.params` correctly maps and coerces all param types
|
|
- [ ] `{Service}BlockMeta` exported with at least 7 templates, each having `icon`, `title`, `prompt`, `modules`, `category`, and `tags`
|
|
|
|
## Example Command
|
|
|
|
When the user asks to add an integration:
|
|
|
|
```
|
|
User: Add a Stripe integration
|
|
|
|
You: I'll add the Stripe integration. Let me:
|
|
|
|
1. First, research the Stripe API using Context7
|
|
2. Create the tools for key operations (payments, subscriptions, etc.)
|
|
3. Create the block with operation dropdown
|
|
4. Register everything
|
|
5. Generate docs
|
|
6. Ask you for the Stripe icon SVG
|
|
|
|
[Proceed with implementation...]
|
|
|
|
[After completing steps 1-5...]
|
|
|
|
I've completed the Stripe integration. Before I can add the icon, please provide the SVG for Stripe.
|
|
You can usually find this in the service's brand/press kit page, or copy it from their website.
|
|
|
|
Paste the SVG code here and I'll convert it to a React component.
|
|
```
|
|
|
|
## File Handling
|
|
|
|
When your integration handles file uploads or downloads, follow these patterns to work with `UserFile` objects consistently.
|
|
|
|
### What is a UserFile?
|
|
|
|
A `UserFile` is the standard file representation in Sim:
|
|
|
|
```typescript
|
|
interface UserFile {
|
|
id: string // Unique identifier
|
|
name: string // Original filename
|
|
url: string // Presigned URL for download
|
|
size: number // File size in bytes
|
|
type: string // MIME type (e.g., 'application/pdf')
|
|
base64?: string // Optional base64 content (if small file)
|
|
key?: string // Internal storage key
|
|
context?: object // Storage context metadata
|
|
}
|
|
```
|
|
|
|
### File Input Pattern (Uploads)
|
|
|
|
For tools that accept file uploads, **always route through an internal API endpoint** rather than calling external APIs directly. This ensures proper file content retrieval.
|
|
|
|
#### 1. Block SubBlocks for File Input
|
|
|
|
Use the basic/advanced mode pattern:
|
|
|
|
```typescript
|
|
// Basic mode: File upload UI
|
|
{
|
|
id: 'uploadFile',
|
|
title: 'File',
|
|
type: 'file-upload',
|
|
canonicalParamId: 'file', // Maps to 'file' param
|
|
placeholder: 'Upload file',
|
|
mode: 'basic',
|
|
multiple: false,
|
|
required: true,
|
|
condition: { field: 'operation', value: 'upload' },
|
|
},
|
|
// Advanced mode: Reference from previous block
|
|
{
|
|
id: 'fileRef',
|
|
title: 'File',
|
|
type: 'short-input',
|
|
canonicalParamId: 'file', // Same canonical param
|
|
placeholder: 'Reference file (e.g., {{file_block.output}})',
|
|
mode: 'advanced',
|
|
required: true,
|
|
condition: { field: 'operation', value: 'upload' },
|
|
},
|
|
```
|
|
|
|
**Critical:** `canonicalParamId` must NOT match any subblock `id`.
|
|
|
|
#### 2. Normalize File Input in Block Config
|
|
|
|
In `tools.config.tool`, use `normalizeFileInput` to handle all input variants:
|
|
|
|
```typescript
|
|
import { normalizeFileInput } from '@/blocks/utils'
|
|
|
|
tools: {
|
|
config: {
|
|
tool: (params) => {
|
|
// Normalize file from basic (uploadFile), advanced (fileRef), or legacy (fileContent)
|
|
const normalizedFile = normalizeFileInput(
|
|
params.uploadFile || params.fileRef || params.fileContent,
|
|
{ single: true }
|
|
)
|
|
if (normalizedFile) {
|
|
params.file = normalizedFile
|
|
}
|
|
return `{service}_${params.operation}`
|
|
},
|
|
},
|
|
}
|
|
```
|
|
|
|
#### 3. Create Internal API Route
|
|
|
|
Create `apps/sim/app/api/tools/{service}/{action}/route.ts`:
|
|
|
|
```typescript
|
|
import { createLogger } from '@sim/logger'
|
|
import { NextResponse, type NextRequest } from 'next/server'
|
|
import { {service}UploadContract } from '@/lib/api/contracts/tools/{service}'
|
|
import { parseRequest } from '@/lib/api/server'
|
|
import { checkInternalAuth } from '@/lib/auth/hybrid'
|
|
import { generateRequestId } from '@/lib/core/utils/request'
|
|
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
|
import { type RawFileInput } from '@/lib/uploads/utils/file-schemas'
|
|
import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils'
|
|
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
|
|
|
|
const logger = createLogger('{Service}UploadAPI')
|
|
|
|
export const POST = withRouteHandler(async (request: NextRequest) => {
|
|
const requestId = generateRequestId()
|
|
|
|
// Auth always runs BEFORE parseRequest — never validate untrusted input before authenticating.
|
|
const authResult = await checkInternalAuth(request, { requireWorkflowId: false })
|
|
if (!authResult.success) {
|
|
return NextResponse.json({ success: false, error: 'Unauthorized' }, { status: 401 })
|
|
}
|
|
|
|
const parsed = await parseRequest({service}UploadContract, request, {})
|
|
if (!parsed.success) return parsed.response
|
|
const data = parsed.data.body
|
|
|
|
let fileBuffer: Buffer
|
|
let fileName: string
|
|
|
|
// Prefer UserFile input, fall back to legacy base64
|
|
if (data.file) {
|
|
const userFiles = processFilesToUserFiles([data.file as RawFileInput], requestId, logger)
|
|
if (userFiles.length === 0) {
|
|
return NextResponse.json({ success: false, error: 'Invalid file' }, { status: 400 })
|
|
}
|
|
const userFile = userFiles[0]
|
|
fileBuffer = await downloadFileFromStorage(userFile, requestId, logger)
|
|
fileName = userFile.name
|
|
} else if (data.fileContent) {
|
|
// Legacy: base64 string (backwards compatibility)
|
|
fileBuffer = Buffer.from(data.fileContent, 'base64')
|
|
fileName = 'file'
|
|
} else {
|
|
return NextResponse.json({ success: false, error: 'File required' }, { status: 400 })
|
|
}
|
|
|
|
// Now call external API with fileBuffer
|
|
const response = await fetch('https://api.{service}.com/upload', {
|
|
method: 'POST',
|
|
headers: { Authorization: `Bearer ${data.accessToken}` },
|
|
body: new Uint8Array(fileBuffer), // Convert Buffer for fetch
|
|
})
|
|
|
|
// ... handle response
|
|
})
|
|
```
|
|
|
|
#### 4. Update Tool to Use Internal Route
|
|
|
|
```typescript
|
|
export const {service}UploadTool: ToolConfig<Params, Response> = {
|
|
id: '{service}_upload',
|
|
// ...
|
|
params: {
|
|
file: { type: 'file', required: false, visibility: 'user-or-llm' },
|
|
fileContent: { type: 'string', required: false, visibility: 'hidden' }, // Legacy
|
|
},
|
|
request: {
|
|
url: '/api/tools/{service}/upload', // Internal route
|
|
method: 'POST',
|
|
body: (params) => ({
|
|
accessToken: params.accessToken,
|
|
file: params.file,
|
|
fileContent: params.fileContent,
|
|
}),
|
|
},
|
|
}
|
|
```
|
|
|
|
### File Output Pattern (Downloads)
|
|
|
|
For tools that return files, use `FileToolProcessor` to store files and return `UserFile` objects.
|
|
|
|
#### In Tool transformResponse
|
|
|
|
```typescript
|
|
import { FileToolProcessor } from '@/executor/utils/file-tool-processor'
|
|
|
|
transformResponse: async (response, context) => {
|
|
const data = await response.json()
|
|
|
|
// Process file outputs to UserFile objects
|
|
const fileProcessor = new FileToolProcessor(context)
|
|
const file = await fileProcessor.processFileData({
|
|
data: data.content, // base64 or buffer
|
|
mimeType: data.mimeType,
|
|
filename: data.filename,
|
|
})
|
|
|
|
return {
|
|
success: true,
|
|
output: { file },
|
|
}
|
|
}
|
|
```
|
|
|
|
#### In API Route (for complex file handling)
|
|
|
|
```typescript
|
|
// Return file data that FileToolProcessor can handle
|
|
return NextResponse.json({
|
|
success: true,
|
|
output: {
|
|
file: {
|
|
data: base64Content,
|
|
mimeType: 'application/pdf',
|
|
filename: 'document.pdf',
|
|
},
|
|
},
|
|
})
|
|
```
|
|
|
|
### Key Helpers Reference
|
|
|
|
| Helper | Location | Purpose |
|
|
|--------|----------|---------|
|
|
| `normalizeFileInput` | `@/blocks/utils` | Normalize file params in block config |
|
|
| `processFilesToUserFiles` | `@/lib/uploads/utils/file-utils` | Convert raw inputs to UserFile[] |
|
|
| `downloadFileFromStorage` | `@/lib/uploads/utils/file-utils.server` | Get file Buffer from UserFile |
|
|
| `FileToolProcessor` | `@/executor/utils/file-tool-processor` | Process tool output files |
|
|
| `isUserFile` | `@/lib/core/utils/user-file` | Type guard for UserFile objects |
|
|
| `FileInputSchema` | `@/lib/uploads/utils/file-schemas` | Zod schema for file validation |
|
|
|
|
### Advanced Mode for Optional Fields
|
|
|
|
Optional fields that are rarely used should be set to `mode: 'advanced'` so they don't clutter the basic UI. Examples: pagination tokens, time range filters, sort order, max results, reply settings.
|
|
|
|
### WandConfig for Complex Inputs
|
|
|
|
Use `wandConfig` for fields that are hard to fill out manually:
|
|
- **Timestamps**: Use `generationType: 'timestamp'` to inject current date context into the AI prompt
|
|
- **JSON arrays**: Use `generationType: 'json-object'` for structured data
|
|
- **Complex queries**: Use a descriptive prompt explaining the expected format
|
|
|
|
```typescript
|
|
{
|
|
id: 'startTime',
|
|
title: 'Start Time',
|
|
type: 'short-input',
|
|
mode: 'advanced',
|
|
wandConfig: {
|
|
enabled: true,
|
|
prompt: 'Generate an ISO 8601 timestamp. Return ONLY the timestamp string.',
|
|
generationType: 'timestamp',
|
|
},
|
|
}
|
|
```
|
|
|
|
### OAuth Scopes (Centralized System)
|
|
|
|
Scopes are maintained in a single source of truth and reused everywhere:
|
|
|
|
1. **Define scopes** in `lib/oauth/oauth.ts` under `OAUTH_PROVIDERS[provider].services[service].scopes`
|
|
2. **Add descriptions** in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` for the OAuth modal UI
|
|
3. **Reference in auth.ts** using `getCanonicalScopesForProvider(providerId)` from `@/lib/oauth/utils`
|
|
4. **Reference in blocks** using `getScopesForService(serviceId)` from `@/lib/oauth/utils`
|
|
|
|
**Never hardcode scope arrays** in `auth.ts` or block `requiredScopes`. Always import from the centralized source.
|
|
|
|
```typescript
|
|
// In auth.ts (Better Auth config)
|
|
scopes: getCanonicalScopesForProvider('{service}'),
|
|
|
|
// In block credential sub-block
|
|
requiredScopes: getScopesForService('{service}'),
|
|
```
|
|
|
|
### Common Gotchas
|
|
|
|
1. **OAuth serviceId must match** - The `serviceId` in oauth-input must match the OAuth provider configuration
|
|
2. **All tool IDs MUST be snake_case** - `stripe_create_payment`, not `stripeCreatePayment`. This applies to tool `id` fields, registry keys, `tools.access` arrays, and `tools.config.tool` return values
|
|
3. **Block type is snake_case** - `type: 'stripe'`, not `type: 'Stripe'`
|
|
4. **Alphabetical ordering** - Keep imports and registry entries alphabetically sorted
|
|
5. **Required can be conditional** - Use `required: { field: 'op', value: 'create' }` instead of always true
|
|
6. **DependsOn clears options** - When a dependency changes, selector options are refetched
|
|
7. **Never pass Buffer directly to fetch** - Convert to `new Uint8Array(buffer)` for TypeScript compatibility
|
|
8. **Always handle legacy file params** - Keep hidden `fileContent` params for backwards compatibility
|
|
9. **Optional fields use advanced mode** - Set `mode: 'advanced'` on rarely-used optional fields
|
|
10. **Complex inputs need wandConfig** - Timestamps, JSON arrays, and other hard-to-type values should have `wandConfig` enabled
|
|
11. **Never hardcode scopes** - Use `getScopesForService()` in blocks and `getCanonicalScopesForProvider()` in auth.ts
|
|
12. **Always add scope descriptions** - New scopes must have entries in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`
|