chore: import upstream snapshot with attribution
Publish CLI Package / publish-npm (push) Waiting to run
Publish Python SDK / publish-pypi (push) Waiting to run
Publish TypeScript SDK / publish-npm (push) Waiting to run
CI / Migrate Dev DB (push) Has been skipped
CI / Detect Version (push) Has been cancelled
CI / Migrate DB (push) Has been cancelled
CI / Build Dev ECR (./docker/app.Dockerfile, ECR_APP) (push) Has been cancelled
CI / Build Dev ECR (./docker/db.Dockerfile, ECR_MIGRATIONS) (push) Has been cancelled
CI / Build Dev ECR (./docker/pii.Dockerfile, ECR_PII) (push) Has been cancelled
CI / Build Dev ECR (./docker/realtime.Dockerfile, ECR_REALTIME) (push) Has been cancelled
CI / Deploy Trigger.dev (Dev) (push) Has been cancelled
CI / Build AMD64 (./docker/app.Dockerfile, ECR_APP, ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Build AMD64 (./docker/db.Dockerfile, ECR_MIGRATIONS, ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Build AMD64 (./docker/pii.Dockerfile, ECR_PII, ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Build AMD64 (./docker/realtime.Dockerfile, ECR_REALTIME, ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/app.Dockerfile, ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/db.Dockerfile, ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/pii.Dockerfile, ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/realtime.Dockerfile, ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Check Docs Changes (push) Has been cancelled
CI / Process Docs (push) Has been cancelled
CI / Create GitHub Release (push) Has been cancelled
CI / Test and Build (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:20:55 +08:00
commit d25d482dc2
13754 changed files with 4996608 additions and 0 deletions
+89
View File
@@ -0,0 +1,89 @@
# Integration documentation generator
`generate-docs.ts` compiles the per-service **integration** pages under
`apps/docs/content/docs/en/integrations/` from the block/tool/trigger registry in
`apps/sim`. The ontology it encodes: everything is a block, and an integration is one
block that has **Actions** and, optionally, a **Trigger**.
> **Golden rule:** the generated `.mdx` files are *derived artifacts*, not the source of
> truth. Do not hand-edit them — your changes are overwritten on the next run. The only
> editable region is the `MANUAL-CONTENT` block (see below). To change what a page says,
> edit the TypeScript in `apps/sim` and regenerate.
## Where an integration lives canonically
For a service like Gmail, three TS sources define it:
| Source | What it is | What it feeds in the page |
| --- | --- | --- |
| `apps/sim/blocks/blocks/<service>.ts` | The **block**: `type`, `name`, `category` (`tools` for integrations), `bgColor`, config sub-blocks, `tools.access` (which actions it exposes), an optional `triggers` capability, `outputs` | Header / `BlockInfoCard`, Usage Instructions, and *which* actions + trigger appear |
| `apps/sim/tools/<service>/*.ts` | Each **action's** params + outputs | Every `### <action>``#### Input` / `#### Output` under `## Actions` |
| `apps/sim/triggers/<provider>/` | The **trigger's** config fields + outputs | The `## Triggers` section |
| `apps/sim/components/icons.tsx` | The brand glyph | The page icon |
The block references actions by id in `tools.access`; the generator looks each one up in
`apps/sim/tools/`.
## What the generator does
Run with `cd apps/sim && bun run generate-docs` (or `bun run scripts/generate-docs.ts`
from the repo root). One pass (`generateAllBlockDocs`):
1. **Copies icons** `apps/sim/components/icons.tsx``apps/docs/components/icons.tsx` and
builds `apps/docs/components/ui/icon-mapping.ts`.
2. **Block pass** — for each integration block (`category: 'tools'`, plus the `memory` /
`knowledge` / `table` exceptions), writes `integrations/<service>.mdx`:
`BlockInfoCard` + Usage Instructions + `## Actions`.
3. **Trigger pass** (`generateAllTriggerDocs`) — reads `apps/sim/triggers/<provider>/` and
**appends a `## Triggers` section** to that service's page, or writes a standalone page
for trigger-only services.
4. Writes `integrations/meta.json` and regenerates the landing page's `integrations.json`.
### Hand-written pages it never touches
Core block pages (`blocks/*`), the native trigger pages (`triggers/{start,schedule,webhook,rss,table}`),
the integrations overview (`integrations/index.mdx`), and the service-account pages are
fully hand-written. The generator skips them via `HANDWRITTEN_INTEGRATION_DOCS`,
`HANDWRITTEN_TRIGGER_DOCS`, and `SKIP_TRIGGER_PROVIDERS`. Add a page name to those sets if
you hand-author a page the generator would otherwise produce.
## Manual content (the one editable region)
Each generated page may carry hand-written prose inside marker comments. The generator
preserves anything between the markers and overwrites everything else, so this survives
every regeneration:
```mdx
{/* MANUAL-CONTENT-START:intro */}
[AgentMail](https://agentmail.to/) is an API-first email platform…
{/* MANUAL-CONTENT-END */}
```
Supported section names: `intro` (after the `BlockInfoCard` — the most common),
`usage`, `configuration`, `outputs`, `notes`. The merge is by marker name
(`extractManualContent` + `mergeWithManualContent`), so a section is re-inserted at the
matching spot in the freshly generated structure.
> If you **move** the output folder, reseed manual content from the old location first —
> the generator only preserves markers it finds in the *existing output file*, so a fresh
> folder starts with none.
## Practical: to change…
- **An action's params/outputs, a trigger, or to add a service** → edit
`apps/sim/{blocks,tools,triggers}` and re-run the generator.
- **A page's prose intro** → edit its `MANUAL-CONTENT:intro` block directly; it survives regen.
- **The overview / service-account / core-block / native-trigger pages** → hand-edit freely.
## Gotchas
- **Never hand-edit `apps/docs/components/icons.tsx`** — step 1 overwrites it from the sim
app. Components that need an icon the sim app lacks should define it locally or use
`lucide-react` (see `components/workflow-preview/block-icons.tsx`).
- The generator is the source of truth for `integrations/` and its `meta.json`; manual
edits there are transient.
## CI
The generator runs in CI on pushes to the main branch and commits the regenerated docs
back. Keep block/tool/trigger metadata accurate in `apps/sim` and the docs follow.
File diff suppressed because it is too large Load Diff
+146
View File
@@ -0,0 +1,146 @@
#!/usr/bin/env bun
/**
* Audits brand icons that render "bare" (without their colored tile) for
* theme-safety. The suggested-actions surface and other bare contexts draw a
* block's icon on a neutral page in both light and dark mode. An icon whose
* SVG hardcodes only near-white fills vanishes on a light page; one that
* hardcodes only near-black fills vanishes on a dark page. The fix is to draw
* monochrome marks with `fill='currentColor'` (which adapts via the
* theme-aware foreground) and reserve hardcoded fills for genuinely
* multi-color brand logos.
*
* Scope: blocks that contribute suggested-action prompt rows — i.e. blocks
* whose meta defines `templates`. New integrations land here, so this is where
* regressions are caught. Multi-color icons and `currentColor` icons pass.
*
* Each block's main `icon:` AND every template's own `icon:` is audited (a
* template may reuse another block's brand icon). Only icons imported from
* `@/components/icons` are checked — that is the only module this script can
* resolve; `@sim/emcn/icons` are design-system line icons drawn with
* `currentColor` and are safe by construction, so they are intentionally
* skipped.
*
* Limitation: this catches purely-monochrome icons (only near-white or only
* near-black fills). It cannot catch an icon whose large primary shape is
* white but which also has a small vivid accent (the accent clears the
* heuristic) — that needs a visual light+dark check. Always eyeball new icons
* on the suggested-actions surface in both themes.
*
* Run: `bun run scripts/check-bare-icons.ts`
*/
import { readdir, readFile } from 'node:fs/promises'
import path from 'node:path'
import { perceivedBrightness } from '../apps/sim/lib/colors'
const ROOT = path.resolve(import.meta.dir, '..')
const BLOCKS_DIR = path.join(ROOT, 'apps/sim/blocks/blocks')
const ICONS_FILE = path.join(ROOT, 'apps/sim/components/icons.tsx')
const isNearWhite = (c: string) => {
const b = perceivedBrightness(c)
return b !== null && b > 0.9
}
const isNearBlack = (c: string) => {
const b = perceivedBrightness(c)
return b !== null && b < 0.1
}
/** Extract each exported icon's source body, keyed by component name. */
function indexIconBodies(src: string): Map<string, string> {
const bodies = new Map<string, string>()
const starts: Array<{ name: string; index: number }> = []
for (const m of src.matchAll(/export (?:function|const) (\w+)\s*[=(]/g)) {
starts.push({ name: m[1], index: m.index })
}
for (let i = 0; i < starts.length; i++) {
const end = i + 1 < starts.length ? starts[i + 1].index : src.length
bodies.set(starts[i].name, src.slice(starts[i].index, end))
}
return bodies
}
interface Hazard {
block: string
icon: string
kind: 'light' | 'dark'
detail: string
}
function analyzeIcon(body: string): { hazard: 'light' | 'dark' | null; detail: string } {
if (/currentColor/.test(body)) return { hazard: null, detail: 'uses currentColor' }
if (/url\(#|<stop|inearGradient|adialGradient/.test(body))
return { hazard: null, detail: 'gradient fill' }
const colors: string[] = []
for (const m of body.matchAll(/(?:fill|stroke)=(?:'([^']*)'|"([^"]*)")/g)) {
const v = (m[1] ?? m[2] ?? '').trim()
if (v && v.toLowerCase() !== 'none') colors.push(v)
}
const literal = colors.filter((c) => c.toLowerCase() !== 'currentcolor')
if (literal.length === 0) return { hazard: null, detail: 'no literal fills' }
const vivid = literal.filter((c) => !isNearWhite(c) && !isNearBlack(c))
if (vivid.length > 0) return { hazard: null, detail: `multi-color (${vivid[0]})` }
if (literal.every(isNearWhite))
return { hazard: 'light', detail: `only near-white fills (${literal.join(', ')})` }
if (literal.every(isNearBlack))
return { hazard: 'dark', detail: `only near-black fills (${literal.join(', ')})` }
return { hazard: null, detail: 'mixed' }
}
async function main() {
const iconsSrc = await readFile(ICONS_FILE, 'utf8')
const iconBodies = indexIconBodies(iconsSrc)
const blockFiles = (await readdir(BLOCKS_DIR)).filter((f) => f.endsWith('.ts'))
const hazards: Hazard[] = []
const seen = new Set<string>()
for (const file of blockFiles) {
const src = await readFile(path.join(BLOCKS_DIR, file), 'utf8')
if (!/\btemplates:\s*\[/.test(src)) continue
const brandIcons = new Set<string>()
for (const m of src.matchAll(
/import\s*(?:type\s*)?{([^}]*)}\s*from\s*'@\/components\/icons'/g
)) {
for (const name of m[1].split(',')) {
const trimmed = name.trim()
if (trimmed) brandIcons.add(trimmed)
}
}
for (const m of src.matchAll(/\bicon:\s*(\w+)/g)) {
const iconName = m[1]
if (!brandIcons.has(iconName)) continue
const key = `${file}:${iconName}`
if (seen.has(key)) continue
seen.add(key)
const body = iconBodies.get(iconName)
if (!body) continue
const { hazard, detail } = analyzeIcon(body)
if (hazard) {
hazards.push({ block: file.replace('.ts', ''), icon: iconName, kind: hazard, detail })
}
}
}
if (hazards.length === 0) {
console.log('✓ All suggested-action brand icons render safely bare in light and dark mode.')
process.exit(0)
}
console.error(`\nFound ${hazards.length} bare-icon hazard(s):\n`)
for (const h of hazards) {
const mode = h.kind === 'light' ? 'invisible on LIGHT pages' : 'invisible on DARK pages'
console.error(` ${h.block} (${h.icon}) — ${mode}`)
console.error(` ${h.detail}`)
console.error(
` Fix: draw the monochrome shape with fill='currentColor' in components/icons.tsx`
)
console.error(
` so it adapts to the theme bare and to the tile foreground (getTileIconColorClass).\n`
)
}
process.exit(1)
}
main()
+233
View File
@@ -0,0 +1,233 @@
#!/usr/bin/env bun
/**
* Guards against the Next.js `'use client'` server-import foot-gun.
*
* Next.js rewrites EVERY export of a `'use client'` module into a client
* reference in the server bundle. Server-evaluated code can only *render* such
* an export as a component or pass it as a prop — *calling* one throws at
* runtime ("Attempted to call X from the server but X is on the client"). The
* crash for an object export looks like `tableKeys.list is not a function`.
* `next build` does NOT catch this; only SSR/runtime does.
*
* This script flags any **value** import (not `import type`) that resolves to a
* `'use client'` module from a server-evaluated, non-JSX surface — the places
* that never legitimately render a client component and so only ever import a
* client module to (illegally) call its values:
*
* - `apps/sim/app/** /prefetch*.ts` (RSC server prefetch)
* - `apps/sim/app/api/** /route.ts(x)` (route handlers)
* - `apps/sim/triggers/**` (trigger.dev tasks/pollers/webhooks)
* - `apps/sim/blocks/**` (block definitions — evaluated server-side)
*
* Fix: move the imported query-key factory / standalone fetcher / mapper /
* constant into a non-`'use client'` module (e.g. `hooks/queries/utils/*-keys.ts`
* or `hooks/queries/utils/fetch-*.ts`) and import it from there. See the rule in
* `.claude/rules/sim-queries.md`.
*
* Escape hatch: `// client-boundary-allow: <reason>` on the line directly above
* the import (reason required). Use only for a genuinely browser-only code path.
*
* Usage:
* bun run scripts/check-client-boundary-imports.ts # report
* bun run scripts/check-client-boundary-imports.ts --check # CI gate (fail on any)
*/
import { readdir, readFile } from 'node:fs/promises'
import path from 'node:path'
const ROOT = path.resolve(import.meta.dir, '..')
const APP_DIR = path.join(ROOT, 'apps/sim')
/** Server-evaluated, non-JSX surfaces. A file matches if its path passes one. */
function isServerSurface(rel: string): boolean {
if (/(^|\/)prefetch[^/]*\.ts$/.test(rel)) return true
if (/^app\/api\/.+\/route\.tsx?$/.test(rel)) return true
if (/^triggers\//.test(rel)) return true
if (/^blocks\//.test(rel)) return true
return false
}
const SOURCE_EXTENSIONS = ['.ts', '.tsx']
const ALLOW_DIRECTIVE = 'client-boundary-allow'
async function listFiles(dir: string): Promise<string[]> {
const out: string[] = []
let entries: Awaited<ReturnType<typeof readdir>>
try {
entries = await readdir(dir, { withFileTypes: true })
} catch {
return out
}
for (const entry of entries) {
const full = path.join(dir, entry.name)
if (entry.isDirectory()) {
if (entry.name === 'node_modules' || entry.name === '.next') continue
out.push(...(await listFiles(full)))
} else if (SOURCE_EXTENSIONS.includes(path.extname(entry.name))) {
out.push(full)
}
}
return out
}
const useClientCache = new Map<string, boolean>()
async function isUseClientModule(absFile: string): Promise<boolean> {
const cached = useClientCache.get(absFile)
if (cached !== undefined) return cached
let content: string
try {
content = await readFile(absFile, 'utf8')
} catch {
useClientCache.set(absFile, false)
return false
}
// The directive must be the first statement (comments/blank lines may precede it).
let isClient = false
for (const raw of content.split('\n')) {
const line = raw.trim()
if (line === '' || line.startsWith('//') || line.startsWith('/*') || line.startsWith('*')) {
continue
}
isClient = line === "'use client'" || line === '"use client"'
break
}
useClientCache.set(absFile, isClient)
return isClient
}
/** Resolve an import specifier to an absolute source file, or null if external/unresolved. */
async function resolveSpecifier(spec: string, fromFile: string): Promise<string | null> {
let base: string
if (spec.startsWith('@/')) {
base = path.join(APP_DIR, spec.slice(2))
} else if (spec.startsWith('./') || spec.startsWith('../')) {
base = path.resolve(path.dirname(fromFile), spec)
} else {
return null // external package
}
const candidates = [
base,
...SOURCE_EXTENSIONS.map((ext) => base + ext),
...SOURCE_EXTENSIONS.map((ext) => path.join(base, `index${ext}`)),
]
for (const candidate of candidates) {
if (!SOURCE_EXTENSIONS.includes(path.extname(candidate))) continue
try {
await readFile(candidate, 'utf8')
return candidate
} catch {}
}
return null
}
interface ImportInfo {
line: number
specifier: string
clause: string
}
/** Parse `import ... from '...'` statements, skipping side-effect-only imports. */
function parseImports(content: string): ImportInfo[] {
const lines = content.split('\n')
const imports: ImportInfo[] = []
const re = /^\s*import\s+([\s\S]*?)\s+from\s+['"]([^'"]+)['"]/
for (let i = 0; i < lines.length; i++) {
if (!/^\s*import\b/.test(lines[i]) || !lines[i].includes('import')) continue
// Join up to 12 following lines to capture multi-line import clauses.
const block = lines.slice(i, i + 12).join('\n')
const match = re.exec(block)
if (!match) continue
imports.push({ line: i + 1, clause: match[1], specifier: match[2] })
}
return imports
}
/** True when the import brings in at least one runtime VALUE (not purely types). */
function importsAValue(clause: string): boolean {
const trimmed = clause.trim()
if (trimmed.startsWith('type ')) return false // `import type { ... }` / `import type X`
const braceStart = trimmed.indexOf('{')
// A default or namespace binding outside the braces is always a value.
const beforeBrace = braceStart === -1 ? trimmed : trimmed.slice(0, braceStart)
if (beforeBrace.replace(/[,\s]/g, '').length > 0) return true
if (braceStart === -1) return true
const inner = trimmed.slice(braceStart + 1, trimmed.lastIndexOf('}'))
// A named import is a value unless every member is `type`-prefixed.
return inner
.split(',')
.map((s) => s.trim())
.filter(Boolean)
.some((member) => !member.startsWith('type '))
}
function hasAllowDirective(content: string, importLine: number): boolean {
const lines = content.split('\n')
for (let i = importLine - 2; i >= 0 && i >= importLine - 5; i--) {
const line = lines[i]?.trim() ?? ''
if (line === '' || line.startsWith('//') || line.startsWith('*') || line.startsWith('/*')) {
if (line.includes(ALLOW_DIRECTIVE)) {
const reason =
line
.split(ALLOW_DIRECTIVE)[1]
?.replace(/^[:\s]+/, '')
.trim() ?? ''
return reason.length > 0
}
continue
}
break
}
return false
}
interface Violation {
file: string
line: number
specifier: string
}
async function main() {
const checkMode = process.argv.includes('--check')
const allFiles = await listFiles(APP_DIR)
const violations: Violation[] = []
for (const absFile of allFiles) {
const rel = path.relative(APP_DIR, absFile)
if (!isServerSurface(rel)) continue
// A server file that is itself `'use client'` is a client component — out of scope.
if (await isUseClientModule(absFile)) continue
const content = await readFile(absFile, 'utf8')
for (const imp of parseImports(content)) {
if (!importsAValue(imp.clause)) continue
const resolved = await resolveSpecifier(imp.specifier, absFile)
if (!resolved) continue
if (!(await isUseClientModule(resolved))) continue
if (hasAllowDirective(content, imp.line)) continue
violations.push({ file: rel, line: imp.line, specifier: imp.specifier })
}
}
if (violations.length === 0) {
console.log(
"✓ Client-boundary import check passed (no server file imports a value from a 'use client' module)."
)
return
}
console.error(
`\n✗ ${violations.length} server file(s) import a runtime value from a 'use client' module.\n` +
` On the server these resolve to client-reference stubs and throw when called (e.g. 'X.list is not a function').\n` +
` Move the imported factory/fetcher/constant into a non-'use client' module (hooks/queries/utils/*-keys.ts or fetch-*.ts).\n` +
` See .claude/rules/sim-queries.md. Escape hatch: // ${ALLOW_DIRECTIVE}: <reason> above the import.\n`
)
for (const v of violations) {
console.error(` ${v.file}:${v.line} imports from '${v.specifier}'`)
}
if (checkMode) process.exit(1)
}
main().catch((error) => {
console.error(error)
process.exit(1)
})
+229
View File
@@ -0,0 +1,229 @@
#!/usr/bin/env bun
/**
* Validates that every `<path d='…'>` in `apps/sim/components/icons.tsx` is
* syntactically well-formed SVG path data.
*
* Malformed path data does not crash the build or fail TypeScript — the browser
* silently drops the bad segment and logs `<path> attribute d: Expected number`
* / `Expected arc flag` to the console. A bulk icon reformat once corrupted
* several brand icons this way (dropped arc-flag digits, mangled cubic operands
* like `c00,00,00`), flooding the integrations page with dozens of console
* errors that no existing check caught. This script is that missing gate.
*
* It walks each `d` string against the SVG path grammar and fails if any
* command has the wrong operand count or an arc flag that is not `0`/`1`.
* Number scanning handles the compact forms the spec allows (packed decimals
* `.5.5`, sign-delimited `1-2`, exponents, and arc flags packed against
* neighbours like `001.39`), so valid minified paths pass unflagged.
*
* Run: `bun run scripts/check-icon-paths.ts`
*/
import { readFile } from 'node:fs/promises'
import path from 'node:path'
const ROOT = path.resolve(import.meta.dir, '..')
/** Defaults to the shared icon module; overridable via argv for testing. */
const ICONS_FILE = process.argv[2]
? path.resolve(process.argv[2])
: path.join(ROOT, 'apps/sim/components/icons.tsx')
/** Operand count per path command; arc (`a`) is handled specially for flags. */
const OPERANDS: Record<string, number> = {
m: 2,
l: 2,
h: 1,
v: 1,
c: 6,
s: 4,
q: 4,
t: 2,
a: 7,
z: 0,
}
const isWsp = (ch: string) => ch === ' ' || ch === '\t' || ch === '\n' || ch === '\r' || ch === '\f'
/** Skip whitespace and comma separators; returns the next significant index. */
function skipSep(d: string, i: number): number {
while (i < d.length && (isWsp(d[i]) || d[i] === ',')) i++
return i
}
/**
* Scan one number starting at `i` (after separators are skipped). Returns the
* index past the number, or `-1` if no valid number begins here. Mirrors the
* SVG number grammar: optional sign, integer/fraction, optional exponent.
*/
function scanNumber(d: string, i: number): number {
const start = i
if (d[i] === '+' || d[i] === '-') i++
let digits = 0
while (i < d.length && d[i] >= '0' && d[i] <= '9') {
i++
digits++
}
if (d[i] === '.') {
i++
while (i < d.length && d[i] >= '0' && d[i] <= '9') {
i++
digits++
}
}
if (digits === 0) return -1
if (d[i] === 'e' || d[i] === 'E') {
let j = i + 1
if (d[j] === '+' || d[j] === '-') j++
let expDigits = 0
while (j < d.length && d[j] >= '0' && d[j] <= '9') {
j++
expDigits++
}
if (expDigits > 0) i = j
}
return i > start ? i : -1
}
interface PathError {
reason: string
/** 0-based offset into the `d` string where parsing failed. */
offset: number
}
/**
* Validate a single `d` string. Returns the first structural error, or `null`
* when the path is well-formed.
*/
function validatePath(d: string): PathError | null {
let i = skipSep(d, 0)
if (i >= d.length) return { reason: 'empty path data', offset: 0 }
let cmd = ''
// The first command must be a moveto.
if (d[i] !== 'M' && d[i] !== 'm') return { reason: 'path must start with M/m', offset: i }
while (i < d.length) {
i = skipSep(d, i)
if (i >= d.length) break
const ch = d[i]
if (/[a-zA-Z]/.test(ch)) {
if (!(ch.toLowerCase() in OPERANDS)) {
return { reason: `unknown command '${ch}'`, offset: i }
}
cmd = ch
i++
if (cmd === 'z' || cmd === 'Z') continue
} else if (cmd === '' || cmd === 'z' || cmd === 'Z') {
return { reason: `expected a command, found '${ch}'`, offset: i }
}
// After an explicit M/m, repeated operand groups are implicit L/l.
const effective = cmd === 'M' ? 'L' : cmd === 'm' ? 'l' : cmd
const key = effective.toLowerCase()
if (key === 'a') {
const err = scanArcGroup(d, i)
if (typeof err === 'string') return { reason: err, offset: i }
i = err
} else {
const count = OPERANDS[key]
for (let n = 0; n < count; n++) {
const before = skipSep(d, i)
const next = scanNumber(d, before)
if (next < 0) {
return { reason: `expected number for '${cmd}' command`, offset: before }
}
i = next
}
}
}
return null
}
/**
* Scan one 7-operand arc group: rx ry x-axis-rotation large-arc-flag
* sweep-flag x y. Flags are a single `0`/`1` that may be packed against the
* next token (e.g. `001.39`). Returns the next index, or an error string.
*/
function scanArcGroup(d: string, i: number): number | string {
for (let n = 0; n < 3; n++) {
const before = skipSep(d, i)
const next = scanNumber(d, before)
if (next < 0) return `expected number in arc command`
i = next
}
for (let n = 0; n < 2; n++) {
const before = skipSep(d, i)
if (d[before] !== '0' && d[before] !== '1') {
return `expected arc flag ('0' or '1')`
}
i = before + 1
}
for (let n = 0; n < 2; n++) {
const before = skipSep(d, i)
const next = scanNumber(d, before)
if (next < 0) return `expected number in arc command`
i = next
}
return i
}
interface Finding {
icon: string
line: number
reason: string
snippet: string
}
/**
* Find the nearest preceding icon export for a source offset. Matches both
* `export function XxxIcon` and `export const XxxIcon =` forms so arrow-function
* icons are attributed correctly.
*/
function iconNameAt(src: string, offset: number): string {
const before = src.slice(0, offset)
const matches = [...before.matchAll(/export (?:function|const) (\w+)\s*[=(]/g)]
return matches.length ? matches[matches.length - 1][1] : '<unknown>'
}
async function main() {
const src = await readFile(ICONS_FILE, 'utf8')
const findings: Finding[] = []
for (const m of src.matchAll(/\bd=(?:'([^']*)'|"([^"]*)")/g)) {
const d = m[1] ?? m[2] ?? ''
if (!d.trim()) continue
const err = validatePath(d)
if (!err) continue
// Offset of the `d` value within the file → the corrupt char.
const valueStart = m.index + m[0].indexOf(d)
const fileOffset = valueStart + err.offset
const line = src.slice(0, fileOffset).split('\n').length
const around = d.slice(Math.max(0, err.offset - 20), err.offset + 20)
findings.push({
icon: iconNameAt(src, m.index),
line,
reason: err.reason,
snippet: `${around}`,
})
}
if (findings.length === 0) {
console.log('✓ All icon <path> data in components/icons.tsx is valid SVG.')
process.exit(0)
}
console.error(`\nFound ${findings.length} malformed icon path(s) in components/icons.tsx:\n`)
for (const f of findings) {
console.error(` ${f.icon} (icons.tsx:${f.line}) — ${f.reason}`)
console.error(` near: ${f.snippet}`)
console.error(
' Malformed path data renders nothing and floods the console with SVG parse errors.'
)
console.error(' Fix the d attribute (correct operand counts; arc flags must be 0 or 1).\n')
}
process.exit(1)
}
main()
+200
View File
@@ -0,0 +1,200 @@
/**
* Run with: bun test scripts/check-migrations-safety.test.ts
* (Root scripts are bun-native and not part of the turbo/vitest workspaces.)
*/
import { describe, expect, test } from 'bun:test'
import { lintSql } from './check-migrations-safety.ts'
const rules = (sql: string) => lintSql(sql).map((f) => `${f.tier}:${f.rule}`)
describe('additive / safe', () => {
test('nullable add column passes', () => {
expect(lintSql('ALTER TABLE "webhook" ADD COLUMN "provider_config" json;')).toEqual([])
})
test('NOT NULL with DEFAULT passes', () => {
expect(lintSql('ALTER TABLE "user" ADD COLUMN "flag" boolean DEFAULT false NOT NULL;')).toEqual(
[]
)
})
test('CREATE TABLE plus index and FK on that new table passes', () => {
const sql = `CREATE TABLE "kb" ("id" text PRIMARY KEY NOT NULL, "user_id" text NOT NULL);
--> statement-breakpoint
CREATE INDEX "kb_user_id_idx" ON "kb" USING btree ("user_id");
--> statement-breakpoint
ALTER TABLE "kb" ADD CONSTRAINT "kb_user_fk" FOREIGN KEY ("user_id") REFERENCES "user"("id");`
expect(lintSql(sql)).toEqual([])
})
test('CONCURRENTLY index after a COMMIT breakpoint passes', () => {
const sql = `COMMIT;
--> statement-breakpoint
SET lock_timeout = 0;
--> statement-breakpoint
CREATE INDEX CONCURRENTLY IF NOT EXISTS "idx_x" ON "embedding" ("kb_id");`
expect(lintSql(sql)).toEqual([])
})
})
describe('hard errors', () => {
test('ADD COLUMN NOT NULL without default', () => {
expect(rules('ALTER TABLE "user" ADD COLUMN "email" text NOT NULL;')).toEqual([
'error:add-not-null-no-default',
])
})
test('RENAME column', () => {
expect(rules('ALTER TABLE "marketplace" RENAME COLUMN "executions" TO "views";')).toEqual([
'error:rename',
])
})
test('CREATE INDEX on existing table without CONCURRENTLY', () => {
expect(rules('CREATE INDEX "idx_y" ON "embedding" ("kb_id");')).toEqual([
'error:index-not-concurrent',
])
})
test('CONCURRENTLY index without IF NOT EXISTS', () => {
const sql = `COMMIT;
--> statement-breakpoint
CREATE INDEX CONCURRENTLY "idx_z" ON "embedding" ("kb_id");`
expect(rules(sql)).toEqual(['error:concurrent-index-not-idempotent'])
})
test('CONCURRENTLY index without a preceding COMMIT', () => {
expect(
rules('CREATE INDEX CONCURRENTLY IF NOT EXISTS "idx_z" ON "embedding" ("kb_id");')
).toEqual(['error:concurrent-index-no-commit'])
})
test('ADD FOREIGN KEY on existing table without NOT VALID', () => {
expect(
rules(
'ALTER TABLE "session" ADD CONSTRAINT "s_fk" FOREIGN KEY ("uid") REFERENCES "user"("id");'
)
).toEqual(['error:constraint-not-valid'])
})
})
describe('annotate tier', () => {
const drop = 'ALTER TABLE "webhook" DROP COLUMN "secret";'
test('DROP COLUMN unannotated fails', () => {
expect(rules(drop)).toEqual(['error:drop-column'])
})
test('DROP COLUMN annotated passes', () => {
const sql = `-- migration-safe: secret read removed in v0.6.1 (#1234), shipped two deploys ago\n${drop}`
expect(lintSql(sql)).toEqual([])
})
test('annotation tolerates an intervening statement-breakpoint line', () => {
const sql = `ALTER TABLE "webhook" ADD COLUMN "provider_config" json;
--> statement-breakpoint
-- migration-safe: secret read removed in v0.6.1 (#1234)
${drop}`
expect(lintSql(sql)).toEqual([])
})
test('dangling annotation with empty reason fails', () => {
const sql = `-- migration-safe:\n${drop}`
const found = lintSql(sql)
expect(found).toHaveLength(1)
expect(found[0].tier).toBe('error')
expect(found[0].message).toContain('no reason')
})
test('annotation on the wrong statement does not bleed', () => {
const sql = `-- migration-safe: removing secret
ALTER TABLE "webhook" ADD COLUMN "x" json;
--> statement-breakpoint
${drop}`
expect(rules(sql)).toEqual(['error:drop-column'])
})
test('type change and DROP TABLE are annotate-tier', () => {
expect(
rules(
'ALTER TABLE "user_table_rows" ALTER COLUMN "order_key" SET DATA TYPE text COLLATE "C";'
)
).toEqual(['error:alter-type'])
expect(rules('DROP TABLE "marketplace_execution" CASCADE;')).toEqual(['error:drop-table'])
})
})
describe('warnings (non-blocking)', () => {
test('UPDATE backfill warns but does not error', () => {
const found = lintSql('UPDATE "user_table_definitions" SET "schema" = \'{}\' WHERE id = \'1\';')
expect(found.map((f) => f.tier)).toEqual(['warn'])
})
test('UPDATE without WHERE flags the whole-table note', () => {
const found = lintSql('UPDATE "user" SET "active" = true;')
expect(found[0].tier).toBe('warn')
expect(found[0].message).toContain('no WHERE')
})
})
describe('review fixes', () => {
test('RENAME CONSTRAINT is metadata-only — not flagged', () => {
expect(
lintSql('ALTER TABLE "permission_group" RENAME CONSTRAINT "old_fk" TO "new_fk";')
).toEqual([])
})
test('ALTER INDEX ... RENAME is metadata-only — not flagged', () => {
expect(lintSql('ALTER INDEX "old_idx" RENAME TO "new_idx";')).toEqual([])
})
test('table RENAME TO is still a hard error', () => {
expect(rules('ALTER TABLE "marketplace" RENAME TO "listings";')).toEqual(['error:rename'])
})
test('plain DROP INDEX is a hard error (ACCESS EXCLUSIVE lock)', () => {
expect(rules('DROP INDEX "permission_group_workspace_name_unique";')).toEqual([
'error:drop-index-not-concurrent',
])
})
test('DROP INDEX CONCURRENTLY after a COMMIT passes clean', () => {
const sql = `COMMIT;
--> statement-breakpoint
DROP INDEX CONCURRENTLY IF EXISTS "stale_idx";`
expect(lintSql(sql)).toEqual([])
})
test('DROP INDEX CONCURRENTLY without IF EXISTS is not idempotent', () => {
const sql = `COMMIT;
--> statement-breakpoint
DROP INDEX CONCURRENTLY "stale_idx";`
expect(rules(sql)).toEqual(['error:concurrent-drop-index-not-idempotent'])
})
test('DROP INDEX CONCURRENTLY without a preceding COMMIT errors', () => {
expect(rules('DROP INDEX CONCURRENTLY IF EXISTS "stale_idx";')).toEqual([
'error:concurrent-drop-index-no-commit',
])
})
test('alter-type does not match TYPE inside a string default', () => {
expect(lintSql(`ALTER TABLE "x" ALTER COLUMN "y" SET DEFAULT 'change TYPE later';`)).toEqual([])
})
})
describe('parser robustness', () => {
test('semicolon inside a string literal does not split', () => {
expect(lintSql(`ALTER TABLE "x" ADD COLUMN "y" text DEFAULT 'a;b' NOT NULL;`)).toEqual([])
})
test('dollar-quoted DO block is one statement; FK on a new table is suppressed', () => {
const sql = `CREATE TABLE "jobs" ("id" text PRIMARY KEY NOT NULL, "wid" text NOT NULL);
--> statement-breakpoint
DO $$ BEGIN
ALTER TABLE "jobs" ADD CONSTRAINT "jobs_fk" FOREIGN KEY ("wid") REFERENCES "workspace"("id");
EXCEPTION WHEN duplicate_object THEN null;
END $$;`
expect(lintSql(sql)).toEqual([])
})
})
+491
View File
@@ -0,0 +1,491 @@
#!/usr/bin/env bun
/**
* Guards new Drizzle migrations against deploy-window downtime.
*
* During a deploy the previously-deployed app code keeps serving against the
* freshly-migrated schema (blue/green keeps both versions live). A migration
* that is backward-incompatible with that older code — drops a column it still
* reads, renames, adds a NOT NULL its inserts don't populate — throws until the
* new code takes over. The fix is the expand/contract discipline: additive now,
* destructive only after the dependent code is gone.
*
* This lint is the deterministic half of that guard (the `/db-migrate` skill is
* the judgment half). It classifies every statement in migrations added on this
* branch:
* - HARD ERROR: ops that are essentially never one-deploy-safe. Rewrite them.
* - ANNOTATE: legitimate contract-phase ops. Acknowledge each with a
* `-- migration-safe: <reason>` comment on the preceding line(s),
* only after confirming the dependent code already shipped out.
* - WARN: data backfills — surfaced for review, never block.
*
* Scope is new migration files only (git diff vs base); the existing corpus is
* grandfathered. Usage:
* bun run scripts/check-migrations-safety.ts [baseRef] # base defaults to origin/staging
* bun run scripts/check-migrations-safety.ts --all # whole corpus
* bun run scripts/check-migrations-safety.ts --dir <path> # a directory
*/
import { execFileSync } from 'node:child_process'
import { readdir, readFile } from 'node:fs/promises'
import path from 'node:path'
const ROOT = path.resolve(import.meta.dir, '..')
const MIGRATIONS_DIR = 'packages/db/migrations'
const ANNOTATION_PREFIX = '-- migration-safe:'
type Tier = 'error' | 'warn'
interface Finding {
line: number
statement: string
tier: Tier
rule: string
message: string
}
interface Statement {
sql: string
startLine: number
}
/** Strip quotes and any schema prefix so `"public"."user"` and `"user"` match. */
function bareName(raw: string): string {
const unquoted = raw.replace(/"/g, '')
const parts = unquoted.split('.')
return (parts[parts.length - 1] ?? unquoted).toLowerCase()
}
/**
* Split SQL into statements with their 1-based start line, respecting line
* comments (`--`), block comments, and single-quoted strings so a `;` inside
* any of them does not terminate a statement.
*/
function parseStatements(content: string): Statement[] {
const statements: Statement[] = []
let buf = ''
let startOffset = -1
let inLine = false
let inBlock = false
let inStr = false
let dollarTag: string | null = null
const lineAt = (offset: number): number => {
let line = 1
for (let i = 0; i < offset; i++) if (content[i] === '\n') line++
return line
}
const flush = () => {
const sql = buf.trim()
if (sql.length > 0 && startOffset >= 0) statements.push({ sql, startLine: lineAt(startOffset) })
buf = ''
startOffset = -1
}
for (let i = 0; i < content.length; i++) {
const c = content[i]
const next = content[i + 1]
if (inLine) {
if (c === '\n') inLine = false
continue
}
if (inBlock) {
if (c === '*' && next === '/') {
inBlock = false
i++
}
continue
}
if (inStr) {
buf += c
if (c === "'") {
if (next === "'") {
buf += "'"
i++
} else {
inStr = false
}
}
continue
}
if (dollarTag) {
if (c === '$' && content.startsWith(dollarTag, i)) {
buf += dollarTag
i += dollarTag.length - 1
dollarTag = null
} else {
buf += c
}
continue
}
if (c === '$') {
const tag = /^\$[A-Za-z_]*\$/.exec(content.slice(i))?.[0]
if (tag) {
if (startOffset < 0) startOffset = i
dollarTag = tag
buf += tag
i += tag.length - 1
continue
}
}
if (c === '-' && next === '-') {
inLine = true
i++
continue
}
if (c === '/' && next === '*') {
inBlock = true
i++
continue
}
if (c === "'") {
inStr = true
if (startOffset < 0) startOffset = i
buf += c
continue
}
if (c === ';') {
flush()
continue
}
if (startOffset < 0 && !/\s/.test(c)) startOffset = i
buf += c
}
flush()
return statements
}
/**
* Mirror of the api-validation annotation reader, for `--` SQL comments:
* scans up to three consecutive non-empty preceding lines for the prefix.
* `allowed` when a non-empty reason follows; `missingReason` flags a dangling
* annotation so it fails rather than silently passing.
*/
function readAnnotation(
lines: string[],
startLine: number
): { allowed: boolean; missingReason: boolean } {
let inspected = 0
for (let i = startLine - 2; i >= 0 && inspected < 3; i--) {
const trimmed = lines[i]?.trim() ?? ''
if (trimmed.length === 0) continue
inspected++
if (!trimmed.startsWith('--')) return { allowed: false, missingReason: false }
const idx = trimmed.indexOf(ANNOTATION_PREFIX)
if (idx === -1) continue
const reason = trimmed.slice(idx + ANNOTATION_PREFIX.length).trim()
if (reason.length === 0) return { allowed: false, missingReason: true }
return { allowed: true, missingReason: false }
}
return { allowed: false, missingReason: false }
}
interface RawMatch {
kind: 'error' | 'annotate' | 'warn'
rule: string
message: string
}
/**
* Classify one statement. `createdTables` holds tables created in the same
* migration — ops against a brand-new table have no old rows and no live
* traffic, so they are always safe and skipped. `sawCommit` tracks whether a
* `COMMIT;` breakpoint preceded a CONCURRENTLY index (see migrate.ts).
*/
function classify(sql: string, createdTables: Set<string>, sawCommit: boolean): RawMatch[] {
const s = sql.replace(/\s+/g, ' ').trim()
const matches: RawMatch[] = []
const alterTable = s.match(/\bALTER TABLE (?:IF EXISTS )?(?:ONLY )?("?[.\w]+"?)/i)
const targetTable = alterTable ? bareName(alterTable[1]) : null
const onNewTable = targetTable !== null && createdTables.has(targetTable)
if (/^CREATE (?:UNIQUE )?INDEX\b/i.test(s)) {
const on = s.match(/\bON ("?[.\w]+"?)/i)
const indexTable = on ? bareName(on[1]) : null
const concurrent = /\bCONCURRENTLY\b/i.test(s)
if (!(indexTable && createdTables.has(indexTable))) {
if (!concurrent) {
matches.push({
kind: 'error',
rule: 'index-not-concurrent',
message:
'CREATE INDEX on an existing table write-locks it for the whole build. Use CREATE INDEX CONCURRENTLY IF NOT EXISTS after a COMMIT; breakpoint (see packages/db/scripts/migrate.ts).',
})
} else if (!/\bIF NOT EXISTS\b/i.test(s)) {
matches.push({
kind: 'error',
rule: 'concurrent-index-not-idempotent',
message:
'CREATE INDEX CONCURRENTLY must be IF NOT EXISTS — a failed build replays from the top and a partial INVALID index would be skipped forever.',
})
} else if (!sawCommit) {
matches.push({
kind: 'error',
rule: 'concurrent-index-no-commit',
message:
'CREATE INDEX CONCURRENTLY cannot run inside the migration transaction. Precede it with a COMMIT; breakpoint and SET lock_timeout = 0 (see packages/db/scripts/migrate.ts).',
})
}
}
}
if (
!onNewTable &&
/\bADD COLUMN\b/i.test(s) &&
/\bNOT NULL\b/i.test(s) &&
!/\bDEFAULT\b/i.test(s)
) {
matches.push({
kind: 'error',
rule: 'add-not-null-no-default',
message:
'ADD COLUMN NOT NULL with no DEFAULT breaks old inserts (and fails on existing rows). Add it nullable or with a DEFAULT, backfill, then SET NOT NULL in a later migration once code populates it.',
})
}
if (/\bRENAME COLUMN\b/i.test(s) || /^ALTER TABLE\b[^;]*\bRENAME TO\b/i.test(s)) {
matches.push({
kind: 'error',
rule: 'rename',
message:
'RENAME of a column/table breaks old code reading the old name. Add the new column/table, dual-write in code, then drop the old one in a later deploy.',
})
}
if (
!onNewTable &&
/\bADD CONSTRAINT\b/i.test(s) &&
/\b(FOREIGN KEY|CHECK)\b/i.test(s) &&
!/\bNOT VALID\b/i.test(s)
) {
matches.push({
kind: 'error',
rule: 'constraint-not-valid',
message:
'ADD CONSTRAINT FOREIGN KEY/CHECK on an existing table locks it and rejects old writes that violate it. Add it NOT VALID, then VALIDATE CONSTRAINT in a separate step.',
})
}
if (!onNewTable) {
if (/^DROP TABLE\b/i.test(s)) {
matches.push({ kind: 'annotate', rule: 'drop-table', message: 'DROP TABLE' })
}
if (/\bDROP COLUMN\b/i.test(s)) {
matches.push({ kind: 'annotate', rule: 'drop-column', message: 'DROP COLUMN' })
}
if (/\bDROP CONSTRAINT\b/i.test(s)) {
matches.push({ kind: 'annotate', rule: 'drop-constraint', message: 'DROP CONSTRAINT' })
}
if (/\bDROP DEFAULT\b/i.test(s)) {
matches.push({ kind: 'annotate', rule: 'drop-default', message: 'DROP DEFAULT' })
}
if (/\bSET NOT NULL\b/i.test(s)) {
matches.push({ kind: 'annotate', rule: 'set-not-null', message: 'SET NOT NULL' })
}
if (/\bSET DATA TYPE\b/i.test(s) || /\bALTER COLUMN ("?[.\w]+"?) TYPE\b/i.test(s)) {
matches.push({ kind: 'annotate', rule: 'alter-type', message: 'column type change' })
}
}
if (/^DROP INDEX\b/i.test(s)) {
if (!/\bCONCURRENTLY\b/i.test(s)) {
matches.push({
kind: 'error',
rule: 'drop-index-not-concurrent',
message:
'Plain DROP INDEX takes an ACCESS EXCLUSIVE lock on the table for the whole drop. Use DROP INDEX CONCURRENTLY after a COMMIT; breakpoint (see packages/db/scripts/migrate.ts).',
})
} else if (!/\bIF EXISTS\b/i.test(s)) {
matches.push({
kind: 'error',
rule: 'concurrent-drop-index-not-idempotent',
message:
'DROP INDEX CONCURRENTLY must be IF EXISTS — a failed run replays from the top and would abort re-dropping an already-gone index.',
})
} else if (!sawCommit) {
matches.push({
kind: 'error',
rule: 'concurrent-drop-index-no-commit',
message:
'DROP INDEX CONCURRENTLY cannot run inside the migration transaction. Precede it with a COMMIT; breakpoint (see packages/db/scripts/migrate.ts).',
})
}
}
if (/^(UPDATE|DELETE)\b/i.test(s)) {
const noWhere = !/\bWHERE\b/i.test(s)
matches.push({
kind: 'warn',
rule: 'data-backfill',
message: noWhere
? 'data backfill with no WHERE rewrites/locks the whole table. Confirm it is batched, idempotent, and safe under concurrent writes.'
: 'data backfill. Confirm it is batched, idempotent, and safe under concurrent writes.',
})
}
return matches
}
const ANNOTATE_GUIDANCE =
'is a contract-phase op. Confirm the old code no longer reads/writes it (it must have shipped in an earlier deploy — not this same PR), then acknowledge with a `-- migration-safe: <reason>` comment on the line above.'
/** Lint a single migration's SQL. Returns only actionable findings. */
export function lintSql(content: string): Finding[] {
const lines = content.split('\n')
const statements = parseStatements(content)
const createdTables = new Set<string>()
for (const { sql } of statements) {
const m = sql.match(/^CREATE TABLE (?:IF NOT EXISTS )?("?[.\w]+"?)/i)
if (m) createdTables.add(bareName(m[1]))
}
const findings: Finding[] = []
let sawCommit = false
for (const { sql, startLine } of statements) {
for (const match of classify(sql, createdTables, sawCommit)) {
if (match.kind === 'error') {
findings.push({
line: startLine,
statement: sql,
tier: 'error',
rule: match.rule,
message: match.message,
})
} else if (match.kind === 'warn') {
findings.push({
line: startLine,
statement: sql,
tier: 'warn',
rule: match.rule,
message: match.message,
})
} else {
const ann = readAnnotation(lines, startLine)
if (ann.allowed) continue
findings.push({
line: startLine,
statement: sql,
tier: 'error',
rule: match.rule,
message: ann.missingReason
? `${match.message}: \`-- migration-safe:\` annotation has no reason. Give it a real justification.`
: `${match.message} ${ANNOTATE_GUIDANCE}`,
})
}
}
if (/^COMMIT\b/i.test(sql.trim())) sawCommit = true
}
return findings
}
function git(args: string[]): string | null {
try {
return execFileSync('git', args, {
cwd: ROOT,
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'ignore'],
}).trim()
} catch {
return null
}
}
/** New migration files on this branch vs base, plus uncommitted ones locally. */
function changedMigrationFiles(baseRef: string): string[] {
const files = new Set<string>()
const inDir = (p: string) => p.startsWith(`${MIGRATIONS_DIR}/`) && p.endsWith('.sql')
const mergeBase = git(['merge-base', baseRef, 'HEAD']) ?? baseRef
const committed = git([
'diff',
'--name-only',
'--diff-filter=AM',
mergeBase,
'HEAD',
'--',
MIGRATIONS_DIR,
])
if (committed === null) return [] // git unavailable → fail open (handled by caller)
for (const f of committed.split('\n')) if (inDir(f)) files.add(f)
const status = git(['status', '--porcelain', '--', MIGRATIONS_DIR])
if (status) {
for (const raw of status.split('\n')) {
const p = raw.slice(3).trim()
if (inDir(p)) files.add(p)
}
}
return [...files]
}
async function listSqlFiles(dir: string): Promise<string[]> {
const out: string[] = []
const entries = await readdir(dir, { withFileTypes: true })
for (const e of entries) {
const full = path.join(dir, e.name)
if (e.isDirectory()) out.push(...(await listSqlFiles(full)))
else if (e.name.endsWith('.sql')) out.push(full)
}
return out
}
async function resolveFiles(argv: string[]): Promise<string[] | null> {
if (argv.includes('--all')) {
return (await listSqlFiles(path.join(ROOT, MIGRATIONS_DIR))).map((f) => path.relative(ROOT, f))
}
const dirIdx = argv.indexOf('--dir')
if (dirIdx !== -1) {
const dir = argv[dirIdx + 1]
if (!dir) throw new Error('--dir requires a path')
return (await listSqlFiles(path.resolve(dir))).map((f) => path.relative(ROOT, f))
}
const baseRef = argv.find((a) => !a.startsWith('--')) ?? 'origin/staging'
const files = changedMigrationFiles(baseRef)
if (files.length === 0 && git(['rev-parse', 'HEAD']) === null) {
console.warn('⚠ git unavailable — skipping migration safety check.')
return null
}
return files
}
async function main() {
const files = await resolveFiles(process.argv.slice(2))
if (files === null) process.exit(0)
if (files.length === 0) {
console.log('✓ No new migrations to check.')
process.exit(0)
}
let errors = 0
let warnings = 0
for (const rel of files) {
const content = await readFile(path.join(ROOT, rel), 'utf8')
const findings = lintSql(content)
if (findings.length === 0) continue
console.error(`\n${rel}`)
for (const f of findings.sort((a, b) => a.line - b.line)) {
const icon = f.tier === 'error' ? '✗' : '⚠'
if (f.tier === 'error') errors++
else warnings++
console.error(` ${icon} ${rel}:${f.line} [${f.rule}]`)
console.error(` ${f.statement.replace(/\s+/g, ' ').slice(0, 120)}`)
console.error(`${f.message}`)
}
}
if (errors === 0) {
console.log(
warnings > 0
? `\n✓ No blocking migration issues (${warnings} warning(s) to review).`
: '\n✓ Migrations are backward-compatible.'
)
process.exit(0)
}
console.error(
`\nFound ${errors} blocking migration issue(s). Rewrite hard errors into expand/contract, or annotate contract ops once safe. See the /db-migrate skill.`
)
process.exit(1)
}
if (import.meta.main) main()
+77
View File
@@ -0,0 +1,77 @@
#!/usr/bin/env bun
import { readdir, readFile } from 'node:fs/promises'
import path from 'node:path'
const ROOT = path.resolve(import.meta.dir, '..')
const PACKAGES_DIR = path.join(ROOT, 'packages')
const FORBIDDEN_PATTERNS: Array<{ pattern: RegExp; description: string }> = [
{ pattern: /from\s+['"]@\/(?!\*)/g, description: "'@/' path alias (apps/sim-only)" },
{ pattern: /from\s+['"]\.\.\/\.\.\/apps\//g, description: 'relative import into apps/' },
{ pattern: /from\s+['"]apps\//g, description: "bare 'apps/' import" },
]
const SKIP_DIRS = new Set(['node_modules', 'dist', '.next', '.turbo', 'coverage'])
async function walk(dir: string, results: string[] = []): Promise<string[]> {
const entries = await readdir(dir, { withFileTypes: true })
for (const entry of entries) {
if (SKIP_DIRS.has(entry.name)) continue
const full = path.join(dir, entry.name)
if (entry.isDirectory()) {
await walk(full, results)
} else if (/\.(ts|tsx|mts|cts|js|jsx|mjs|cjs)$/.test(entry.name)) {
results.push(full)
}
}
return results
}
async function main() {
const packagesEntries = await readdir(PACKAGES_DIR, { withFileTypes: true })
const packageDirs = packagesEntries
.filter((entry) => entry.isDirectory())
.map((entry) => path.join(PACKAGES_DIR, entry.name))
const offenders: Array<{ file: string; line: number; description: string; snippet: string }> = []
for (const dir of packageDirs) {
const files = await walk(dir)
for (const file of files) {
const content = await readFile(file, 'utf8')
const lines = content.split('\n')
for (let i = 0; i < lines.length; i++) {
const line = lines[i]
for (const { pattern, description } of FORBIDDEN_PATTERNS) {
pattern.lastIndex = 0
if (pattern.test(line)) {
offenders.push({
file: path.relative(ROOT, file),
line: i + 1,
description,
snippet: line.trim(),
})
}
}
}
}
}
if (offenders.length === 0) {
console.log('✅ Monorepo boundaries OK: no package imports from apps/*')
return
}
console.error('❌ Monorepo boundary violations found:')
for (const offender of offenders) {
console.error(
` ${offender.file}:${offender.line}${offender.description}\n ${offender.snippet}`
)
}
process.exit(1)
}
void main().catch((error) => {
console.error('Monorepo boundary check failed:', error)
process.exit(1)
})
@@ -0,0 +1,4 @@
{
"generatedFrom": "apps/sim (non-strict zone)",
"counts": {}
}
+411
View File
@@ -0,0 +1,411 @@
#!/usr/bin/env bun
/**
* Enforces the project's React Query (TanStack Query v5) conventions.
*
* Biome and `check-api-validation-contracts.ts` cover formatting and the
* raw-fetch / contract boundary. This script catches React-Query-specific
* anti-patterns that neither covers:
*
* 1. missing-stale-time — useQuery/useInfiniteQuery/useSuspenseQuery without an explicit `staleTime`
* 2. queryfn-no-signal — an inline `queryFn` that takes no args (cannot forward the AbortSignal)
* 3. inline-query-key — `queryKey: ['literal', ...]` instead of a colocated key factory
* 4. key-factory-no-root — a `*Keys` factory in hooks/queries/** without an `all` root key
* 5. key-fetch-arg-drift — an identifier the queryFn forwards into the fetch (e.g. `workspaceId`)
* that is absent from the queryKey, so distinct fetch args collide on one
* cache entry. Conservative: only bare camelCase identifiers (never the
* requestJson contract arg, PascalCase/SCREAMING constants, or signal/
* pageParam machinery) are checked, and only when both a queryKey and an
* inline-arrow queryFn with a recognizable call are present.
*
* Enforcement model (mirrors check-api-validation-contracts.ts):
* - STRICT ZONE (apps/sim/hooks/queries/**): zero tolerance — any violation fails.
* - Elsewhere under apps/sim/**: ratcheted against scripts/check-react-query-patterns.baseline.json
* (fails only when a category's count rises above the recorded baseline).
*
* Escape hatch: put `// rq-lint-allow: <reason>` on the line directly above the
* flagged construct (up to 3 preceding comment lines tolerated). The reason must
* be non-empty.
*
* Usage:
* bun run scripts/check-react-query-patterns.ts # report
* bun run scripts/check-react-query-patterns.ts --check # CI gate (strict zone + ratchet)
* bun run scripts/check-react-query-patterns.ts --update-baseline
*/
import { readdir, readFile, writeFile } from 'node:fs/promises'
import path from 'node:path'
const ROOT = path.resolve(import.meta.dir, '..')
const APP_DIR = path.join(ROOT, 'apps/sim')
const BASELINE_PATH = path.join(ROOT, 'scripts/check-react-query-patterns.baseline.json')
const SKIP_DIRS = new Set(['node_modules', '.next', '.turbo', 'coverage', 'dist', 'build'])
const STRICT_PREFIX = 'apps/sim/hooks/queries/'
const ALLOW = 'rq-lint-allow:'
type Category =
| 'missing-stale-time'
| 'queryfn-no-signal'
| 'inline-query-key'
| 'key-factory-no-root'
| 'key-fetch-arg-drift'
interface Violation {
file: string
line: number
category: Category
message: string
snippet: string
}
const SUGGESTION: Record<Category, string> = {
'missing-stale-time':
'add an explicit staleTime (default 0 is rarely correct); e.g. staleTime: 60 * 1000',
'queryfn-no-signal':
'destructure the AbortSignal: queryFn: ({ signal }) => fetchX(..., signal) and forward it',
'inline-query-key':
'use a colocated hierarchical key factory (entityKeys.list(id)) instead of an inline literal key',
'key-factory-no-root':
'every *Keys factory in hooks/queries/** must expose an `all` root key for prefix invalidation',
'key-fetch-arg-drift':
'every identifier the queryFn passes to fetch/requestJson must also appear in the queryKey — ' +
'otherwise distinct fetch args collide on one cache entry (cross-tenant/param cache collision)',
}
async function walk(dir: string, out: string[] = []): Promise<string[]> {
let entries
try {
entries = await readdir(dir, { withFileTypes: true })
} catch {
return out
}
for (const entry of entries) {
if (SKIP_DIRS.has(entry.name)) continue
const full = path.join(dir, entry.name)
if (entry.isDirectory()) await walk(full, out)
else if (/\.(ts|tsx)$/.test(entry.name) && !/\.(test|spec)\.(ts|tsx)$/.test(entry.name))
out.push(full)
}
return out
}
/** Returns the substring of `src` for the balanced bracket group starting at `open` (index of the opening bracket). */
function matchBalanced(src: string, open: number, openCh: string, closeCh: string): string {
let depth = 0
let inStr: string | null = null
for (let i = open; i < src.length; i++) {
const ch = src[i]
const prev = src[i - 1]
if (inStr) {
if (ch === inStr && prev !== '\\') inStr = null
continue
}
if (ch === '"' || ch === "'" || ch === '`') {
inStr = ch
continue
}
if (ch === openCh) depth++
else if (ch === closeCh) {
depth--
if (depth === 0) return src.slice(open, i + 1)
}
}
return src.slice(open)
}
function lineAt(content: string, index: number): number {
let line = 1
for (let i = 0; i < index && i < content.length; i++) if (content[i] === '\n') line++
return line
}
/** True if a `// rq-lint-allow: <reason>` annotation sits within the 3 lines above `line` (1-based). */
function hasAllow(lines: string[], line: number): boolean {
for (let i = line - 2; i >= 0 && i >= line - 5; i--) {
const text = lines[i]?.trim() ?? ''
if (text.includes(ALLOW)) {
const reason = text.slice(text.indexOf(ALLOW) + ALLOW.length).trim()
return reason.length > 0
}
if (text.length > 0 && !text.startsWith('//') && !text.startsWith('*')) break
}
return false
}
const QUERY_CALL = /\b(useQuery|useInfiniteQuery|useSuspenseQuery|useSuspenseInfiniteQuery)\s*\(/g
const QUERYFN_NOARG = /queryFn\s*:\s*(?:async\s+)?\(\s*\)\s*=>/
const QUERYFN_PRESENT = /queryFn\s*:/
const INLINE_KEY = /queryKey\s*:\s*\[\s*[`'"]/
const KEYS_FACTORY = /\b(?:export\s+)?const\s+\w*[kK]eys\s*[:=][^=]*?=?\s*\{/g
/**
* Identifiers that are part of the queryFn machinery (not fetch params) and
* must never be flagged as drift even though they appear in the call.
*/
const QUERYFN_NOISE = new Set([
'signal',
'pageParam',
'meta',
'queryKey',
'direction',
'client',
'true',
'false',
'null',
'undefined',
])
/**
* Extracts the value slice of `key: ...` from an options object, reading up to
* the property-terminating top-level comma. Nested brackets (arrays, calls, and
* arrow-function bodies including their own param parens) are skipped via depth
* tracking, so this correctly returns the whole `({ signal }) => fetchX(...)`
* arrow for `queryFn`, not just its parameter list.
*/
function extractOptionValue(obj: string, key: string): string | null {
const re = new RegExp(`\\b${key}\\s*:`, 'g')
const m = re.exec(obj)
if (!m) return null
let i = m.index + m[0].length
while (i < obj.length && /\s/.test(obj[i])) i++
let depth = 0
let inStr: string | null = null
let j = i
for (; j < obj.length; j++) {
const c = obj[j]
const prev = obj[j - 1]
if (inStr) {
if (c === inStr && prev !== '\\') inStr = null
continue
}
if (c === '"' || c === "'" || c === '`') {
inStr = c
continue
}
if (c === '(' || c === '[' || c === '{') depth++
else if (c === ')' || c === ']' || c === '}') {
if (depth === 0) break
depth--
} else if (c === ',' && depth === 0) break
}
return obj.slice(i, j)
}
/**
* "key/fetch-arg drift": an identifier the queryFn forwards into the fetch
* (the args of the first call expression inside the queryFn body) that does NOT
* textually appear in the queryKey. Conservative — only fires when both the
* queryKey and an inline-arrow queryFn with a recognizable call are present, and
* only for bare identifiers (optionally `x as T` / `x!`), never member access,
* literals, or queryFn machinery (signal/pageParam/etc.).
*/
function findKeyFetchArgDrift(obj: string): string[] {
const keyExpr = extractOptionValue(obj, 'queryKey')
const fnExpr = extractOptionValue(obj, 'queryFn')
if (!keyExpr || !fnExpr) return []
// Drop the arrow prefix (`async`, `(params) =>`, optional `{ return`) so the
// search begins at the queryFn body, then locate the first call expression and
// balance its argument list.
const bodyStart = /=>\s*(?:\{\s*(?:return\s+)?)?/.exec(fnExpr)
const body = bodyStart ? fnExpr.slice(bodyStart.index + bodyStart[0].length) : fnExpr
const call = /(?:await\s+)?([A-Za-z_$][\w$.]*)\s*\(/.exec(body)
if (!call) return []
const callee = call[1]
const openParen = call.index + call[0].length - 1
const argList = matchBalanced(body, openParen, '(', ')')
// Split top-level args.
const args: string[] = []
let depth = 0
let cur = ''
for (let i = 1; i < argList.length - 1; i++) {
const c = argList[i]
if (c === '(' || c === '[' || c === '{') depth++
else if (c === ')' || c === ']' || c === '}') depth--
if (c === ',' && depth === 0) {
args.push(cur)
cur = ''
} else cur += c
}
if (cur.trim()) args.push(cur)
// `requestJson(contract, ...)` / `ensureQueryData(contract, ...)` etc. take the
// contract constant as their first arg — that is module-level config, never a
// per-hook identifier, so drop it before checking.
const dropsFirstArg = /(?:^|\.)(requestJson|requestText|ensureQueryData|fetchQuery)$/.test(callee)
const checkArgs = dropsFirstArg ? args.slice(1) : args
const drifted: string[] = []
for (const raw of checkArgs) {
const id = raw
.trim()
.replace(/\s+as\s+[\w$.<>[\]| ]+$/, '')
.replace(/!+$/, '')
.trim()
// Only bare identifiers; skip member access, calls, literals, destructures, spreads.
if (!/^[A-Za-z_$][\w$]*$/.test(id)) continue
if (QUERYFN_NOISE.has(id)) continue
// Skip module-level constants (contracts/config), which are camelCase config
// ending in `Contract`, PascalCase, or SCREAMING_SNAKE — never hook params.
if (/Contract$/.test(id) || /^[A-Z]/.test(id) || /^[A-Z0-9_]+$/.test(id)) continue
// Present in the key (as a whole word) → no drift.
const inKey = new RegExp(`\\b${id}\\b`).test(keyExpr)
if (!inKey && !drifted.includes(id)) drifted.push(id)
}
return drifted
}
function scanFile(rel: string, content: string): Violation[] {
const lines = content.split('\n')
const violations: Violation[] = []
const add = (index: number, category: Category, snippet: string) => {
const line = lineAt(content, index)
if (hasAllow(lines, line)) return
violations.push({ file: rel, line, category, message: SUGGESTION[category], snippet })
}
// 1 & 2: query call objects — staleTime + queryFn signal
QUERY_CALL.lastIndex = 0
let m: RegExpExecArray | null = QUERY_CALL.exec(content)
for (; m !== null; m = QUERY_CALL.exec(content)) {
const parenStart = m.index + m[0].length - 1
const arg = matchBalanced(content, parenStart, '(', ')')
// Only inspect the literal options object form `useQuery({ ... })`.
const braceRel = arg.indexOf('{')
if (braceRel === -1) continue
const obj = matchBalanced(arg, braceRel, '{', '}')
// Accept both `staleTime: ...` and the shorthand `staleTime,`; skip objects that spread options (...opts).
if (!/\bstaleTime\b/.test(obj) && !/\.\.\.\w/.test(obj)) {
add(m.index, 'missing-stale-time', `${m[1]}({ ... }) without staleTime`)
}
if (QUERYFN_PRESENT.test(obj) && QUERYFN_NOARG.test(obj)) {
add(m.index, 'queryfn-no-signal', `${m[1]} queryFn takes no args`)
}
for (const id of findKeyFetchArgDrift(obj)) {
add(
m.index,
'key-fetch-arg-drift',
`${m[1]}: '${id}' passed to fetch but absent from queryKey`
)
}
}
// 3: inline query keys
for (let i = 0; i < lines.length; i++) {
if (INLINE_KEY.test(lines[i])) {
const line = i + 1
if (!hasAllow(lines, line)) {
violations.push({
file: rel,
line,
category: 'inline-query-key',
message: SUGGESTION['inline-query-key'],
snippet: lines[i].trim(),
})
}
}
}
// 4: key factory must have an `all` root (hooks/queries/** only, excluding util key files that compose others)
if (rel.startsWith(STRICT_PREFIX)) {
KEYS_FACTORY.lastIndex = 0
let k: RegExpExecArray | null = KEYS_FACTORY.exec(content)
for (; k !== null; k = KEYS_FACTORY.exec(content)) {
const braceIdx = content.indexOf('{', k.index)
if (braceIdx === -1) continue
const obj = matchBalanced(content, braceIdx, '{', '}')
if (!/\ball\s*:/.test(obj)) {
add(k.index, 'key-factory-no-root', 'key factory missing `all` root key')
}
}
}
return violations
}
interface Baseline {
generatedFrom: string
counts: Record<string, number>
}
async function loadBaseline(): Promise<Baseline> {
try {
return JSON.parse(await readFile(BASELINE_PATH, 'utf8'))
} catch {
return { generatedFrom: 'none', counts: {} }
}
}
async function main() {
const update = process.argv.includes('--update-baseline')
const check = process.argv.includes('--check')
const files = await walk(APP_DIR)
const all: Violation[] = []
for (const file of files) {
const rel = path.relative(ROOT, file)
const content = await readFile(file, 'utf8')
if (!/\buse(Query|InfiniteQuery|SuspenseQuery|Mutation)\b|[kK]eys\s*[:=]/.test(content))
continue
all.push(...scanFile(rel, content))
}
const strict = all.filter((v) => v.file.startsWith(STRICT_PREFIX))
const ratchet = all.filter((v) => !v.file.startsWith(STRICT_PREFIX))
const counts: Record<string, number> = {}
for (const v of ratchet) counts[v.category] = (counts[v.category] ?? 0) + 1
if (update) {
const baseline: Baseline = { generatedFrom: 'apps/sim (non-strict zone)', counts }
await writeFile(BASELINE_PATH, `${JSON.stringify(baseline, null, 2)}\n`)
console.log(`✓ Baseline written: ${JSON.stringify(counts)}`)
process.exit(0)
}
console.log(`React Query pattern audit — scanned ${files.length} files`)
console.log(` strict zone (${STRICT_PREFIX}**) violations: ${strict.length}`)
console.log(` ratchet zone violations: ${ratchet.length} ${JSON.stringify(counts)}`)
let failed = false
if (strict.length > 0) {
failed = true
console.error(`\n✗ ${strict.length} violation(s) in the strict zone (${STRICT_PREFIX}**):\n`)
for (const v of strict) {
console.error(` ${v.file}:${v.line} [${v.category}]`)
console.error(` ${v.snippet}`)
console.error(`${v.message}\n`)
}
}
if (!check && ratchet.length > 0) {
console.error(`\nRatchet-zone occurrences (not failing without --check):`)
for (const v of ratchet) {
console.error(` ${v.file}:${v.line} [${v.category}] ${v.snippet}`)
}
}
if (check) {
const baseline = await loadBaseline()
for (const [category, count] of Object.entries(counts)) {
const base = baseline.counts[category] ?? 0
if (count > base) {
failed = true
console.error(
`\n✗ ratchet regression: ${category} rose to ${count} (baseline ${base}). ` +
`Fix the new occurrence(s) or annotate with // ${ALLOW} <reason>.`
)
for (const v of ratchet.filter((x) => x.category === category)) {
console.error(` ${v.file}:${v.line} ${v.snippet}`)
}
}
}
}
if (failed) process.exit(1)
console.log('\n✓ React Query pattern audit passed.')
process.exit(0)
}
main()
+67
View File
@@ -0,0 +1,67 @@
#!/usr/bin/env bun
import { mkdtemp, readdir, rm, stat } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import path from 'node:path'
import { $ } from 'bun'
const MAX_PRUNED_PACKAGE_COUNT = 25
async function listPackages(root: string): Promise<string[]> {
try {
const entries = await readdir(root)
const result: string[] = []
for (const entry of entries) {
const full = path.join(root, entry)
const s = await stat(full)
if (s.isDirectory()) {
result.push(entry)
}
}
return result
} catch {
return []
}
}
async function main() {
const scratch = await mkdtemp(path.join(tmpdir(), 'sim-realtime-prune-'))
try {
console.log(`Pruning @sim/realtime into ${scratch}`)
await $`bunx turbo prune @sim/realtime --docker --out-dir=${scratch}`.quiet()
const apps = await listPackages(path.join(scratch, 'json', 'apps'))
const packages = await listPackages(path.join(scratch, 'json', 'packages'))
const total = apps.length + packages.length
console.log(`Pruned apps (${apps.length}): ${apps.join(', ') || '(none)'}`)
console.log(`Pruned packages (${packages.length}): ${packages.join(', ') || '(none)'}`)
console.log(`Total pruned workspaces: ${total}`)
if (total > MAX_PRUNED_PACKAGE_COUNT) {
console.error(
`\n❌ Pruned realtime dep graph has ${total} workspaces (limit: ${MAX_PRUNED_PACKAGE_COUNT}).`
)
console.error(
'A new package was pulled into @sim/realtime. Ensure only pure, single-purpose packages are in its dep graph.'
)
process.exit(1)
}
const unexpectedApps = apps.filter((name) => name !== 'realtime')
if (unexpectedApps.length > 0) {
console.error(
`\n❌ Pruned realtime tree pulled in unexpected apps/: ${unexpectedApps.join(', ')}`
)
process.exit(1)
}
console.log('\n✅ Realtime prune size within expected bounds')
} finally {
await rm(scratch, { recursive: true, force: true })
}
}
void main().catch((error) => {
console.error('Realtime prune check failed:', error)
process.exit(1)
})
+154
View File
@@ -0,0 +1,154 @@
#!/usr/bin/env bun
/**
* Enforces use of shared @sim/utils helpers over inline implementations.
*
* Biome's noRestrictedImports covers import-based bans (nanoid, uuid, crypto named imports).
* This script catches patterns that static import analysis misses — global property access,
* inline idioms, and reimplemented helpers that should live in @sim/utils.
*/
import { readdir, readFile } from 'node:fs/promises'
import path from 'node:path'
const ROOT = path.resolve(import.meta.dir, '..')
const SCAN_DIRS = [path.join(ROOT, 'apps'), path.join(ROOT, 'packages')]
const SKIP_DIRS = new Set(['node_modules', 'dist', '.next', '.turbo', 'coverage', 'bundles'])
/** Files that implement the utilities themselves — allowed to use the underlying primitives. */
const ALLOWLISTED_FILES = new Set([
'packages/utils/src/errors.ts',
'packages/utils/src/helpers.ts',
'packages/utils/src/random.ts',
'packages/utils/src/id.ts',
'packages/utils/src/object.ts',
'packages/utils/src/retry.ts',
'packages/utils/src/errors.test.ts',
'packages/utils/src/helpers.test.ts',
'packages/utils/src/random.test.ts',
'packages/utils/src/id.test.ts',
'packages/utils/src/object.test.ts',
'packages/utils/src/retry.test.ts',
'packages/cli/src/index.ts',
'packages/ts-sdk/src/index.ts',
// CJS bundle — cannot use ES module imports
'apps/sim/lib/execution/isolated-vm-worker.cjs',
// Uses crypto.getRandomValues() directly (not crypto.randomUUID) — TSDoc comment triggers false positive
'packages/testing/src/factories/id.ts',
])
const BANNED_PATTERNS: Array<{
pattern: RegExp
description: string
suggestion: string
}> = [
// Randomness / ID generation — global property access that import bans miss
{
pattern: /\bMath\.random\s*\(/g,
description: 'Math.random()',
suggestion: 'randomInt / randomFloat / randomItem from @sim/utils/random',
},
{
pattern: /\bcrypto\.randomUUID\s*\(/g,
description: 'crypto.randomUUID()',
suggestion: 'generateId() or generateShortId() from @sim/utils/id',
},
{
pattern: /\bcrypto\.randomBytes\s*\(/g,
description: 'crypto.randomBytes()',
suggestion: 'generateRandomBytes() or generateRandomHex() from @sim/utils/random',
},
// Deep clone idiom
{
pattern: /JSON\.parse\s*\(\s*JSON\.stringify\s*\(/g,
description: 'JSON.parse(JSON.stringify(...))',
suggestion: 'structuredClone() — built-in, no import needed',
},
// Inline error message extraction (excludes null/undefined/false fallbacks — those have different semantics)
{
pattern: /instanceof Error\s*\?\s*\w+\.message\s*:\s*(?!\s*null\b|\s*undefined\b|\s*false\b)./g,
description: 'e instanceof Error ? e.message : fallback',
suggestion: 'getErrorMessage(e, fallback?) from @sim/utils/errors',
},
// Inline sleep
{
pattern: /new Promise\s*[(<]\s*(?:resolve|\(resolve\))\s*=>\s*setTimeout\s*\(\s*resolve/g,
description: 'new Promise(resolve => setTimeout(resolve, ms))',
suggestion: 'sleep(ms) from @sim/utils/helpers',
},
]
async function walk(dir: string, results: string[] = []): Promise<string[]> {
let entries
try {
entries = await readdir(dir, { withFileTypes: true })
} catch {
return results
}
for (const entry of entries) {
if (SKIP_DIRS.has(entry.name)) continue
const full = path.join(dir, entry.name)
if (entry.isDirectory()) {
await walk(full, results)
} else if (/\.(ts|tsx|mts|cts|js|jsx|mjs|cjs)$/.test(entry.name)) {
results.push(full)
}
}
return results
}
interface Violation {
file: string
line: number
description: string
suggestion: string
snippet: string
}
async function main() {
const allFiles: string[] = []
for (const dir of SCAN_DIRS) {
await walk(dir, allFiles)
}
const violations: Violation[] = []
for (const file of allFiles) {
const rel = path.relative(ROOT, file)
if (ALLOWLISTED_FILES.has(rel)) continue
const content = await readFile(file, 'utf8')
const lines = content.split('\n')
for (let i = 0; i < lines.length; i++) {
const line = lines[i]
for (const { pattern, description, suggestion } of BANNED_PATTERNS) {
pattern.lastIndex = 0
if (pattern.test(line)) {
violations.push({
file: rel,
line: i + 1,
description,
suggestion,
snippet: line.trim(),
})
}
}
}
}
if (violations.length === 0) {
console.log('✓ No banned patterns found.')
process.exit(0)
}
console.error(`\nFound ${violations.length} banned pattern(s):\n`)
for (const v of violations) {
console.error(` ${v.file}:${v.line}`)
console.error(`${v.description} → use ${v.suggestion}`)
console.error(` ${v.snippet}\n`)
}
process.exit(1)
}
main()
+371
View File
@@ -0,0 +1,371 @@
#!/usr/bin/env bun
import { readdir, readFile } from 'node:fs/promises'
import path from 'node:path'
const ROOT = path.resolve(import.meta.dir, '..')
const APP_DIR = path.join(ROOT, 'apps/sim')
const SKIP_DIRS = new Set(['node_modules', '.next', '.turbo', 'coverage', 'dist', 'build'])
const SOURCE_EXTENSIONS = new Set(['.ts', '.tsx', '.js', '.jsx'])
const STORE_HOOK_CALL_PATTERN = /\buse[A-Z][A-Za-z0-9_]*Store\s*\(/g
const SAFE_ANNOTATION = 'zustand-v5-safe:'
const UNSAFE_SELECTOR_PATTERNS: Array<{ pattern: RegExp; reason: string }> = [
{
pattern: /=>\s*\(\s*\{/,
reason: 'selector returns a fresh object literal; wrap it in useShallow',
},
{
pattern: /\breturn\s+\{/,
reason: 'selector returns a fresh object literal; wrap it in useShallow',
},
{
pattern: /=>\s*\[/,
reason: 'selector returns a fresh array literal; wrap it in useShallow',
},
{
pattern: /\breturn\s+\[/,
reason: 'selector returns a fresh array literal; wrap it in useShallow',
},
{
pattern:
/(?:=>|return)\s+Object\.(?:values|entries)\s*\([^)]*\)(?!\s*\.\s*(?:length|some|every)\b)/,
reason:
'selector allocates a derived collection; use useStoreWithEqualityFn or memoize outside',
},
{
pattern: /\bObject\.fromEntries\s*\(/,
reason: 'selector allocates a derived object; use useStoreWithEqualityFn or memoize outside',
},
{
pattern: /\bObject\.keys\s*\([^)]*\)(?!\s*\.length\b)/,
reason: 'selector allocates Object.keys; return a primitive or use useShallow',
},
{
pattern: /(?:=>|return)\s+[^;{}]*\.(?:map|filter|reduce)\s*\(/,
reason: 'selector allocates a derived value; use useStoreWithEqualityFn or memoize outside',
},
{
pattern: /\bnew\s+(?:Set|Map)\s*\(/,
reason:
'selector returns a fresh collection; use useStoreWithEqualityFn or a stable store reference',
},
{
pattern: /\?\?\s*(?:\(\s*\)\s*=>|\{\s*\}|\[\s*\])/,
reason: 'selector uses an unstable fallback reference; move the fallback to module scope',
},
{
pattern: /\|\|\s*(?:\(\s*\)\s*=>|\{\s*\}|\[\s*\])/,
reason: 'selector uses an unstable fallback reference; move the fallback to module scope',
},
]
interface Violation {
file: string
line: number
description: string
snippet: string
}
async function walk(dir: string, results: string[] = []): Promise<string[]> {
const entries = await readdir(dir, { withFileTypes: true })
for (const entry of entries) {
if (SKIP_DIRS.has(entry.name)) continue
const full = path.join(dir, entry.name)
if (entry.isDirectory()) {
await walk(full, results)
continue
}
if (SOURCE_EXTENSIONS.has(path.extname(entry.name))) {
results.push(full)
}
}
return results
}
function findMatchingParen(source: string, openIndex: number): number {
let depth = 0
let quote: '"' | "'" | '`' | null = null
let escaped = false
let lineComment = false
let blockComment = false
for (let index = openIndex; index < source.length; index++) {
const char = source[index]
const next = source[index + 1]
if (lineComment) {
if (char === '\n') lineComment = false
continue
}
if (blockComment) {
if (char === '*' && next === '/') {
blockComment = false
index++
}
continue
}
if (quote) {
if (escaped) {
escaped = false
continue
}
if (char === '\\') {
escaped = true
continue
}
if (char === quote) {
quote = null
}
continue
}
if (char === '/' && next === '/') {
lineComment = true
index++
continue
}
if (char === '/' && next === '*') {
blockComment = true
index++
continue
}
if (char === '"' || char === "'" || char === '`') {
quote = char
continue
}
if (char === '(') depth++
if (char === ')') {
depth--
if (depth === 0) return index
}
}
return -1
}
function splitTopLevelArguments(args: string): string[] {
const result: string[] = []
let start = 0
let depth = 0
let quote: '"' | "'" | '`' | null = null
let escaped = false
for (let index = 0; index < args.length; index++) {
const char = args[index]
if (quote) {
if (escaped) {
escaped = false
continue
}
if (char === '\\') {
escaped = true
continue
}
if (char === quote) quote = null
continue
}
if (char === '"' || char === "'" || char === '`') {
quote = char
continue
}
if (char === '(' || char === '[' || char === '{') depth++
if (char === ')' || char === ']' || char === '}') depth--
if (char === ',' && depth === 0) {
result.push(args.slice(start, index).trim())
start = index + 1
}
}
const finalArg = args.slice(start).trim()
if (finalArg) result.push(finalArg)
return result
}
function lineNumberAt(source: string, index: number): number {
let line = 1
for (let i = 0; i < index; i++) {
if (source[i] === '\n') line++
}
return line
}
function hasSafeAnnotation(source: string, callStart: number): boolean {
const before = source.slice(0, callStart)
const lines = before.split('\n')
for (let i = lines.length - 1; i >= Math.max(0, lines.length - 4); i--) {
const trimmed = lines[i]?.trim()
if (!trimmed) continue
if (trimmed.includes(SAFE_ANNOTATION) && trimmed.split(SAFE_ANNOTATION)[1]?.trim()) {
return true
}
if (!trimmed.startsWith('//') && !trimmed.startsWith('*') && !trimmed.startsWith('/*')) {
return false
}
}
return false
}
function oneLineSnippet(source: string, start: number, end: number): string {
return source.slice(start, end).replace(/\s+/g, ' ').trim().slice(0, 180)
}
function auditFile(file: string, source: string): Violation[] {
const violations: Violation[] = []
STORE_HOOK_CALL_PATTERN.lastIndex = 0
for (
let match = STORE_HOOK_CALL_PATTERN.exec(source);
match;
match = STORE_HOOK_CALL_PATTERN.exec(source)
) {
const callStart = match.index
const callee = match[0].replace(/\s*\($/, '')
if (callee === 'useSyncExternalStore') continue
if (hasSafeAnnotation(source, callStart)) continue
const openParenIndex = source.indexOf('(', callStart)
const closeParenIndex = findMatchingParen(source, openParenIndex)
if (closeParenIndex === -1) continue
const args = splitTopLevelArguments(source.slice(openParenIndex + 1, closeParenIndex))
const line = lineNumberAt(source, callStart)
const snippet = oneLineSnippet(source, callStart, closeParenIndex + 1)
if (args.length === 0) {
violations.push({
file,
line,
description: `${callee} subscribes to the entire store; select only the fields needed`,
snippet,
})
continue
}
if (args.length > 1) {
violations.push({
file,
line,
description: `${callee} passes a second equality argument; Zustand v5 create() hooks ignore the v4 pattern. Use useShallow or useStoreWithEqualityFn.`,
snippet,
})
continue
}
const selector = args[0]
if (!selector || selector.startsWith('useShallow(')) continue
for (const { pattern, reason } of UNSAFE_SELECTOR_PATTERNS) {
pattern.lastIndex = 0
if (pattern.test(selector)) {
if (returnsPrimitiveDerivedValue(selector)) continue
if (usesReferenceFallbackOnlyInsideBlockBody(selector)) continue
violations.push({
file,
line,
description: `${callee} ${reason}`,
snippet,
})
break
}
}
}
return violations
}
function returnsPrimitiveDerivedValue(selector: string): boolean {
return (
/\bObject\.(?:keys|values|entries)\s*\([^)]*\)\s*\.\s*(?:length|some|every)\b/.test(selector) ||
/\bObject\.keys\s*\([^)]*\)\.length\b/.test(selector) ||
/\.(?:map|filter)\s*\([^)]*\)\s*\.\s*(?:length|some|every|join)\b/.test(selector)
)
}
function usesReferenceFallbackOnlyInsideBlockBody(selector: string): boolean {
if (!/\)\s*=>\s*\{/.test(selector)) return false
const returnExpressions = [...selector.matchAll(/\breturn\s+([^;\n}]+)/g)].map((match) =>
match[1].trim()
)
return (
returnExpressions.length > 0 &&
returnExpressions.every((expression) => isPrimitiveReturnExpression(expression, selector))
)
}
function isPrimitiveReturnExpression(expression: string, selector: string): boolean {
const normalized = expression
.trim()
.replace(/^\((.*)\)$/, '$1')
.trim()
if (/^(?:true|false|null|undefined)\b/.test(normalized)) return true
if (/^(?:['"`]|\d)/.test(normalized)) return true
if (/^(?:!|typeof\b)/.test(normalized)) return true
if (/^(?:Boolean|Number|String)\s*\(/.test(normalized)) return true
if (/(?:===|!==|==|!=|>=|<=|>|<)/.test(normalized)) return true
if (/\.(?:length|some|every|includes|has)\s*(?:\(|$)/.test(normalized)) return true
if (/^[A-Za-z_$][\w$]*$/.test(normalized)) {
return isIdentifierAssignedPrimitive(normalized, selector)
}
return false
}
function isIdentifierAssignedPrimitive(identifier: string, selector: string): boolean {
const declarationPattern = new RegExp(`\\b(?:const|let)\\s+${identifier}\\s*=\\s*([^;\\n]+)`)
const declaration = selector.match(declarationPattern)
if (!declaration) return false
return isPrimitiveReturnExpression(declaration[1], selector)
}
async function main() {
const files = await walk(APP_DIR)
const violations: Violation[] = []
for (const file of files) {
const source = await readFile(file, 'utf8')
const relativeFile = path.relative(ROOT, file)
violations.push(...auditFile(relativeFile, source))
}
if (violations.length === 0) {
console.log('✅ Zustand v5 selector audit OK')
return
}
console.error('❌ Zustand v5 selector hazards found:')
console.error(
`Add useShallow/useStoreWithEqualityFn, split into primitive selectors, or document intentional exceptions with // ${SAFE_ANNOTATION} <reason>.`
)
for (const violation of violations) {
console.error(
` ${violation.file}:${violation.line}${violation.description}\n ${violation.snippet}`
)
}
process.exit(1)
}
void main().catch((error) => {
console.error('Zustand v5 selector audit failed:', error)
process.exit(1)
})
+417
View File
@@ -0,0 +1,417 @@
#!/usr/bin/env bun
import { execSync } from 'node:child_process'
import { Octokit } from '@octokit/rest'
import { sleep } from '@sim/utils/helpers'
const GITHUB_TOKEN = process.env.GH_PAT
const REPO_OWNER = 'simstudioai'
const REPO_NAME = 'sim'
if (!GITHUB_TOKEN) {
console.error('❌ GH_PAT environment variable is required')
process.exit(1)
}
const targetVersion = process.argv[2]
if (!targetVersion) {
console.error('❌ Version argument is required')
console.error('Usage: bun run scripts/create-single-release.ts v0.3.XX')
process.exit(1)
}
const octokit = new Octokit({
auth: GITHUB_TOKEN,
})
interface VersionCommit {
hash: string
version: string
title: string
date: string
author: string
}
interface CommitDetail {
hash: string
message: string
author: string
githubUsername: string
prNumber?: string
}
function execCommand(command: string): string {
try {
return execSync(command, { encoding: 'utf8' }).trim()
} catch (error) {
console.error(`❌ Command failed: ${command}`)
throw error
}
}
function findVersionCommit(version: string): VersionCommit | null {
console.log(`🔍 Finding commit for version ${version}...`)
const gitLog = execCommand('git log --oneline --format="%H|%s|%ai|%an" main')
const lines = gitLog.split('\n').filter((line) => line.trim())
for (const line of lines) {
const [hash, message, date, author] = line.split('|')
const versionMatch = message.match(/^\s*(v\d+\.\d+\.?\d*):\s*(.+)$/)
if (versionMatch && versionMatch[1] === version) {
return {
hash,
version,
title: versionMatch[2],
date: new Date(date).toISOString(),
author,
}
}
}
return null
}
function findPreviousVersionCommit(currentVersion: string): VersionCommit | null {
console.log(`🔍 Finding previous version before ${currentVersion}...`)
const gitLog = execCommand('git log --oneline --format="%H|%s|%ai|%an" main')
const lines = gitLog.split('\n').filter((line) => line.trim())
let foundCurrent = false
for (const line of lines) {
const [hash, message, date, author] = line.split('|')
const versionMatch = message.match(/^\s*(v\d+\.\d+\.?\d*):\s*(.+)$/)
if (versionMatch) {
if (versionMatch[1] === currentVersion) {
foundCurrent = true
continue
}
if (foundCurrent) {
return {
hash,
version: versionMatch[1],
title: versionMatch[2],
date: new Date(date).toISOString(),
author,
}
}
}
}
return null
}
async function fetchGitHubCommitDetails(
commitHashes: string[]
): Promise<Map<string, CommitDetail>> {
console.log(`🔍 Fetching GitHub commit details for ${commitHashes.length} commits...`)
const commitMap = new Map<string, CommitDetail>()
for (let i = 0; i < commitHashes.length; i++) {
const hash = commitHashes[i]
try {
const { data: commit } = await octokit.rest.repos.getCommit({
owner: REPO_OWNER,
repo: REPO_NAME,
ref: hash,
})
const prMatch = commit.commit.message.match(/\(#(\d+)\)/)
const prNumber = prMatch ? prMatch[1] : undefined
const githubUsername = commit.author?.login || commit.committer?.login || 'unknown'
let cleanMessage = commit.commit.message.split('\n')[0]
if (prNumber) {
cleanMessage = cleanMessage.replace(/\s*\(#\d+\)\s*$/, '')
}
commitMap.set(hash, {
hash,
message: cleanMessage,
author: commit.commit.author?.name || 'Unknown',
githubUsername,
prNumber,
})
await sleep(100)
} catch (error: any) {
console.warn(`⚠️ Could not fetch commit ${hash.substring(0, 7)}: ${error?.message || error}`)
try {
const gitData = execCommand(`git log --format="%s|%an" -1 ${hash}`).split('|')
let message = gitData[0] || 'Unknown commit'
const prMatch = message.match(/\(#(\d+)\)/)
const prNumber = prMatch ? prMatch[1] : undefined
if (prNumber) {
message = message.replace(/\s*\(#\d+\)\s*$/, '')
}
commitMap.set(hash, {
hash,
message,
author: gitData[1] || 'Unknown',
githubUsername: 'unknown',
prNumber,
})
} catch (fallbackError) {
console.error(`❌ Failed to get fallback data for ${hash.substring(0, 7)}`)
}
}
}
return commitMap
}
async function getCommitsBetweenVersions(
currentCommit: VersionCommit,
previousCommit?: VersionCommit
): Promise<CommitDetail[]> {
try {
let range: string
if (previousCommit) {
range = `${previousCommit.hash}..${currentCommit.hash}`
console.log(
`🔍 Getting commits between ${previousCommit.version} and ${currentCommit.version}`
)
} else {
range = `${currentCommit.hash}~10..${currentCommit.hash}`
console.log(`🔍 Getting commits before first version ${currentCommit.version}`)
}
const gitLog = execCommand(`git log --oneline --format="%H|%s" ${range}`)
if (!gitLog.trim()) {
console.log(`⚠️ No commits found in range ${range}`)
return []
}
const commitEntries = gitLog.split('\n').filter((line) => line.trim())
const nonVersionCommits = commitEntries.filter((line) => {
const [, message] = line.split('|')
const isVersionCommit = message.match(/^v\d+\.\d+/)
if (isVersionCommit) {
console.log(`⏭️ Skipping version commit: ${message.substring(0, 50)}...`)
return false
}
return true
})
console.log(`📋 After filtering version commits: ${nonVersionCommits.length} commits`)
if (nonVersionCommits.length === 0) {
return []
}
const commitHashes = nonVersionCommits.map((line) => line.split('|')[0])
const commitMap = await fetchGitHubCommitDetails(commitHashes)
return commitHashes.map((hash) => commitMap.get(hash)!).filter(Boolean)
} catch (error) {
console.error(`❌ Error getting commits between versions:`, error)
return []
}
}
function categorizeCommit(message: string): 'features' | 'fixes' | 'improvements' | 'other' {
const msgLower = message.toLowerCase()
if (/^feat(\(|:|!)/.test(msgLower)) {
return 'features'
}
if (/^fix(\(|:|!)/.test(msgLower)) {
return 'fixes'
}
if (/^(improvement|improve|perf|refactor)(\(|:|!)/.test(msgLower)) {
return 'improvements'
}
if (/^(chore|docs|style|test|ci|build)(\(|:|!)/.test(msgLower)) {
return 'other'
}
if (msgLower.includes('feat') || msgLower.includes('implement') || msgLower.includes('new ')) {
return 'features'
}
if (msgLower.includes('fix') || msgLower.includes('bug') || msgLower.includes('error')) {
return 'fixes'
}
if (
msgLower.includes('improve') ||
msgLower.includes('enhance') ||
msgLower.includes('upgrade') ||
msgLower.includes('optimization') ||
msgLower.includes('add') ||
msgLower.includes('update')
) {
return 'improvements'
}
return 'other'
}
async function generateReleaseBody(
versionCommit: VersionCommit,
previousCommit?: VersionCommit
): Promise<string> {
console.log(`📝 Generating release body for ${versionCommit.version}...`)
const commits = await getCommitsBetweenVersions(versionCommit, previousCommit)
if (commits.length === 0) {
console.log(`⚠️ No commits found, using simple format`)
return `${versionCommit.title}
[View changes on GitHub](https://github.com/${REPO_OWNER}/${REPO_NAME}/compare/${previousCommit?.version || 'v1.0.0'}...${versionCommit.version})`
}
console.log(`📋 Processing ${commits.length} commits for categorization`)
const features = commits.filter((c) => categorizeCommit(c.message) === 'features')
const fixes = commits.filter((c) => categorizeCommit(c.message) === 'fixes')
const improvements = commits.filter((c) => categorizeCommit(c.message) === 'improvements')
const others = commits.filter((c) => categorizeCommit(c.message) === 'other')
console.log(
`📊 Categories: ${features.length} features, ${improvements.length} improvements, ${fixes.length} fixes, ${others.length} other`
)
let body = ''
if (features.length > 0) {
body += '## Features\n\n'
for (const commit of features) {
const prLink = commit.prNumber ? ` (#${commit.prNumber})` : ''
body += `- ${commit.message}${prLink}\n`
}
body += '\n'
}
if (improvements.length > 0) {
body += '## Improvements\n\n'
for (const commit of improvements) {
const prLink = commit.prNumber ? ` (#${commit.prNumber})` : ''
body += `- ${commit.message}${prLink}\n`
}
body += '\n'
}
if (fixes.length > 0) {
body += '## Bug Fixes\n\n'
for (const commit of fixes) {
const prLink = commit.prNumber ? ` (#${commit.prNumber})` : ''
body += `- ${commit.message}${prLink}\n`
}
body += '\n'
}
if (others.length > 0) {
body += '## Other Changes\n\n'
for (const commit of others) {
const prLink = commit.prNumber ? ` (#${commit.prNumber})` : ''
body += `- ${commit.message}${prLink}\n`
}
body += '\n'
}
const uniqueContributors = new Set<string>()
commits.forEach((commit) => {
if (commit.githubUsername && commit.githubUsername !== 'unknown') {
uniqueContributors.add(commit.githubUsername)
}
})
if (uniqueContributors.size > 0) {
body += '## Contributors\n\n'
for (const contributor of Array.from(uniqueContributors).sort()) {
body += `- @${contributor}\n`
}
body += '\n'
}
body += `[View changes on GitHub](https://github.com/${REPO_OWNER}/${REPO_NAME}/compare/${previousCommit?.version || 'v1.0.0'}...${versionCommit.version})`
return body.trim()
}
async function main() {
try {
console.log(`🚀 Creating single release for ${targetVersion}...`)
const versionCommit = findVersionCommit(targetVersion)
if (!versionCommit) {
console.error(`❌ No commit found for version ${targetVersion}`)
process.exit(1)
}
console.log(
`✅ Found version commit: ${versionCommit.hash.substring(0, 7)} - ${versionCommit.title}`
)
const previousCommit = findPreviousVersionCommit(targetVersion)
if (previousCommit) {
console.log(`✅ Found previous version: ${previousCommit.version}`)
} else {
console.log(`️ No previous version found (this might be the first release)`)
}
try {
const existingRelease = await octokit.rest.repos.getReleaseByTag({
owner: REPO_OWNER,
repo: REPO_NAME,
tag: targetVersion,
})
if (existingRelease.data) {
console.log(`️ Release ${targetVersion} already exists, skipping creation`)
console.log(
`🔗 View release: https://github.com/${REPO_OWNER}/${REPO_NAME}/releases/tag/${targetVersion}`
)
return
}
} catch (error: any) {
if (error.status !== 404) {
throw error
}
}
const releaseBody = await generateReleaseBody(versionCommit, previousCommit || undefined)
console.log(`🚀 Creating GitHub release for ${targetVersion}...`)
await octokit.rest.repos.createRelease({
owner: REPO_OWNER,
repo: REPO_NAME,
tag_name: targetVersion,
name: targetVersion,
body: releaseBody,
draft: false,
prerelease: false,
target_commitish: versionCommit.hash,
})
console.log(`✅ Successfully created release: ${targetVersion}`)
console.log(
`🔗 View release: https://github.com/${REPO_OWNER}/${REPO_NAME}/releases/tag/${targetVersion}`
)
} catch (error) {
console.error('❌ Script failed:', error)
process.exit(1)
}
}
main()
+19
View File
@@ -0,0 +1,19 @@
import { spawnSync } from 'node:child_process'
export function formatGeneratedSource(source: string, stdinFilePath: string, cwd: string): string {
const result = spawnSync('bunx', ['biome', 'format', '--stdin-file-path', stdinFilePath], {
cwd,
encoding: 'utf8',
input: source,
})
if (result.status !== 0) {
throw new Error(
`Failed to format generated source for ${stdinFilePath}:\n${
result.stderr || result.stdout || 'unknown error'
}`
)
}
return result.stdout
}
+3939
View File
File diff suppressed because it is too large Load Diff
+120
View File
@@ -0,0 +1,120 @@
#!/usr/bin/env bun
// Drive every mothership contract generator, then biome-format the
// outputs so the committed files match what biome produces on commit
// (avoids the stale-drift that comes from comparing raw json2ts output
// against biome-formatted source).
//
// `--check` regenerates into a temp directory, formats identically,
// and compares against the committed files — same semantics as the
// old per-script `--check`, but accounts for post-generate formatting.
import { spawnSync } from 'node:child_process'
import { readFileSync } from 'node:fs'
import { dirname, join, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
const ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..')
const GENERATORS = [
'scripts/sync-mothership-stream-contract.ts',
'scripts/sync-tool-catalog.ts',
'scripts/sync-trace-spans-contract.ts',
'scripts/sync-trace-attributes-contract.ts',
'scripts/sync-trace-attribute-values-contract.ts',
'scripts/sync-trace-events-contract.ts',
'scripts/sync-metrics-contract.ts',
'scripts/sync-vfs-snapshot-contract.ts',
]
// Generated files under this path. We biome-format this whole dir on
// each generate (and the temp copy on each check).
const GENERATED_DIR = 'apps/sim/lib/copilot/generated'
// `tool-schemas-v1.ts` goes through biome's `--unsafe` bracket-quote
// fixer which reformats every key of TOOL_RUNTIME_SCHEMAS. Strip it
// from the format pass so generator output stays stable on both sides.
const FORMAT_EXCLUDE = new Set(['tool-schemas-v1.ts'])
function run(cmd: string[], cwd: string, env: NodeJS.ProcessEnv = process.env): void {
const result = spawnSync(cmd[0], cmd.slice(1), {
cwd,
env,
stdio: 'inherit',
})
if (result.status !== 0) {
process.exit(result.status ?? 1)
}
}
function runGenerators(outputOverride?: string): void {
const env = { ...process.env }
for (const script of GENERATORS) {
const args = ['bun', 'run', script]
if (outputOverride) {
// Individual scripts don't accept a custom output dir; for
// --check we generate in place and snapshot before/after via
// git-index comparison (see runCheck).
}
run(args, ROOT, env)
}
}
function formatGenerated(dir: string): void {
const files = readdirNoThrow(dir).filter((f) => !FORMAT_EXCLUDE.has(f) && f.endsWith('.ts'))
if (files.length === 0) return
const paths = files.map((f) => join(dir, f))
run(['bunx', 'biome', 'check', '--write', ...paths], ROOT)
}
function readdirNoThrow(dir: string): string[] {
try {
// Bun has fs.readdirSync available as a CommonJS import
const fs = require('node:fs') as typeof import('node:fs')
return fs.readdirSync(dir)
} catch {
return []
}
}
function runCheck(): void {
const targetDir = resolve(ROOT, GENERATED_DIR)
// Snapshot current committed state
const committed: Record<string, string> = {}
for (const f of readdirNoThrow(targetDir)) {
if (!f.endsWith('.ts')) continue
committed[f] = readFileSync(join(targetDir, f), 'utf8')
}
// Regenerate in place + format, then diff against the snapshot
runGenerators()
formatGenerated(targetDir)
const stale: string[] = []
for (const [name, oldContent] of Object.entries(committed)) {
if (FORMAT_EXCLUDE.has(name)) continue
const newContent = readFileSync(join(targetDir, name), 'utf8')
if (newContent !== oldContent) stale.push(name)
}
// Restore the committed state regardless of outcome (--check is readonly).
for (const [name, content] of Object.entries(committed)) {
const fs = require('node:fs') as typeof import('node:fs')
fs.writeFileSync(join(targetDir, name), content, 'utf8')
}
if (stale.length > 0) {
console.error(`Generated contracts are stale: ${stale.join(', ')}. Run: bun run mship:generate`)
process.exit(1)
}
console.log('All generated contracts up to date.')
}
function runGenerate(): void {
runGenerators()
formatGenerated(resolve(ROOT, GENERATED_DIR))
console.log('Generated + formatted mothership contracts.')
}
const checkOnly = process.argv.includes('--check')
if (checkOnly) runCheck()
else runGenerate()
+142
View File
@@ -0,0 +1,142 @@
import { mkdir, readFile, writeFile } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { formatGeneratedSource } from './format-generated-source'
/**
* Generate `apps/sim/lib/copilot/generated/metrics-v1.ts` from the Go-side
* `contracts/metrics-v1.schema.json` contract.
*
* The contract is a single-enum JSON Schema listing every canonical mothership
* OTel METRIC name. Go and Sim BOTH emit mothership metrics (the agent loop in
* Go; server-side tool/VFS/file instrumentation in Sim), so both sides MUST
* emit identical metric names for `histogram_quantile(sum by (le) …)` over the
* GoSim union to be valid. We emit:
* - A `Metric` const object keyed by PascalCase identifier whose values are
* the exact wire names, so call sites read `meter.createHistogram(
* Metric.CopilotToolDuration)` instead of a raw string literal.
* - A `MetricKey` / `MetricValue` union pair.
* - A sorted `MetricValues` readonly array for tests/enumeration.
*
* Label allowlists and histogram bucket boundaries are NOT encoded in the
* schema (name-only). The Go side owns the label-cardinality allowlist
* (contracts/metrics_v1.go) and the shared bucket constant
* (internal/telemetry/metrics.go); the Sim emitter MUST use the identical
* label keys and bucket boundaries by hand.
*
* This is the metric-name twin of `sync-trace-attributes-contract.ts`; the two
* share the enum-extraction + PascalCase + collision-detection pattern.
*/
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
const ROOT = resolve(SCRIPT_DIR, '..')
const DEFAULT_CONTRACT_PATH = resolve(ROOT, '../copilot/copilot/contracts/metrics-v1.schema.json')
const OUTPUT_PATH = resolve(ROOT, 'apps/sim/lib/copilot/generated/metrics-v1.ts')
function extractMetricNames(schema: Record<string, unknown>): string[] {
const defs = (schema.$defs ?? {}) as Record<string, unknown>
const nameDef = defs.MetricsV1Name
if (
!nameDef ||
typeof nameDef !== 'object' ||
!Array.isArray((nameDef as Record<string, unknown>).enum)
) {
throw new Error('metrics-v1.schema.json is missing $defs.MetricsV1Name.enum')
}
const enumValues = (nameDef as Record<string, unknown>).enum as unknown[]
if (!enumValues.every((v) => typeof v === 'string')) {
throw new Error('MetricsV1Name enum must be string-only')
}
return (enumValues as string[]).slice().sort()
}
/**
* Convert a wire metric name like `copilot.request.duration` into an
* identifier-safe PascalCase key like `CopilotRequestDuration`. Same algorithm
* as the trace-attributes sync script so readers learn one and reuse it.
*/
function toIdentifier(name: string): string {
const parts = name.split(/[^A-Za-z0-9]+/).filter(Boolean)
if (parts.length === 0) {
throw new Error(`Cannot derive identifier for metric name: ${name}`)
}
const ident = parts.map((p) => p.charAt(0).toUpperCase() + p.slice(1).toLowerCase()).join('')
if (/^[0-9]/.test(ident)) {
throw new Error(`Derived identifier "${ident}" for metric "${name}" starts with a digit`)
}
return ident
}
function render(metricNames: string[]): string {
const pairs = metricNames.map((name) => ({ name, ident: toIdentifier(name) }))
const seen = new Map<string, string>()
for (const p of pairs) {
const prev = seen.get(p.ident)
if (prev && prev !== p.name) {
throw new Error(`Identifier collision: "${prev}" and "${p.name}" both map to "${p.ident}"`)
}
seen.set(p.ident, p.name)
}
const constLines = pairs.map((p) => ` ${p.ident}: ${JSON.stringify(p.name)},`).join('\n')
const arrayEntries = metricNames.map((n) => ` ${JSON.stringify(n)},`).join('\n')
return `// AUTO-GENERATED FILE. DO NOT EDIT.
//
// Source: copilot/copilot/contracts/metrics-v1.schema.json
// Regenerate with: bun run metrics-contract:generate
//
// Canonical mothership OTel metric names. Call sites should reference
// \`Metric.<Identifier>\` (e.g. \`Metric.CopilotToolDuration\`) rather than raw
// string literals, so the Go-side contract is the single source of truth and
// typos become compile errors.
//
// NAMES ONLY. Label keys and histogram bucket boundaries are NOT in this
// contract — Go owns the label-cardinality allowlist and the shared bucket
// constant, and the Sim emitter MUST mirror those by hand so the GoSim metric
// union is queryable as one series set.
export const Metric = {
${constLines}
} as const;
export type MetricKey = keyof typeof Metric;
export type MetricValue = (typeof Metric)[MetricKey];
/** Readonly sorted list of every canonical mothership metric name. */
export const MetricValues: readonly MetricValue[] = [
${arrayEntries}
] as const;
`
}
async function main() {
const checkOnly = process.argv.includes('--check')
const inputArg = process.argv.find((a) => a.startsWith('--input='))
const inputPath = inputArg
? resolve(ROOT, inputArg.slice('--input='.length))
: DEFAULT_CONTRACT_PATH
const raw = await readFile(inputPath, 'utf8')
const schema = JSON.parse(raw)
const metricNames = extractMetricNames(schema)
const rendered = formatGeneratedSource(render(metricNames), OUTPUT_PATH, ROOT)
if (checkOnly) {
const existing = await readFile(OUTPUT_PATH, 'utf8').catch(() => null)
if (existing !== rendered) {
throw new Error('Generated metrics contract is stale. Run: bun run metrics-contract:generate')
}
console.log('Metrics contract is up to date.')
return
}
await mkdir(dirname(OUTPUT_PATH), { recursive: true })
await writeFile(OUTPUT_PATH, rendered, 'utf8')
console.log(`Generated metrics types -> ${OUTPUT_PATH}`)
}
main().catch((err) => {
console.error(err)
process.exit(1)
})
+103
View File
@@ -0,0 +1,103 @@
import { mkdir, readFile, writeFile } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { compile } from 'json-schema-to-typescript'
import { formatGeneratedSource } from './format-generated-source'
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
const ROOT = resolve(SCRIPT_DIR, '..')
const DEFAULT_CONTRACT_PATH = resolve(
ROOT,
'../copilot/copilot/contracts/mothership-stream-v1.schema.json'
)
const OUTPUT_PATH = resolve(ROOT, 'apps/sim/lib/copilot/generated/mothership-stream-v1.ts')
const RUNTIME_SCHEMA_OUTPUT_PATH = resolve(
ROOT,
'apps/sim/lib/copilot/generated/mothership-stream-v1-schema.ts'
)
function generateRuntimeConstants(schema: Record<string, unknown>, existingTypes: string): string {
const defs = (schema.$defs ?? schema.definitions ?? {}) as Record<string, unknown>
const lines: string[] = []
for (const [name, def] of Object.entries(defs)) {
if (!def || typeof def !== 'object') continue
const defObj = def as Record<string, unknown>
const enumValues = defObj.enum
if (!Array.isArray(enumValues) || enumValues.length === 0) continue
if (!enumValues.every((v) => typeof v === 'string')) continue
const typeAlias = (enumValues as string[]).map((v) => JSON.stringify(v)).join(' | ')
const entries = (enumValues as string[])
.map((v) => ` ${JSON.stringify(v)}: ${JSON.stringify(v)}`)
.join(',\n')
if (!existingTypes.includes(`export type ${name} =`)) {
lines.push(`export type ${name} = ${typeAlias}\n`)
}
lines.push(`export const ${name} = {\n${entries},\n} as const;\n`)
}
return lines.join('\n')
}
function renderRuntimeSchemaModule(schema: unknown): string {
return [
'// AUTO-GENERATED FILE. DO NOT EDIT.',
'// Generated from copilot/contracts/mothership-stream-v1.schema.json',
'//',
'',
'export type JsonSchema = unknown',
'',
`export const MOTHERSHIP_STREAM_V1_SCHEMA: JsonSchema = ${JSON.stringify(schema, null, 2)}`,
'',
].join('\n')
}
async function main() {
const checkOnly = process.argv.includes('--check')
const inputPathArg = process.argv.find((arg) => arg.startsWith('--input='))
const inputPath = inputPathArg
? resolve(ROOT, inputPathArg.slice('--input='.length))
: DEFAULT_CONTRACT_PATH
const raw = await readFile(inputPath, 'utf8')
const schema = JSON.parse(raw)
const types = await compile(schema, 'MothershipStreamV1EventEnvelope', {
bannerComment: '// AUTO-GENERATED FILE. DO NOT EDIT.\n//',
unreachableDefinitions: true,
additionalProperties: false,
})
const constants = generateRuntimeConstants(schema, types)
const rendered = formatGeneratedSource(
constants ? `${types}\n${constants}\n` : types,
OUTPUT_PATH,
ROOT
)
const renderedSchemaModule = formatGeneratedSource(
renderRuntimeSchemaModule(schema),
RUNTIME_SCHEMA_OUTPUT_PATH,
ROOT
)
if (checkOnly) {
const existing = await readFile(OUTPUT_PATH, 'utf8').catch(() => null)
const existingSchemaModule = await readFile(RUNTIME_SCHEMA_OUTPUT_PATH, 'utf8').catch(
() => null
)
if (existing !== rendered || existingSchemaModule !== renderedSchemaModule) {
throw new Error(
`Generated mothership stream contract is stale. Run: bun run mship-contracts:generate`
)
}
return
}
await mkdir(dirname(OUTPUT_PATH), { recursive: true })
await writeFile(OUTPUT_PATH, rendered, 'utf8')
await writeFile(RUNTIME_SCHEMA_OUTPUT_PATH, renderedSchemaModule, 'utf8')
}
await main()
+234
View File
@@ -0,0 +1,234 @@
import { mkdir, readFile, writeFile } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { formatGeneratedSource } from './format-generated-source'
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
const ROOT = resolve(SCRIPT_DIR, '..')
const DEFAULT_CATALOG_PATH = resolve(ROOT, '../copilot/copilot/contracts/tool-catalog-v1.json')
const OUTPUT_PATH = resolve(ROOT, 'apps/sim/lib/copilot/generated/tool-catalog-v1.ts')
const RUNTIME_SCHEMA_OUTPUT_PATH = resolve(
ROOT,
'apps/sim/lib/copilot/generated/tool-schemas-v1.ts'
)
function snakeToPascal(s: string): string {
return s
.split('_')
.map((w) => w.charAt(0).toUpperCase() + w.slice(1))
.join('')
}
function toCamelIdentifier(value: string): string {
const parts = value.split(/[^a-zA-Z0-9]+/).filter(Boolean)
if (parts.length === 0) return 'value'
const camel = parts
.map((part, index) => {
const lower = part.toLowerCase()
if (index === 0) return lower
return lower.charAt(0).toUpperCase() + lower.slice(1)
})
.join('')
return /^[0-9]/.test(camel) ? `v${camel}` : camel
}
function getTopLevelOperationEnum(tool: Record<string, unknown>): string[] | undefined {
const parameters =
typeof tool.parameters === 'object' && tool.parameters !== null
? (tool.parameters as Record<string, unknown>)
: null
const properties =
parameters && typeof parameters.properties === 'object' && parameters.properties !== null
? (parameters.properties as Record<string, unknown>)
: null
const operation =
properties && typeof properties.operation === 'object' && properties.operation !== null
? (properties.operation as Record<string, unknown>)
: null
const values = operation?.enum
if (!Array.isArray(values) || values.some((value) => typeof value !== 'string')) {
return undefined
}
return values as string[]
}
function inferTSType(values: unknown[]): string {
const unique = [...new Set(values.filter((v) => v !== undefined && v !== null))]
if (unique.length === 0) return 'string'
if (unique.every((v) => typeof v === 'string')) {
return unique
.map((v) => JSON.stringify(v))
.sort()
.join(' | ')
}
if (unique.every((v) => typeof v === 'boolean')) return 'boolean'
if (unique.every((v) => typeof v === 'number')) return 'number'
return 'unknown'
}
function renderRuntimeSchemaModule(catalog: { tools: Record<string, unknown>[] }): string {
const lines: string[] = [
'// AUTO-GENERATED FILE. DO NOT EDIT.',
'// Generated from copilot/contracts/tool-catalog-v1.json',
'//',
'',
'export type JsonSchema = unknown',
'',
'export interface ToolRuntimeSchemaEntry {',
' parameters?: JsonSchema;',
' resultSchema?: JsonSchema;',
'}',
'',
'export const TOOL_RUNTIME_SCHEMAS: Record<string, ToolRuntimeSchemaEntry> = {',
]
for (const tool of catalog.tools) {
const id = JSON.stringify(tool.id)
const parameters =
'parameters' in tool ? JSON.stringify(tool.parameters ?? null, null, 2) : 'undefined'
const resultSchema =
'resultSchema' in tool ? JSON.stringify(tool.resultSchema ?? null, null, 2) : 'undefined'
lines.push(` [${id}]: {`)
lines.push(
` parameters: ${parameters === 'null' ? 'undefined' : parameters.replace(/\n/g, '\n ')},`
)
lines.push(
` resultSchema: ${resultSchema === 'null' ? 'undefined' : resultSchema.replace(/\n/g, '\n ')},`
)
lines.push(' },')
}
lines.push('}')
lines.push('')
return lines.join('\n')
}
function generateInterface(tools: Record<string, unknown>[]): string {
if (tools.length === 0) return 'export interface ToolCatalogEntry {}\n'
const allKeys = new Set<string>()
for (const tool of tools) {
for (const key of Object.keys(tool)) {
allKeys.add(key)
}
}
const requiredKeys = new Set<string>()
for (const key of allKeys) {
if (tools.every((t) => key in t)) {
requiredKeys.add(key)
}
}
const lines: string[] = ['export interface ToolCatalogEntry {']
for (const key of [...allKeys].sort()) {
const values = tools.map((t) => t[key])
const tsType = inferTSType(values)
const optional = requiredKeys.has(key) ? '' : '?'
lines.push(` ${key}${optional}: ${tsType};`)
}
lines.push('}')
return lines.join('\n')
}
async function main() {
const checkOnly = process.argv.includes('--check')
const inputPathArg = process.argv.find((arg) => arg.startsWith('--input='))
const inputPath = inputPathArg
? resolve(ROOT, inputPathArg.slice('--input='.length))
: DEFAULT_CATALOG_PATH
const raw = await readFile(inputPath, 'utf8')
const catalog = JSON.parse(raw) as { version: string; tools: Record<string, unknown>[] }
const iface = generateInterface(catalog.tools)
const lines: string[] = [
'// AUTO-GENERATED FILE. DO NOT EDIT.',
'// Generated from copilot/contracts/tool-catalog-v1.json',
'//',
'',
iface,
'',
]
const constNames: string[] = []
for (const tool of catalog.tools) {
const constName = snakeToPascal(tool.id as string)
constNames.push(constName)
const fields: string[] = []
for (const [key, value] of Object.entries(tool)) {
fields.push(` ${key}: ${JSON.stringify(value)}`)
}
lines.push(`export const ${constName}: ToolCatalogEntry = {`)
lines.push(`${fields.join(',\n')},`)
lines.push('};')
lines.push('')
}
for (const tool of catalog.tools) {
const constName = snakeToPascal(tool.id as string)
const operationEnum = getTopLevelOperationEnum(tool)
if (!operationEnum || operationEnum.length === 0) continue
const operationConstName = `${constName}Operation`
const seenKeys = new Set<string>()
const members = operationEnum.map((value, index) => {
let key = toCamelIdentifier(value)
if (seenKeys.has(key)) key = `${key}${index + 1}`
seenKeys.add(key)
return { key, value }
})
lines.push(`export const ${operationConstName} = {`)
for (const member of members) {
lines.push(` ${member.key}: ${JSON.stringify(member.value)},`)
}
lines.push('} as const;')
lines.push('')
lines.push(
`export type ${operationConstName} = (typeof ${operationConstName})[keyof typeof ${operationConstName}];`
)
lines.push('')
lines.push(`export const ${operationConstName}Values = [`)
for (const member of members) {
lines.push(` ${operationConstName}.${member.key},`)
}
lines.push(`] as const;`)
lines.push('')
}
lines.push(`export const TOOL_CATALOG: Record<string, ToolCatalogEntry> = {`)
for (let i = 0; i < catalog.tools.length; i++) {
lines.push(` [${constNames[i]}.id]: ${constNames[i]},`)
}
lines.push('};')
lines.push('')
const rendered = formatGeneratedSource(lines.join('\n'), OUTPUT_PATH, ROOT)
const runtimeSchemaRendered = formatGeneratedSource(
renderRuntimeSchemaModule(catalog),
RUNTIME_SCHEMA_OUTPUT_PATH,
ROOT
)
if (checkOnly) {
const existing = await readFile(OUTPUT_PATH, 'utf8').catch(() => null)
const existingRuntime = await readFile(RUNTIME_SCHEMA_OUTPUT_PATH, 'utf8').catch(() => null)
if (existing !== rendered || existingRuntime !== runtimeSchemaRendered) {
throw new Error(`Generated tool catalog is stale. Run: bun run mship-tools:generate`)
}
return
}
await mkdir(dirname(OUTPUT_PATH), { recursive: true })
await writeFile(OUTPUT_PATH, rendered, 'utf8')
await mkdir(dirname(RUNTIME_SCHEMA_OUTPUT_PATH), { recursive: true })
await writeFile(RUNTIME_SCHEMA_OUTPUT_PATH, runtimeSchemaRendered, 'utf8')
}
await main()
@@ -0,0 +1,149 @@
import { mkdir, readFile, writeFile } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { formatGeneratedSource } from './format-generated-source'
/**
* Generate `apps/sim/lib/copilot/generated/trace-attribute-values-v1.ts`
* from the Go-side `contracts/trace-attribute-values-v1.schema.json`
* contract.
*
* Unlike span-names / attribute-keys / event-names (each of which is a
* single enum), this contract carries MULTIPLE enums — one per span
* attribute whose value set is closed. The schema's `$defs` holds one
* definition per enum (e.g. `CopilotRequestCancelReason`,
* `CopilotAbortOutcome`, …). For each $def we emit a TS `as const`
* object named after the Go type, so call sites read as:
*
* span.setAttribute(
* TraceAttr.CopilotRequestCancelReason,
* CopilotRequestCancelReason.ExplicitStop,
* )
*
* Skipped $defs: anything that doesn't have a string-only `enum`
* array. That filters out wrapper structs the reflector adds
* incidentally (e.g. `TraceAttributeValuesV1AllDefs`).
*/
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
const ROOT = resolve(SCRIPT_DIR, '..')
const DEFAULT_CONTRACT_PATH = resolve(
ROOT,
'../copilot/copilot/contracts/trace-attribute-values-v1.schema.json'
)
const OUTPUT_PATH = resolve(ROOT, 'apps/sim/lib/copilot/generated/trace-attribute-values-v1.ts')
interface ExtractedEnum {
/** The Go type name — becomes the TS const + type name. */
name: string
/** The value strings, sorted for diff stability. */
values: string[]
}
function extractEnums(schema: Record<string, unknown>): ExtractedEnum[] {
const defs = (schema.$defs ?? {}) as Record<string, unknown>
const out: ExtractedEnum[] = []
for (const [name, def] of Object.entries(defs)) {
if (!def || typeof def !== 'object') continue
const enumValues = (def as Record<string, unknown>).enum
if (!Array.isArray(enumValues)) continue
if (!enumValues.every((v) => typeof v === 'string')) continue
out.push({ name, values: (enumValues as string[]).slice().sort() })
}
out.sort((a, b) => a.name.localeCompare(b.name))
return out
}
/**
* PascalCase identifier for a wire enum value. Mirrors the algorithm
* used by the span-names + attribute-keys scripts, so
* `explicit_stop` -> `ExplicitStop`, matching what a reader would
* guess from Go's exported constants.
*/
function toValueIdent(value: string): string {
const parts = value.split(/[^A-Za-z0-9]+/).filter(Boolean)
if (parts.length === 0) {
throw new Error(`Cannot derive identifier for enum value: ${value}`)
}
const ident = parts.map((p) => p.charAt(0).toUpperCase() + p.slice(1).toLowerCase()).join('')
if (/^[0-9]/.test(ident)) {
throw new Error(`Derived identifier "${ident}" for value "${value}" starts with a digit`)
}
return ident
}
function renderEnum(e: ExtractedEnum): string {
const seen = new Map<string, string>()
const lines = e.values.map((v) => {
const ident = toValueIdent(v)
const prev = seen.get(ident)
if (prev && prev !== v) {
throw new Error(
`Enum ${e.name}: identifier collision — "${prev}" and "${v}" both map to "${ident}"`
)
}
seen.set(ident, v)
return ` ${ident}: ${JSON.stringify(v)},`
})
return `export const ${e.name} = {
${lines.join('\n')}
} as const;
export type ${e.name}Key = keyof typeof ${e.name};
export type ${e.name}Value = (typeof ${e.name})[${e.name}Key];`
}
function render(enums: ExtractedEnum[]): string {
const body = enums.map(renderEnum).join('\n\n')
return `// AUTO-GENERATED FILE. DO NOT EDIT.
//
// Source: copilot/copilot/contracts/trace-attribute-values-v1.schema.json
// Regenerate with: bun run trace-attribute-values-contract:generate
//
// Canonical closed-set value vocabularies for mothership OTel
// attributes. Call sites should reference e.g.
// \`CopilotRequestCancelReason.ExplicitStop\` rather than the raw
// string literal, so typos become compile errors and the Go contract
// remains the single source of truth.
${body}
`
}
async function main() {
const checkOnly = process.argv.includes('--check')
const inputArg = process.argv.find((a) => a.startsWith('--input='))
const inputPath = inputArg
? resolve(ROOT, inputArg.slice('--input='.length))
: DEFAULT_CONTRACT_PATH
const raw = await readFile(inputPath, 'utf8')
const schema = JSON.parse(raw)
const enums = extractEnums(schema)
if (enums.length === 0) {
throw new Error(
'No enum $defs found in trace-attribute-values-v1.schema.json — did you add the Go type to TraceAttributeValuesV1AllDefs?'
)
}
const rendered = formatGeneratedSource(render(enums), OUTPUT_PATH, ROOT)
if (checkOnly) {
const existing = await readFile(OUTPUT_PATH, 'utf8').catch(() => null)
if (existing !== rendered) {
throw new Error(
'Generated trace attribute values contract is stale. Run: bun run trace-attribute-values-contract:generate'
)
}
console.log('Trace attribute values contract is up to date.')
return
}
await mkdir(dirname(OUTPUT_PATH), { recursive: true })
await writeFile(OUTPUT_PATH, rendered, 'utf8')
console.log(`Generated trace attribute values types -> ${OUTPUT_PATH}`)
}
main().catch((err) => {
console.error(err)
process.exit(1)
})
+156
View File
@@ -0,0 +1,156 @@
import { mkdir, readFile, writeFile } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { formatGeneratedSource } from './format-generated-source'
/**
* Generate `apps/sim/lib/copilot/generated/trace-attributes-v1.ts`
* from the Go-side `contracts/trace-attributes-v1.schema.json`
* contract.
*
* The contract is a single-enum JSON Schema listing every CUSTOM
* (non-OTel-semconv) span attribute key used in mothership. We emit:
* - A `TraceAttr` const object keyed by PascalCase identifier whose
* values are the exact wire strings, so call sites look like
* `span.setAttribute(TraceAttr.ChatId, …)` instead of the raw
* `span.setAttribute('chat.id', …)`.
* - A `TraceAttrKey` union and a `TraceAttrValue` union type so
* helpers that take an attribute key are well-typed.
* - A sorted `TraceAttrValues` readonly array for tests/enumeration.
*
* This is the attribute-key twin of `sync-trace-spans-contract.ts`
* (span names). The two files share the enum-extraction + identifier
* PascalCase + collision-detection pattern so a reader who understands
* one understands both.
*
* For OTel semantic-convention keys (e.g. `http.request.method`,
* `db.system`, `gen_ai.system`, `messaging.*`, `net.*`,
* `service.name`, `deployment.environment`), import from
* `@opentelemetry/semantic-conventions` directly — they live in the
* upstream package, not in this contract.
*/
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
const ROOT = resolve(SCRIPT_DIR, '..')
const DEFAULT_CONTRACT_PATH = resolve(
ROOT,
'../copilot/copilot/contracts/trace-attributes-v1.schema.json'
)
const OUTPUT_PATH = resolve(ROOT, 'apps/sim/lib/copilot/generated/trace-attributes-v1.ts')
function extractAttrKeys(schema: Record<string, unknown>): string[] {
const defs = (schema.$defs ?? {}) as Record<string, unknown>
const nameDef = defs.TraceAttributesV1Name
if (
!nameDef ||
typeof nameDef !== 'object' ||
!Array.isArray((nameDef as Record<string, unknown>).enum)
) {
throw new Error('trace-attributes-v1.schema.json is missing $defs.TraceAttributesV1Name.enum')
}
const enumValues = (nameDef as Record<string, unknown>).enum as unknown[]
if (!enumValues.every((v) => typeof v === 'string')) {
throw new Error('TraceAttributesV1Name enum must be string-only')
}
return (enumValues as string[]).slice().sort()
}
/**
* Convert a wire attribute key like `copilot.vfs.input.media_type_claimed`
* into an identifier-safe PascalCase key like
* `CopilotVfsInputMediaTypeClaimed`.
*
* Same algorithm as the span-name sync script so readers can learn one
* and reuse it.
*/
function toIdentifier(name: string): string {
const parts = name.split(/[^A-Za-z0-9]+/).filter(Boolean)
if (parts.length === 0) {
throw new Error(`Cannot derive identifier for attribute key: ${name}`)
}
const ident = parts.map((p) => p.charAt(0).toUpperCase() + p.slice(1).toLowerCase()).join('')
if (/^[0-9]/.test(ident)) {
throw new Error(`Derived identifier "${ident}" for attribute "${name}" starts with a digit`)
}
return ident
}
function render(attrKeys: string[]): string {
const pairs = attrKeys.map((name) => ({ name, ident: toIdentifier(name) }))
// Identifier collisions silently override earlier keys and break
// type safety — fail loudly instead.
const seen = new Map<string, string>()
for (const p of pairs) {
const prev = seen.get(p.ident)
if (prev && prev !== p.name) {
throw new Error(`Identifier collision: "${prev}" and "${p.name}" both map to "${p.ident}"`)
}
seen.set(p.ident, p.name)
}
const constLines = pairs.map((p) => ` ${p.ident}: ${JSON.stringify(p.name)},`).join('\n')
const arrayEntries = attrKeys.map((n) => ` ${JSON.stringify(n)},`).join('\n')
return `// AUTO-GENERATED FILE. DO NOT EDIT.
//
// Source: copilot/copilot/contracts/trace-attributes-v1.schema.json
// Regenerate with: bun run trace-attributes-contract:generate
//
// Canonical custom mothership OTel span attribute keys. Call sites
// should reference \`TraceAttr.<Identifier>\` (e.g.
// \`TraceAttr.ChatId\`, \`TraceAttr.ToolCallId\`) rather than raw
// string literals, so the Go-side contract is the single source of
// truth and typos become compile errors.
//
// For OTel semantic-convention keys (\`http.*\`, \`db.*\`,
// \`gen_ai.*\`, \`net.*\`, \`messaging.*\`, \`service.*\`,
// \`deployment.environment\`), import from
// \`@opentelemetry/semantic-conventions\` directly — those are owned
// by the upstream OTel spec, not by this contract.
export const TraceAttr = {
${constLines}
} as const;
export type TraceAttrKey = keyof typeof TraceAttr;
export type TraceAttrValue = (typeof TraceAttr)[TraceAttrKey];
/** Readonly sorted list of every canonical custom attribute key. */
export const TraceAttrValues: readonly TraceAttrValue[] = [
${arrayEntries}
] as const;
`
}
async function main() {
const checkOnly = process.argv.includes('--check')
const inputArg = process.argv.find((a) => a.startsWith('--input='))
const inputPath = inputArg
? resolve(ROOT, inputArg.slice('--input='.length))
: DEFAULT_CONTRACT_PATH
const raw = await readFile(inputPath, 'utf8')
const schema = JSON.parse(raw)
const attrKeys = extractAttrKeys(schema)
const rendered = formatGeneratedSource(render(attrKeys), OUTPUT_PATH, ROOT)
if (checkOnly) {
const existing = await readFile(OUTPUT_PATH, 'utf8').catch(() => null)
if (existing !== rendered) {
throw new Error(
'Generated trace attributes contract is stale. Run: bun run trace-attributes-contract:generate'
)
}
console.log('Trace attributes contract is up to date.')
return
}
await mkdir(dirname(OUTPUT_PATH), { recursive: true })
await writeFile(OUTPUT_PATH, rendered, 'utf8')
console.log(`Generated trace attributes types -> ${OUTPUT_PATH}`)
}
main().catch((err) => {
console.error(err)
process.exit(1)
})
+125
View File
@@ -0,0 +1,125 @@
import { mkdir, readFile, writeFile } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { formatGeneratedSource } from './format-generated-source'
/**
* Generate `apps/sim/lib/copilot/generated/trace-events-v1.ts` from
* the Go-side `contracts/trace-events-v1.schema.json` contract.
*
* Mirrors the span-names + attribute-keys sync scripts exactly — the
* only difference is the $defs key (`TraceEventsV1Name`), the output
* path, and the generated const name (`TraceEvent`). Keeping the
* scripts structurally identical means a reader who understands one
* understands all three, and drift between them gets caught
* immediately in code review.
*/
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
const ROOT = resolve(SCRIPT_DIR, '..')
const DEFAULT_CONTRACT_PATH = resolve(
ROOT,
'../copilot/copilot/contracts/trace-events-v1.schema.json'
)
const OUTPUT_PATH = resolve(ROOT, 'apps/sim/lib/copilot/generated/trace-events-v1.ts')
function extractEventNames(schema: Record<string, unknown>): string[] {
const defs = (schema.$defs ?? {}) as Record<string, unknown>
const nameDef = defs.TraceEventsV1Name
if (
!nameDef ||
typeof nameDef !== 'object' ||
!Array.isArray((nameDef as Record<string, unknown>).enum)
) {
throw new Error('trace-events-v1.schema.json is missing $defs.TraceEventsV1Name.enum')
}
const enumValues = (nameDef as Record<string, unknown>).enum as unknown[]
if (!enumValues.every((v) => typeof v === 'string')) {
throw new Error('TraceEventsV1Name enum must be string-only')
}
return (enumValues as string[]).slice().sort()
}
function toIdentifier(name: string): string {
const parts = name.split(/[^A-Za-z0-9]+/).filter(Boolean)
if (parts.length === 0) {
throw new Error(`Cannot derive identifier for event name: ${name}`)
}
const ident = parts.map((p) => p.charAt(0).toUpperCase() + p.slice(1).toLowerCase()).join('')
if (/^[0-9]/.test(ident)) {
throw new Error(`Derived identifier "${ident}" for event "${name}" starts with a digit`)
}
return ident
}
function render(eventNames: string[]): string {
const pairs = eventNames.map((name) => ({ name, ident: toIdentifier(name) }))
const seen = new Map<string, string>()
for (const p of pairs) {
const prev = seen.get(p.ident)
if (prev && prev !== p.name) {
throw new Error(`Identifier collision: "${prev}" and "${p.name}" both map to "${p.ident}"`)
}
seen.set(p.ident, p.name)
}
const constLines = pairs.map((p) => ` ${p.ident}: ${JSON.stringify(p.name)},`).join('\n')
const arrayEntries = eventNames.map((n) => ` ${JSON.stringify(n)},`).join('\n')
return `// AUTO-GENERATED FILE. DO NOT EDIT.
//
// Source: copilot/copilot/contracts/trace-events-v1.schema.json
// Regenerate with: bun run trace-events-contract:generate
//
// Canonical mothership OTel span event names. Call sites should
// reference \`TraceEvent.<Identifier>\` (e.g.
// \`TraceEvent.RequestCancelled\`) rather than raw string literals,
// so the Go-side contract is the single source of truth and typos
// become compile errors.
export const TraceEvent = {
${constLines}
} as const;
export type TraceEventKey = keyof typeof TraceEvent;
export type TraceEventValue = (typeof TraceEvent)[TraceEventKey];
/** Readonly sorted list of every canonical event name. */
export const TraceEventValues: readonly TraceEventValue[] = [
${arrayEntries}
] as const;
`
}
async function main() {
const checkOnly = process.argv.includes('--check')
const inputArg = process.argv.find((a) => a.startsWith('--input='))
const inputPath = inputArg
? resolve(ROOT, inputArg.slice('--input='.length))
: DEFAULT_CONTRACT_PATH
const raw = await readFile(inputPath, 'utf8')
const schema = JSON.parse(raw)
const eventNames = extractEventNames(schema)
const rendered = formatGeneratedSource(render(eventNames), OUTPUT_PATH, ROOT)
if (checkOnly) {
const existing = await readFile(OUTPUT_PATH, 'utf8').catch(() => null)
if (existing !== rendered) {
throw new Error(
'Generated trace events contract is stale. Run: bun run trace-events-contract:generate'
)
}
console.log('Trace events contract is up to date.')
return
}
await mkdir(dirname(OUTPUT_PATH), { recursive: true })
await writeFile(OUTPUT_PATH, rendered, 'utf8')
console.log(`Generated trace events types -> ${OUTPUT_PATH}`)
}
main().catch((err) => {
console.error(err)
process.exit(1)
})
+143
View File
@@ -0,0 +1,143 @@
import { mkdir, readFile, writeFile } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { formatGeneratedSource } from './format-generated-source'
/**
* Generate `apps/sim/lib/copilot/generated/trace-spans-v1.ts` from the
* Go-side `contracts/trace-spans-v1.schema.json` contract.
*
* The contract is a single-enum JSON Schema. We emit:
* - A `TraceSpansV1Name` const object (key-as-value) for ergonomic
* access: `TraceSpansV1Name['copilot.vfs.read_file']`.
* - A `TraceSpansV1NameValue` union type.
* - A sorted `TraceSpansV1Names` readonly array (useful for tests that
* verify coverage, and for tooling that wants to enumerate names).
*
* We deliberately do NOT pass through `json-schema-to-typescript` —
* it would generate a noisy `TraceSpansV1` object type for the wrapper
* that drives reflection; the wrapper type has no runtime use on the Sim
* side and would obscure the actual enum.
*/
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
const ROOT = resolve(SCRIPT_DIR, '..')
const DEFAULT_CONTRACT_PATH = resolve(
ROOT,
'../copilot/copilot/contracts/trace-spans-v1.schema.json'
)
const OUTPUT_PATH = resolve(ROOT, 'apps/sim/lib/copilot/generated/trace-spans-v1.ts')
function extractSpanNames(schema: Record<string, unknown>): string[] {
const defs = (schema.$defs ?? {}) as Record<string, unknown>
const nameDef = defs.TraceSpansV1Name
if (
!nameDef ||
typeof nameDef !== 'object' ||
!Array.isArray((nameDef as Record<string, unknown>).enum)
) {
throw new Error('trace-spans-v1.schema.json is missing $defs.TraceSpansV1Name.enum')
}
const enumValues = (nameDef as Record<string, unknown>).enum as unknown[]
if (!enumValues.every((v) => typeof v === 'string')) {
throw new Error('TraceSpansV1Name enum must be string-only')
}
return (enumValues as string[]).slice().sort()
}
/**
* Convert a wire name like "copilot.recovery.check_replay_gap" into an
* identifier-safe PascalCase key like "CopilotRecoveryCheckReplayGap",
* so call sites read as `TraceSpan.CopilotRecoveryCheckReplayGap`
* instead of `TraceSpan["copilot.recovery.check_replay_gap"]`.
*
* Splits on `.`, `_`, and non-alphanumeric characters; capitalizes each
* part; collapses. Strict mapping (not a best-effort heuristic), so the
* same input always produces the same identifier.
*/
function toIdentifier(name: string): string {
const parts = name.split(/[^A-Za-z0-9]+/).filter(Boolean)
if (parts.length === 0) {
throw new Error(`Cannot derive identifier for span name: ${name}`)
}
const ident = parts.map((p) => p.charAt(0).toUpperCase() + p.slice(1).toLowerCase()).join('')
// Safety: identifiers may not start with a digit.
if (/^[0-9]/.test(ident)) {
throw new Error(`Derived identifier "${ident}" for span "${name}" starts with a digit`)
}
return ident
}
function render(spanNames: string[]): string {
const pairs = spanNames.map((name) => ({ name, ident: toIdentifier(name) }))
// Guard against collisions: if two wire names ever collapse to the
// same PascalCase identifier, we want a clear build failure, not a
// silent override.
const seen = new Map<string, string>()
for (const p of pairs) {
const prev = seen.get(p.ident)
if (prev && prev !== p.name) {
throw new Error(`Identifier collision: "${prev}" and "${p.name}" both map to "${p.ident}"`)
}
seen.set(p.ident, p.name)
}
const constLines = pairs.map((p) => ` ${p.ident}: ${JSON.stringify(p.name)},`).join('\n')
const arrayEntries = spanNames.map((n) => ` ${JSON.stringify(n)},`).join('\n')
return `// AUTO-GENERATED FILE. DO NOT EDIT.
//
// Source: copilot/copilot/contracts/trace-spans-v1.schema.json
// Regenerate with: bun run trace-spans-contract:generate
//
// Canonical mothership OTel span names. Call sites should reference
// \`TraceSpan.<Identifier>\` (e.g. \`TraceSpan.CopilotVfsReadFile\`)
// rather than raw string literals, so the Go-side contract is the
// single source of truth and typos become compile errors.
export const TraceSpan = {
${constLines}
} as const;
export type TraceSpanKey = keyof typeof TraceSpan;
export type TraceSpanValue = (typeof TraceSpan)[TraceSpanKey];
/** Readonly sorted list of every canonical span name. */
export const TraceSpanValues: readonly TraceSpanValue[] = [
${arrayEntries}
] as const;
`
}
async function main() {
const checkOnly = process.argv.includes('--check')
const inputArg = process.argv.find((a) => a.startsWith('--input='))
const inputPath = inputArg
? resolve(ROOT, inputArg.slice('--input='.length))
: DEFAULT_CONTRACT_PATH
const raw = await readFile(inputPath, 'utf8')
const schema = JSON.parse(raw)
const spanNames = extractSpanNames(schema)
const rendered = formatGeneratedSource(render(spanNames), OUTPUT_PATH, ROOT)
if (checkOnly) {
const existing = await readFile(OUTPUT_PATH, 'utf8').catch(() => null)
if (existing !== rendered) {
throw new Error(
'Generated trace spans contract is stale. Run: bun run trace-spans-contract:generate'
)
}
console.log('Trace spans contract is up to date.')
return
}
await mkdir(dirname(OUTPUT_PATH), { recursive: true })
await writeFile(OUTPUT_PATH, rendered, 'utf8')
console.log(`Generated trace spans types -> ${OUTPUT_PATH}`)
}
main().catch((err) => {
console.error(err)
process.exit(1)
})
+46
View File
@@ -0,0 +1,46 @@
import { mkdir, readFile, writeFile } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { compile } from 'json-schema-to-typescript'
import { formatGeneratedSource } from './format-generated-source'
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
const ROOT = resolve(SCRIPT_DIR, '..')
// Matches the sibling sync scripts' canonical layout. In a repo where the Go
// service lives at `mothership/copilot`, pass `--input=` (e.g.
// `--input=../mothership/copilot/contracts/vfs-snapshot-v1.schema.json`).
const DEFAULT_CONTRACT_PATH = resolve(
ROOT,
'../copilot/copilot/contracts/vfs-snapshot-v1.schema.json'
)
const OUTPUT_PATH = resolve(ROOT, 'apps/sim/lib/copilot/generated/vfs-snapshot-v1.ts')
async function main() {
const checkOnly = process.argv.includes('--check')
const inputPathArg = process.argv.find((arg) => arg.startsWith('--input='))
const inputPath = inputPathArg
? resolve(ROOT, inputPathArg.slice('--input='.length))
: DEFAULT_CONTRACT_PATH
const raw = await readFile(inputPath, 'utf8')
const schema = JSON.parse(raw)
const types = await compile(schema, 'VfsSnapshotV1', {
bannerComment: '// AUTO-GENERATED FILE. DO NOT EDIT.\n//',
unreachableDefinitions: true,
additionalProperties: false,
})
const rendered = formatGeneratedSource(types, OUTPUT_PATH, ROOT)
if (checkOnly) {
const existing = await readFile(OUTPUT_PATH, 'utf8').catch(() => null)
if (existing !== rendered) {
throw new Error('Generated vfs snapshot contract is stale. Run: bun run mship:generate')
}
return
}
await mkdir(dirname(OUTPUT_PATH), { recursive: true })
await writeFile(OUTPUT_PATH, rendered, 'utf8')
}
await main()
+20
View File
@@ -0,0 +1,20 @@
{
"compilerOptions": {
"target": "ES2020",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"esModuleInterop": true,
"allowSyntheticDefaultImports": true,
"strict": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"noEmit": true,
"allowImportingTsExtensions": true
},
"ts-node": {
"esm": true,
"experimentalSpecifierResolution": "node"
},
"include": ["./**/*.ts"]
}