chore: import upstream snapshot with attribution

This commit is contained in:
wehub-resource-sync
2026-07-13 12:58:18 +08:00
commit 6d5d58c1a9
18293 changed files with 3502153 additions and 0 deletions
+7
View File
@@ -0,0 +1,7 @@
* @tylerslaton @jpr5 @ranst91 @marthakelly @mme
# CI supply-chain hardening — allowlist changes require core dev review
.github/config-allowlist.txt @tylerslaton @jpr5 @ranst91 @marthakelly @mme
# Showcases — demo team owns these alongside core dev
examples/showcases/ @CopilotKit/demo @tylerslaton @jpr5 @ranst91 @marthakelly @mme
+81
View File
@@ -0,0 +1,81 @@
name: 🐛 Bug Report
description: Found a bug? Report it here!
title: "🐛 Bug: "
labels: ["bug"]
assignees: []
body:
- type: markdown
attributes:
value: |
Thank you for taking the time to fill out this bug report. Please choose a short and descriptive title for your issue.
- type: textarea
id: reproduction
attributes:
label: ♻️ Reproduction Steps
description: Explain how to reproduce the bug
value: |
1.
2.
3.
...
validations:
required: true
- type: textarea
id: expected-behavior
attributes:
label: ✅ Expected Behavior
description: |
Tell us what you expected to happen.
If relevant, please include screenshots as well.
placeholder: |
e.g. "When clicking the Copilot chat button, I expected X..."
validations:
required: true
- type: textarea
id: actual-behavior
attributes:
label: ❌ Actual Behavior
description: |
Tell us what actually happened.
If relevant, please include screenshots as well.
placeholder: |
e.g. "An error appeared in the console, and nothing happened."
validations:
required: true
- type: textarea
attributes:
label: Screenshots
description: If applicable, add screenshots to help explain your problem.
- type: textarea
id: version
attributes:
label: 𝌚 CopilotKit Version
description: |
Please run the following command in your project and paste the output below:
```bash
npm ls | grep "@copilotkit"
```
placeholder: |
Run `npm ls | grep "@copilotkit"` and paste the output here.
render: shell
validations:
required: true
- type: textarea
id: logs
attributes:
label: 📄 Logs (Optional)
description: |
Please provide any logs or error messages that may be useful.
- For Copilot Runtime, please provide any errors in the terminal.
- For CopilotKit on the browser, please provide any errors that appear in the browser's developer tools.
placeholder: |
This is optional.
render: shell
validations:
required: false
@@ -0,0 +1,55 @@
name: 🚀 Feature Request
description: Have an idea for enhancements or features? Let us know!
title: "🚀 Feature Request: "
labels: ["feature request"]
assignees: []
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to suggest a feature!
> **Before you file:** Search existing issues to avoid duplicates.
- type: checkboxes
id: preflight
attributes:
label: Pre-flight Checklist
options:
- label: I have searched existing issues, and this hasn't been requested yet.
required: true
- type: textarea
id: problem
attributes:
label: Problem or Motivation
description: What problem does this feature solve? Why is it needed?
placeholder: "I'm always frustrated when..."
validations:
required: true
- type: textarea
id: description
attributes:
label: 🎤 Tell us your idea
description: |
We want to make it as easy as possible for you to suggest enhancements or features. Please use the textareas below to describe your idea. Some high-level guidelines:
- What is your idea?
- Do you have any use case(s) in mind?
- Who would find this useful?
placeholder: |
Type your idea here...
validations:
required: true
- type: checkboxes
id: contribution
attributes:
label: Would you like to work on this?
description: We welcome contributions! Let us know if youd like to help implement this feature.
options:
- label: Yes, Id love to work on it!
- label: Im open to collaborating but need guidance.
- label: No, Im just sharing the idea.
validations:
required: false
@@ -0,0 +1,21 @@
name: 📚 Documentation Issue
description: Let us know how we can improve the documentation.
title: "📚 Documentation: "
labels: ["documentation"]
assignees: []
body:
- type: markdown
attributes:
value: |
We appreciate you taking the time to help us make our documentation better!
- type: textarea
id: description
attributes:
label: 💬 Let us know how we can improve our documentation.
description: |
If you are referring to an existing page in the documentation, please provide a link.
placeholder: |
Type your idea here...
validations:
required: true
+8
View File
@@ -0,0 +1,8 @@
blank_issues_enabled: true
contact_links:
- name: Read the Documentation
url: https://docs.copilot.ai
about: Read the documentation to get started.
- name: Join our Discord
url: https://discord.com/invite/6dffbvGU3D
about: Join the Discord server to ask questions and get help.
+1
View File
@@ -0,0 +1 @@
Please describe your issue in as much detail as possible
+37
View File
@@ -0,0 +1,37 @@
<!--
Thank you for sending the PR! We appreciate you spending the time to work on these changes.
Help us understand your motivation by explaining why you decided to make this change.
**Please PLEASE reach out to us first before starting any significant work on new or existing features.**
By the time you've gotten here, you're looking at creating a pull request so hopefully we're not too late.
We love community contributions! That said, we want to make sure we're all on the same page before you start.
Investing a lot of time and effort just to find out it doesn't align with the upstream project feels awful, and we don't want that to happen.
It also helps to make sure the work you're planning isn't already in progress.
As described in our contributing guide, please file an issue first: https://github.com/ag-ui-protocol/ag-ui/issues
Or, reach out to us on Discord: https://discord.com/invite/6dffbvGU3D
You can learn more about contributing to copilotkit here: https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md
Happy contributing!
-->
## What does this PR do?
(Describe the changes introduced in this PR)
## Related PRs and Issues
- (Direct link to related PR or issue, if relevant)
## Checklist
- [ ] I have read the [Contribution Guide](https://github.com/copilotkit/copilotkit/blob/master/CONTRIBUTING.md)
- [ ] If the PR changes or adds functionality, I have updated the relevant documentation
- [ ] "Allow edits by maintainers" is checked (lets us help iterate on your PR directly — faster turnaround for everyone)
+121
View File
@@ -0,0 +1,121 @@
examples/canvas/gemini/next.config.mjs
examples/canvas/langgraph-python/next.config.ts
examples/canvas/llamaindex-composio/next.config.ts
examples/canvas/llamaindex/next.config.ts
examples/canvas/mastra-pm/next.config.ts
examples/canvas/mastra/next.config.ts
examples/canvas/pydantic-ai/next.config.ts
examples/shadcn/next.config.ts
examples/integrations/a2a-a2ui/next.config.js
examples/integrations/a2a-middleware/next.config.ts
examples/integrations/adk/next.config.ts
examples/integrations/agent-spec/next.config.ts
examples/integrations/agentcore/frontend/src/vite-env.d.ts
examples/integrations/agentcore/frontend/vite.config.ts
examples/integrations/agno/next.config.ts
examples/integrations/claude-sdk-python/next.config.ts
examples/integrations/claude-sdk-typescript/next.config.ts
examples/integrations/crewai-crews/next.config.ts
examples/integrations/crewai-flows/next.config.ts
examples/integrations/langgraph-fastapi/next.config.ts
examples/integrations/langgraph-js/next.config.ts
examples/integrations/langgraph-python/next.config.ts
examples/integrations/llamaindex/next.config.ts
examples/integrations/mastra/next.config.ts
examples/integrations/mcp-apps/next.config.ts
examples/integrations/mcp-apps/threejs-server/vite.config.ts
examples/integrations/ms-agent-framework-dotnet/next.config.ts
examples/integrations/ms-agent-framework-python/next.config.ts
examples/integrations/pydantic-ai/next.config.ts
examples/integrations/strands-python/next.config.ts
examples/showcases/a2a-travel/next.config.ts
examples/showcases/a2ui-pdf-analyst/next.config.ts
examples/showcases/adk-dashboard/next.config.ts
examples/showcases/arcade-tools/next.config.ts
examples/showcases/banking/next.config.mjs
examples/showcases/chatkit-studio/apps/playground/next.config.ts
examples/showcases/chatkit-studio/apps/studio/next.config.ts
examples/showcases/chatkit-studio/apps/world/next.config.ts
examples/showcases/daytona-runcode/next.config.ts
examples/showcases/deep-agents-finance-erp/next.config.ts
examples/showcases/deep-agents-job-search/next.config.ts
examples/showcases/deep-agents/next.config.ts
examples/showcases/enterprise-brex/next.config.mjs
examples/showcases/generative-ui-playground/next.config.ts
examples/showcases/langgraph-js-support-agents/apps/web/next.config.ts
examples/showcases/mcp-apps/mcp-server/apps/vite.config.ts
examples/showcases/mcp-apps/next.config.ts
examples/showcases/mcp-demo/next.config.ts
examples/showcases/microsoft-kanban/next.config.ts
examples/showcases/multi-agent-canvas/frontend/next.config.ts
examples/showcases/multi-page/vite.config.ts
examples/showcases/open-mcp-client/apps/threejs-server/vite.config.ts
examples/showcases/open-mcp-client/apps/web/next.config.ts
examples/showcases/oracle-agent-memory/frontend/next.config.ts
examples/showcases/orca/frontend/next.config.mjs
examples/showcases/presentation/next.config.mjs
examples/showcases/pydantic-ai-todos/next.config.ts
examples/showcases/research-canvas/final/frontend/next.config.ts
examples/showcases/research-canvas/frontend/next.config.ts
examples/showcases/research-canvas/start/frontend/next.config.ts
examples/showcases/scene-creator/next.config.ts
examples/showcases/spreadsheet/next.config.mjs
examples/showcases/strands-crm/frontend/next.config.ts
examples/showcases/strands-file-analyzer/next.config.ts
examples/showcases/todo/next.config.mjs
examples/v1/_legacy/copilot-anthropic-pinecone/next.config.mjs
examples/v1/_legacy/copilot-openai-mongodb-atlas-vector-search/next.config.mjs
examples/v1/_legacy/saas-dynamic-dashboards/frontend/next.config.mjs
examples/v1/chat-with-your-data/next.config.ts
examples/v1/form-filling/next.config.ts
examples/v1/next-openai/next.config.js
examples/v1/next-pages-router/next.config.mjs
examples/v1/research-canvas/next.config.mjs
examples/v1/state-machine/next.config.mjs
examples/v1/travel/next.config.mjs
examples/v2/interrupts-langgraph/apps/web/next.config.ts
examples/v2/next-pages-router/next.config.mjs
examples/v2/react-router/vite.config.ts
examples/v2/react/demo/next.config.ts
packages/a2ui-renderer/tsdown.config.ts
packages/agentcore-runner/tsdown.config.ts
packages/core/tsdown.config.ts
packages/demo-agents/tsdown.config.ts
packages/react-core/tsdown.config.ts
packages/react-native/tsdown.config.ts
packages/react-textarea/tsdown.config.ts
packages/react-ui/tsdown.config.ts
packages/runtime-client-gql/tsdown.config.ts
packages/runtime/tsdown.config.ts
packages/sdk-js/tsdown.config.ts
packages/shared/tsdown.config.ts
packages/sqlite-runner/tsdown.config.ts
packages/voice/tsdown.config.ts
packages/vue/vite.config.ts
packages/web-components/tsdown.config.ts
packages/web-inspector/dev/vite.config.ts
packages/web-inspector/tsdown.config.ts
showcase/integrations/ag2/next.config.ts
showcase/integrations/agno/next.config.ts
showcase/integrations/built-in-agent/next.config.ts
showcase/integrations/claude-sdk-python/next.config.ts
showcase/integrations/claude-sdk-typescript/next.config.ts
showcase/integrations/crewai-crews/next.config.ts
showcase/integrations/google-adk/next.config.ts
showcase/integrations/langgraph-fastapi/next.config.ts
showcase/integrations/langgraph-python/next.config.ts
showcase/integrations/langgraph-typescript/next.config.ts
showcase/integrations/langroid/next.config.ts
showcase/integrations/llamaindex/next.config.ts
showcase/integrations/mastra/next.config.ts
showcase/integrations/ms-agent-dotnet/next.config.ts
showcase/integrations/ms-agent-harness-dotnet/next.config.ts
showcase/integrations/ms-agent-python/next.config.ts
showcase/integrations/pydantic-ai/next.config.ts
showcase/integrations/spring-ai/next.config.ts
showcase/integrations/strands/next.config.ts
showcase/integrations/strands-typescript/next.config.ts
showcase/shell-dashboard/next.config.ts
showcase/shell-docs/next.config.ts
showcase/shell-dojo/next.config.ts
showcase/shell/next.config.ts
+52
View File
@@ -0,0 +1,52 @@
#!/usr/bin/env bash
set -euo pipefail
ALLOWLIST=".github/config-allowlist.txt"
if [ ! -f "$ALLOWLIST" ]; then
echo "ERROR: Allowlist file $ALLOWLIST not found" >&2
exit 1
fi
PATTERNS=(
-name 'vite.config.*'
-o -name 'vite_*.mjs' -o -name 'vite_*.ts' -o -name 'vite_*.js'
-o -name 'vite-*.mjs' -o -name 'vite-*.ts' -o -name 'vite-*.js'
-o -name 'tsdown.config.*'
-o -name 'tsup.config.*'
-o -name 'rollup.config.*'
-o -name 'webpack.config.*'
-o -name 'esbuild.config.*'
-o -name 'next.config.*'
)
FOUND=$(find . \
-path './node_modules' -prune -o \
-path './.claude' -prune -o \
-path './.next' -prune -o \
-path '*/node_modules' -prune -o \
-path '*/dist' -prune -o \
-path '*/.next' -prune -o \
\( "${PATTERNS[@]}" \) -print |
sed 's|^\./||' |
sort)
UNEXPECTED=""
while IFS= read -r file; do
[ -z "$file" ] && continue
if ! grep -qxF "$file" "$ALLOWLIST"; then
UNEXPECTED="${UNEXPECTED}${file}"$'\n'
fi
done <<< "$FOUND"
if [ -n "$UNEXPECTED" ]; then
echo "::error::Unexpected build config files detected (not in allowlist):"
echo "$UNEXPECTED" | while IFS= read -r f; do
[ -n "$f" ] && echo " - $f"
done
echo ""
echo "If these are legitimate, add them to $ALLOWLIST and get CODEOWNERS approval."
exit 1
fi
echo "All build config files are on the allowlist."
@@ -0,0 +1,93 @@
name: Auto-merge showcase PRs
on:
pull_request:
branches: [main]
paths:
- "examples/showcases/**"
permissions:
contents: write
pull-requests: write
jobs:
auto-merge:
runs-on: ubuntu-latest
steps:
- name: Check author is on Demo team
id: check-team
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
# Authorize on the PR AUTHOR (immutable for the lifetime of the PR),
# never `context.actor` — `actor` is whoever triggered the most
# recent event, so a team member synchronizing or reopening an
# outsider's PR would otherwise green-light auto-merge of code
# they didn't author.
script: |
const prAuthor = context.payload.pull_request.user.login;
try {
await github.rest.teams.getMembershipForUserInOrg({
org: 'CopilotKit',
team_slug: 'demo',
username: prAuthor,
});
core.setOutput('is_demo', 'true');
} catch {
core.info(`${prAuthor} is not a member of CopilotKit/demo`);
core.setOutput('is_demo', 'false');
}
- name: Check PR only touches showcases
if: steps.check-team.outputs.is_demo == 'true'
id: check-files
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
const { data: files } = await github.rest.pulls.listFiles({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.payload.pull_request.number,
per_page: 100,
});
const allShowcases = files.every(f =>
f.filename.startsWith('examples/showcases/')
);
if (!allShowcases) {
const outside = files
.filter(f => !f.filename.startsWith('examples/showcases/'))
.map(f => f.filename);
core.info(`Files outside showcases: ${outside.join(', ')}`);
}
core.setOutput('only_showcases', allShowcases.toString());
- name: Approve PR
if: steps.check-team.outputs.is_demo == 'true' && steps.check-files.outputs.only_showcases == 'true'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
await github.rest.pulls.createReview({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.payload.pull_request.number,
event: 'APPROVE',
body: 'Auto-approved: showcase-only changes from CopilotKit/demo team member.',
});
- name: Enable auto-merge
if: steps.check-team.outputs.is_demo == 'true' && steps.check-files.outputs.only_showcases == 'true'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
await github.graphql(`
mutation($prId: ID!) {
enablePullRequestAutoMerge(input: {
pullRequestId: $prId,
mergeMethod: MERGE
}) {
clientMutationId
}
}
`, { prId: context.payload.pull_request.node_id });
+335
View File
@@ -0,0 +1,335 @@
name: canary / publish
# Discoverable, one-click canary publisher. Surfaces in the Actions tab so any
# maintainer can publish a prerelease of the branch they're working on without
# learning the manual `canary/*` branch + dispatch dance.
#
# IMPORTANT: this workflow does NOT publish to npm itself. It ORCHESTRATES
# publish-release.yml, which holds the SINGLE npm OIDC trusted-publisher binding
# (see that file's header). Adding a second npm-publishing entry point would
# break OIDC for every @copilotkit/* package. The npm trusted publishers for
# the @copilotkit packages are bound to publish-release.yml + the `npm`
# environment.
#
# Why a separate orchestrator instead of a flag inside publish-release.yml:
# The `npm` GitHub Environment's deployment-branch policy is evaluated against
# the ref a run is TRIGGERED on — NOT against branches created mid-run. So
# publish-release.yml can only publish a canary when its run's ref already
# matches the policy (`canary/*`). Creating a branch inside a run triggered on
# `feature/*` does not change that run's ref, so it would still be rejected.
# This workflow therefore runs on any non-main branch, mirrors it to a
# short-lived `canary/<slug>` ref, dispatches publish-release.yml ON that ref
# (clearing the env gate), waits for it, then deletes the ref.
#
# Token: the branch create/delete and the cross-workflow dispatch use the
# devops-bot GitHub App token, NOT the default GITHUB_TOKEN. Events authenticated
# with GITHUB_TOKEN do not start new workflow runs (recursion prevention), so the
# delegated publish-release run would silently never fire.
on:
workflow_dispatch:
inputs:
scope:
description: "Package scope to publish a canary for. Regenerated from release.config.json — do NOT hand-edit (the release-scope-dropdown-sync CI guard enforces parity)."
required: true
type: choice
options:
- monorepo
- angular
- channels
- channels-discord
- channels-intelligence
- channels-slack
- channels-teams
- channels-telegram
- channels-whatsapp
suffix:
description: "Prerelease suffix (e.g. 'fix-user-issue'); blank = unix timestamp. Allowed: [a-zA-Z0-9._-]+. Reuse a suffix only if the base version moved, else the publish collides."
required: false
type: string
dry_run:
# NOTE: this orchestrator exposes `dry_run` (underscore, matching ag-ui's
# convention), but publish-release.yml's input is `dry-run` (hyphen).
# The dispatch step below maps `dry_run` → `-f dry-run=` accordingly.
description: "Dry run: build + detect but do NOT publish to npm. Useful for previewing what would ship."
required: false
default: false
type: boolean
concurrency:
# Serialize repeated dispatches on the same source branch FOR THE SAME SCOPE.
# Cross-branch ref races are independently prevented by making the canary ref
# unique per run (slug + github.run_id, see the slug step below).
# The scope is folded into the key so canaries for different scopes (e.g.
# `monorepo` vs `angular`) on the same branch run in independent lanes instead
# of queuing behind each other (cancel-in-progress: false).
group: canary-publish-${{ inputs.scope }}-${{ github.ref }}
cancel-in-progress: false
permissions:
# The job's own GITHUB_TOKEN does nothing privileged — every write goes through
# the App token minted below.
contents: read
jobs:
canary:
runs-on: ubuntu-latest
# The delegated publish-release run can take up to ~40 min worst case
# (build ~20 + publish ~20); this orchestrator's ceiling must exceed it,
# otherwise a timeout-kill triggers ref cleanup mid-publish (yanking the
# canary ref out from under a still-running publish). publish-release.yml
# also carries a GLOBAL `concurrency: group: publish-release,
# cancel-in-progress: false`, so our delegated run can sit QUEUED behind
# an unrelated release for a long time before it even starts; the ceiling
# must budget that queue time on top of the ~40-min worst case.
timeout-minutes: 90
steps:
- name: Guard ref
# Canary publishes are for non-main BRANCHES only. Block main (use the
# stable release flow) and block non-branch refs such as tags (a tag
# dispatch would otherwise canary-publish from the tagged commit).
if: github.ref == 'refs/heads/main' || !startsWith(github.ref, 'refs/heads/')
# Pass github.ref via env (matches "Validate suffix" discipline) so the
# ref name is never interpolated into the shell — tag/branch names may
# contain shell metacharacters and direct ${{ }} interpolation here
# would be an expression-injection sink.
env:
REF: ${{ github.ref }}
run: |
echo "::error::Canary publishes are for non-main branches only (got '$REF'). To release from main, use the 'release / create-pr' workflow (stable-release.yml) → merge the release PR, or 'release / publish' with mode=stable for retries."
exit 1
- name: Validate suffix
if: inputs.suffix != ''
env:
SUFFIX: ${{ inputs.suffix }}
run: |
set -euo pipefail
# Validate BEFORE any side effect (token mint, ref creation) so a bad
# suffix can't leave an orphaned canary ref behind. Bash regex matches
# the WHOLE string (grep matches per line and would accept a multi-line
# value whose first line is valid).
if ! [[ "$SUFFIX" =~ ^[a-zA-Z0-9._-]+$ ]]; then
echo "::error::Invalid suffix '$SUFFIX'. Allowed: [a-zA-Z0-9._-]+ (blank = unix timestamp)."
exit 1
fi
- name: Mint devops-bot token
id: app-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: 1108748
private-key: ${{ secrets.DEVOPS_BOT_PRIVATE_KEY }}
# Scoped permissions (v3+): contents=write for create/delete of the
# canary ref; actions=write to dispatch publish-release.yml and watch
# the delegated run. No other scopes are needed.
permission-contents: write
permission-actions: write
- name: Compute canary branch name
id: slug
env:
REF_NAME: ${{ github.ref_name }}
run: |
set -euo pipefail
# Byte-deterministic (LC_ALL=C) transform collapsing the source ref to a
# single path segment under canary/ so it matches the `canary/*`
# deployment-branch policy. tr -s collapses same-char runs (so no `..`),
# and the sed fully strips any leading/trailing `.`/`-` runs.
export LC_ALL=C
SLUG=$(printf '%s' "$REF_NAME" | tr '/' '-' | tr -c 'a-zA-Z0-9._-' '-' | tr -s '.-')
SLUG=$(printf '%s' "$SLUG" | sed -E 's/^[.-]+//; s/[.-]+$//')
if [ -z "$SLUG" ]; then
echo "::error::Could not derive a canary slug from ref '$REF_NAME'"
exit 1
fi
# Append run id AND attempt so every dispatch — including a re-run of
# this same orchestration — owns a UNIQUE canary ref. This prevents two
# dispatches whose source branches slugify to the same value (or a
# re-run reusing the run id) from racing one shared ref, and keeps run
# discovery below unambiguous (exactly one publish run per ref).
REF_SUFFIX="${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
echo "branch=canary/${SLUG}-${REF_SUFFIX}" >> "$GITHUB_OUTPUT"
echo "Canary branch: canary/${SLUG}-${REF_SUFFIX}"
- name: Create or update canary ref
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
BRANCH: ${{ steps.slug.outputs.branch }}
SHA: ${{ github.sha }}
run: |
set -euo pipefail
# Point canary/<slug> at the dispatched ref's HEAD. The ref is unique
# per run AND attempt (slug + run_id + run_attempt — re-runs increment
# the attempt), so the "already exists" arm is defense-in-depth and
# should be unreachable in practice. Force-update only on that specific
# error; any OTHER failure (auth, rate limit, 5xx) must surface, not be
# silently retried.
ERR=$(mktemp)
if gh api --silent -X POST "repos/${GITHUB_REPOSITORY}/git/refs" \
-f ref="refs/heads/${BRANCH}" -f sha="$SHA" 2>"$ERR"; then
echo "Created ${BRANCH} at ${SHA}"
elif grep -qi "already exists" "$ERR"; then
echo "Ref ${BRANCH} already exists; force-updating to ${SHA}"
gh api --silent -X PATCH "repos/${GITHUB_REPOSITORY}/git/refs/heads/${BRANCH}" \
-f sha="$SHA" -F force=true
else
echo "::error::Failed to create canary ref ${BRANCH}:"
cat "$ERR" >&2
exit 1
fi
- name: Dispatch publish-release.yml on the canary ref and wait
id: dispatch
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
BRANCH: ${{ steps.slug.outputs.branch }}
SCOPE: ${{ inputs.scope }}
SUFFIX: ${{ inputs.suffix }}
DRY_RUN: ${{ inputs.dry_run }}
run: |
set -euo pipefail
# (suffix already validated in the "Validate suffix" step above, before
# the canary ref was created)
#
# Input name mapping: this orchestrator's `dry_run` (underscore) maps
# to publish-release.yml's `dry-run` (hyphen). Keep both as-is — the
# underscore matches ag-ui's convention; the hyphen matches cpk's
# existing publish-release.yml signature.
gh workflow run publish-release.yml \
--repo "$GITHUB_REPOSITORY" \
--ref "$BRANCH" \
-f mode=prerelease \
-f scope="$SCOPE" \
-f suffix="$SUFFIX" \
-f dry-run="$DRY_RUN"
# The dispatch went through. If anything from here on fails BEFORE a run
# is located, the cleanup step keeps the ref (a dispatched-but-unindexed
# run may still need it). If the dispatch itself had failed, no run can
# exist and the ref is safe to delete.
echo "dispatched=true" >> "$GITHUB_OUTPUT"
# Residual race: a cancellation landing in the instant between
# `gh workflow run` succeeding above and this output write would route
# cleanup down the "no run exists" path and delete the ref under a
# queued run. Accepted risk — the delegated canary run fails visibly
# at checkout and can simply be re-dispatched.
# The canary ref is unique to this run+attempt, so there is exactly ONE
# publish-release dispatch on it — no timestamp watermark needed (which
# also sidesteps runner/server clock-skew). Poll until it indexes
# (30 x 6s = 3 min tolerance for Actions indexing lag). --limit is
# defensive headroom; the branch/workflow/event filters are applied
# server-side so the matching run is never crowded out.
RUN_ID=""
ERR=$(mktemp)
for i in $(seq 1 30); do
sleep 6
RUN_ID=$(gh run list \
--repo "$GITHUB_REPOSITORY" \
--workflow=publish-release.yml \
--branch "$BRANCH" \
--event workflow_dispatch \
--limit 100 \
--json databaseId \
--jq 'sort_by(.databaseId) | last | .databaseId // empty' 2>"$ERR") || {
RUN_ID=""
echo "::warning::gh run list failed on attempt ${i}; will retry:" >&2
cat "$ERR" >&2
}
if [ -n "$RUN_ID" ]; then
break
fi
done
if [ -z "$RUN_ID" ]; then
echo "::error::Dispatched publish-release run never appeared on ${BRANCH}. Leaving the ref in place for debugging; delete it manually once resolved."
exit 1
fi
# Mark located BEFORE the watch so cleanup runs even if the publish
# fails — but is skipped entirely if we never tracked a run (so we
# never delete a ref a still-pending run may need).
echo "located=true" >> "$GITHUB_OUTPUT"
echo "run_id=${RUN_ID}" >> "$GITHUB_OUTPUT"
RUN_URL=$(gh run view "$RUN_ID" --repo "$GITHUB_REPOSITORY" --json url --jq .url)
echo "Delegated publish run: ${RUN_URL}"
{
echo "## Canary publish"
echo ""
echo "- **Scope:** \`${SCOPE}\`"
echo "- **Source branch:** \`${GITHUB_REF_NAME}\`"
echo "- **Delegated run:** ${RUN_URL}"
} >> "$GITHUB_STEP_SUMMARY"
# --exit-status propagates the publish run's failure to this job.
gh run watch "$RUN_ID" --repo "$GITHUB_REPOSITORY" --exit-status
# --exit-status catches failures, but a non-failure non-success conclusion
# (e.g. skipped) exits 0 with nothing published. Require success explicitly.
CONCLUSION=$(gh run view "$RUN_ID" --repo "$GITHUB_REPOSITORY" --json conclusion --jq .conclusion)
if [ "$CONCLUSION" != "success" ]; then
echo "::error::Delegated publish run concluded '$CONCLUSION' (expected 'success')."
exit 1
fi
- name: Mint cleanup token
id: cleanup-token
# The job ceiling (90 min) exceeds the 1-hour App-token TTL, so the
# token minted at job start can be expired by cleanup time. Mint a
# fresh one. Same condition as the delete step below.
if: always() && steps.slug.outputs.branch != '' && (steps.dispatch.outputs.located == 'true' || steps.dispatch.outputs.dispatched != 'true')
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: 1108748
private-key: ${{ secrets.DEVOPS_BOT_PRIVATE_KEY }}
permission-contents: write
permission-actions: read
- name: Delete canary ref
# Three cases:
# (a) Run was located (located=true) → delete the ref; the delegated
# publish run has finished (success or fail) and no longer needs it.
# Additionally gated below by a live status check on RUN_ID: on
# cancellation/timeout (always() runs cleanup too) a still-queued
# or running delegated run must keep its branch, since
# checkout-by-SHA can fail once the ref is gone.
# (b) Dispatch succeeded (dispatched=true) but the run was never located
# within the poll window → KEEP the ref; a pending publish run may
# still pick it up and would silently break if the ref vanished.
# (c) The dispatch command itself failed or this step never ran
# (dispatched != 'true') → no publish run can exist, so the ref is
# safe to delete (and almost certainly was never created).
# The `steps.slug.outputs.branch != ''` guard skips cleanup entirely when
# the slug step never produced a branch name (a guard/suffix failure that
# bailed before any ref was created). A DELETE on an empty path would
# 404-gracefully anyway, but skipping avoids the spurious API call.
if: always() && steps.slug.outputs.branch != '' && (steps.dispatch.outputs.located == 'true' || steps.dispatch.outputs.dispatched != 'true')
env:
GH_TOKEN: ${{ steps.cleanup-token.outputs.token }}
BRANCH: ${{ steps.slug.outputs.branch }}
RUN_ID: ${{ steps.dispatch.outputs.run_id }}
run: |
# Best-effort cleanup: never fail the job on a delete hiccup, but do
# surface a real error instead of masking every failure as "gone".
set -uo pipefail
# If we tracked a delegated run, only delete the ref once that run has
# completed. always() brings us here on cancellation/timeout too — a
# still-queued run would fail checkout if its branch (and thus the
# commit's reachability) disappears from under it.
if [ -n "${RUN_ID}" ]; then
STATUS=$(gh run view "$RUN_ID" --repo "$GITHUB_REPOSITORY" --json status --jq .status 2>/dev/null || echo unknown)
if [ "$STATUS" != "completed" ]; then
echo "::warning::Delegated run $RUN_ID status is '$STATUS' (not completed); keeping ${BRANCH} — delete it manually once the run finishes."
exit 0
fi
fi
ERR=$(mktemp)
if gh api --silent -X DELETE "repos/${GITHUB_REPOSITORY}/git/refs/heads/${BRANCH}" 2>"$ERR"; then
echo "Deleted ${BRANCH}"
elif grep -qiE "not found|does not exist" "$ERR"; then
echo "Branch ${BRANCH} already gone"
else
echo "::warning::Failed to delete canary ref ${BRANCH} (manual cleanup may be needed):"
cat "$ERR" >&2
fi
+34
View File
@@ -0,0 +1,34 @@
name: cleanup / pr-caches
on:
pull_request:
types:
- closed
permissions:
contents: read
jobs:
pr-caches:
runs-on: ubuntu-latest
permissions:
contents: read
# Needed to delete repository actions caches via `gh cache delete`
actions: write
steps:
- name: Cleanup
run: |
echo "Fetching list of cache key"
cacheKeysForPR=$(gh cache list --ref $BRANCH --limit 100 --json id --jq '.[].id')
## Setting this to not fail the workflow while deleting cache keys.
set +e
echo "Deleting caches..."
for cacheKey in $cacheKeysForPR
do
gh cache delete $cacheKey
done
echo "Done"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_REPO: ${{ github.repository }}
BRANCH: refs/pull/${{ github.event.pull_request.number }}/merge
@@ -0,0 +1,206 @@
# Daily audit for GHCR container packages that are unlinked from a source
# repository (`repository: null` on the GHCR API).
#
# WHY THIS EXISTS
# ---------------
# When a GHCR container package is not linked to a repository, workflow
# builds in `CopilotKit/CopilotKit` get `403 Forbidden` on push to GHCR —
# the workflow's `GITHUB_TOKEN` only has package-write permissions when
# the package is linked to the actor's repo. We hit this twice in quick
# succession (`showcase-harness` caught manually after a failed deploy, and
# `showcase-pocketbase` caught by a preemptive scan). There is NO GitHub
# API to programmatically link a package to a repo — it is UI-only — so
# the only way to prevent future surprises is detect drift early via a
# scheduled audit + Slack alert.
#
# ALLOWLIST (VERIFIED_ACCESS)
# ---------------------------
# Some packages show `repository: null` on the API but have working
# Actions access because the "Manage Actions access" setting was
# configured manually in the GitHub UI. There is NO API to detect this
# setting, so we maintain an explicit allowlist of package names that
# have been verified to have Actions write access. These are excluded
# from the unlinked-package alert to avoid false positives. See the
# VERIFIED_ACCESS array in the audit step below.
#
# This workflow is the operationalization of the lesson captured in
# `feedback_ghcr_new_package_403.md`.
#
# REQUIRED SECRETS
# ----------------
# - ORG_READ_PACKAGES_PAT: a fine-grained PAT with `read:packages` scope,
# org-scoped to `CopilotKit`. The default `secrets.GITHUB_TOKEN` does
# NOT have `read:packages` for the org and cannot list org packages.
# - SLACK_WEBHOOK_GHCR_DRIFT: a CopilotKit-internal Slack incoming-webhook
# URL. Posts to whichever channel the webhook is bound to (intended:
# an internal alerts channel).
#
# If `ORG_READ_PACKAGES_PAT` is unset the workflow fails loudly — drift
# detection silently disabled is worse than no workflow at all.
# If `SLACK_WEBHOOK_GHCR_DRIFT` is unset the workflow logs a warning
# (the audit still runs) so a missing webhook does not mask drift.
#
# EXIT CODES
# ----------
# This workflow exits 0 in all non-error cases (including when drift is
# present). The Slack message IS the alert; failing the workflow on
# drift would create noisy red CI checks on a schedule.
name: GHCR unlinked-package audit
on:
schedule:
# Daily at 14:00 UTC (07:00 PT / 10:00 ET) — low-traffic window,
# well before the US workday's deploy activity.
- cron: "0 14 * * *"
workflow_dispatch: {}
permissions:
contents: read
jobs:
audit:
name: Audit org container packages for unlinked repos
# Hoist the Slack webhook into an env var so step-level `if:`
# expressions can reference it — `secrets.*` is not a valid
# named-value inside `if:` and causes a workflow startup failure.
env:
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_GHCR_DRIFT }}
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Verify ORG_READ_PACKAGES_PAT is set
env:
PAT: ${{ secrets.ORG_READ_PACKAGES_PAT }}
run: |
if [ -z "$PAT" ]; then
echo "::error::ORG_READ_PACKAGES_PAT is not set. This workflow requires a fine-grained PAT with read:packages scope, org-scoped to CopilotKit. See the workflow header comment in .github/workflows/ghcr_unlinked_packages.yml for setup."
exit 1
fi
echo "ORG_READ_PACKAGES_PAT present."
- name: List org container packages and identify unlinked
id: audit
env:
GH_TOKEN: ${{ secrets.ORG_READ_PACKAGES_PAT }}
run: |
set -euo pipefail
# Page through all container packages in the CopilotKit org.
# `--paginate` follows Link headers; per_page=100 minimizes
# request count. `gh api` returns one JSON array per page;
# `--slurp` is unnecessary because gh concatenates pages into
# a single stream when called with `--paginate` on an array
# endpoint.
all_packages_json="$(gh api \
--paginate \
-H "Accept: application/vnd.github+json" \
"/orgs/CopilotKit/packages?package_type=container&per_page=100")"
total="$(echo "$all_packages_json" | jq 'length')"
echo "Total container packages in CopilotKit org: $total"
# Filter for packages where repository is null. Emit a compact
# JSON array of {name, visibility} objects for downstream use.
unlinked_json="$(echo "$all_packages_json" \
| jq -c '[.[] | select(.repository == null) | {name: .name, visibility: .visibility}]')"
# Packages with `repository: null` that have verified Actions access
# configured via the GitHub UI ("Manage Actions access" → CopilotKit →
# Write). These are not truly drifted — pushes work fine — but there
# is no API to detect this, so we maintain an explicit allowlist.
# To add a package here: verify in the package settings UI that
# "Manage Actions access" lists CopilotKit with Write role, then
# add the exact package name to this array.
VERIFIED_ACCESS=("showcase-pocketbase")
# Remove allowlisted packages from the unlinked set.
unlinked_json_before_filter="$unlinked_json"
if [ ${#VERIFIED_ACCESS[@]} -gt 0 ]; then
allowlist_filter=$(printf '"%s",' "${VERIFIED_ACCESS[@]}")
allowlist_filter="[${allowlist_filter%,}]"
unlinked_json="$(echo "$unlinked_json" \
| jq -c --argjson allow "$allowlist_filter" \
'[.[] | select(.name as $n | $allow | index($n) | not)]')"
fi
skipped=$(($(echo "$unlinked_json_before_filter" | jq 'length') - $(echo "$unlinked_json" | jq 'length')))
if [ "$skipped" -gt 0 ]; then
echo "Skipped $skipped package(s) with verified Actions access (allowlist)."
fi
unlinked_count="$(echo "$unlinked_json" | jq 'length')"
echo "Unlinked container packages: $unlinked_count"
# Emit outputs for the next step. Use the multiline-output
# delimiter form for the JSON array so jq output with embedded
# special chars survives intact.
{
echo "unlinked_count=$unlinked_count"
echo "unlinked_json<<EOF"
echo "$unlinked_json"
echo "EOF"
} >> "$GITHUB_OUTPUT"
if [ "$unlinked_count" = "0" ]; then
echo "::notice::No GHCR drift — all CopilotKit org container packages are linked to a repository."
else
echo "::warning::Detected $unlinked_count unlinked container package(s):"
echo "$unlinked_json" | jq -r '.[] | " - \(.name) (\(.visibility))"'
fi
- name: Build Slack payload
id: payload
if: steps.audit.outputs.unlinked_count != '0'
env:
UNLINKED_JSON: ${{ steps.audit.outputs.unlinked_json }}
UNLINKED_COUNT: ${{ steps.audit.outputs.unlinked_count }}
RUN_URL: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
set -euo pipefail
# Build a single `text` field with mrkdwn — keeps the payload
# compatible with both incoming-webhooks and channel webhooks.
# Each unlinked package gets a deep link to its Actions-access
# settings page, where the UI fix lives.
lines="$(echo "$UNLINKED_JSON" | jq -r '.[] | "• <https://github.com/orgs/CopilotKit/packages/container/\(.name)/settings|\(.name)> (\(.visibility))"')"
# Build message via printf — avoids heredoc indentation
# gotchas (closing delimiter must be column-0, which confuses
# YAML linters on `run: |` blocks). mrkdwn renders *bold*,
# _italic_, and <url|label> links.
nl=$'\n'
message=":warning: *GHCR drift detected* — ${UNLINKED_COUNT} container package(s) in the \`CopilotKit\` org are unlinked from a source repository.${nl}${nl}"
message="${message}${lines}${nl}${nl}"
message="${message}*UI fix (per package):* open the settings link above → *Manage Actions access* → *Add Repository* → \`CopilotKit/CopilotKit\` → *Write*.${nl}${nl}"
message="${message}_This drift breaks future workflow builds with \`403 Forbidden\` on push to GHCR. <${RUN_URL}|View audit run>_"
# Emit as a multiline output so the next step can consume it
# without re-quoting through a shell.
{
echo "text<<EOF"
echo "$message"
echo "EOF"
} >> "$GITHUB_OUTPUT"
- name: Notify Slack (drift detected)
if: steps.audit.outputs.unlinked_count != '0' && env.SLACK_WEBHOOK != ''
uses: slackapi/slack-github-action@0d95c9a7becc1e6e297d76df9bc735c44f4cbcbc # v3.0.5
with:
webhook: ${{ secrets.SLACK_WEBHOOK_GHCR_DRIFT }}
webhook-type: incoming-webhook
# Wrap the dynamic message via toJSON so quotes/newlines/
# backslashes inside package names or visibility values are
# safely JSON-encoded instead of breaking the payload.
payload: |
{ "text": ${{ toJSON(steps.payload.outputs.text) }} }
- name: Log (no Slack — webhook unset)
if: steps.audit.outputs.unlinked_count != '0' && env.SLACK_WEBHOOK == ''
run: |
echo "::warning::Drift detected but SLACK_WEBHOOK_GHCR_DRIFT is not set; no Slack notification sent. See run logs above for the unlinked package list."
- name: Log (no drift)
if: steps.audit.outputs.unlinked_count == '0'
run: |
echo "No drift — exiting 0."
+50
View File
@@ -0,0 +1,50 @@
name: "Integrations: Parity Check"
on:
pull_request:
paths:
- "examples/integrations/**"
- ".github/workflows/integrations_parity.yml"
push:
branches: [main]
paths:
- "examples/integrations/**"
- ".github/workflows/integrations_parity.yml"
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
parity-check:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22"
- name: Install pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
with:
run_install: false
- name: Install root dev dependencies (tsx)
run: pnpm install --frozen-lockfile --ignore-scripts --filter=.
- name: Verify integration-demo parity
run: |
echo "Checking examples/integrations/* against north-star..."
echo "If this fails, run: pnpm parity:sync --target=<instance>"
echo "and resolve agent-surface drift manually (see _parity/README.md)."
pnpm parity:check
@@ -0,0 +1,92 @@
name: Lint Release Workflows
# Runs actionlint + shellcheck against the release / create-pr, release /
# publish, and canary / publish pipelines and the scripts they call. Keeps
# these critical, retry-sensitive files from silently regressing on shell or
# action-syntax bugs.
#
# Scope is intentionally narrow: only the release workflows and
# scripts/release/*. Expanding later is cheap; starting narrow avoids
# drowning unrelated changes in pre-existing lint noise.
#
# Workflow mapping (cpk):
# release / create-pr -> .github/workflows/stable-release.yml
# release / publish -> .github/workflows/publish-release.yml
# canary / publish -> .github/workflows/canary.yml
on:
push:
branches: [main]
paths:
- ".github/workflows/stable-release.yml"
- ".github/workflows/publish-release.yml"
- ".github/workflows/canary.yml"
- ".github/workflows/lint-release-workflows.yml"
- "scripts/release/**"
- "release.config.json"
- "nx.json"
pull_request:
paths:
- ".github/workflows/stable-release.yml"
- ".github/workflows/publish-release.yml"
- ".github/workflows/canary.yml"
- ".github/workflows/lint-release-workflows.yml"
- "scripts/release/**"
- "release.config.json"
- "nx.json"
permissions:
contents: read
jobs:
actionlint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Run actionlint on release workflows
uses: reviewdog/action-actionlint@6fb7acc99f4a1008869fa8a0f09cfca740837d9d # v1.72.0
with:
reporter: github-check
level: error
fail_level: error
actionlint_flags: >-
.github/workflows/stable-release.yml
.github/workflows/publish-release.yml
.github/workflows/canary.yml
.github/workflows/lint-release-workflows.yml
shellcheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Install shellcheck
run: sudo apt-get update && sudo apt-get install -y shellcheck
- name: Run shellcheck on release scripts
run: |
set -euo pipefail
shopt -s nullglob globstar
files=(scripts/release/**/*.sh)
if [ ${#files[@]} -eq 0 ]; then
echo "No shell scripts under scripts/release/"
exit 0
fi
shellcheck --severity=warning "${files[@]}"
release-scope-dropdown-sync:
# Verifies the workflow_dispatch `scope` choice dropdowns in
# stable-release.yml, publish-release.yml, and canary.yml match
# release.config.json's `.scopes` keys. These option lists are
# hand-maintained and drift from the config (newly-enrolled packages
# weren't canary-selectable; stale scopes lingered), so this guard fails
# CI whenever they diverge again.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Verify release scope dropdowns match release.config.json
run: bash scripts/release/verify-release-scope-dropdowns.sh
+59
View File
@@ -0,0 +1,59 @@
name: plugin-skills-check
on:
push:
branches: [main]
paths:
- "packages/*/skills/**"
- "skills/runtime/**"
- "skills/react-core/**"
- "skills/a2ui-renderer/**"
- "scripts/sync-plugin-skills.ts"
- "scripts/__tests__/sync-plugin-skills.test.ts"
- ".claude-plugin/**"
- ".github/workflows/plugin-skills-check.yml"
# The plugin version pins to packages/runtime/package.json, so a release
# bump there must re-trigger the drift check or the pin silently rots.
- "packages/runtime/package.json"
pull_request:
paths:
- "packages/*/skills/**"
- "skills/runtime/**"
- "skills/react-core/**"
- "skills/a2ui-renderer/**"
- "scripts/sync-plugin-skills.ts"
- "scripts/__tests__/sync-plugin-skills.test.ts"
- ".claude-plugin/**"
- ".github/workflows/plugin-skills-check.yml"
- "packages/runtime/package.json"
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
check:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20
cache: pnpm
- run: pnpm install --frozen-lockfile
- name: Run sync script unit tests
run: pnpm exec vitest run scripts/__tests__/sync-plugin-skills.test.ts
- name: Check plugin skill mirror is in sync
run: pnpm check:plugin-skills
+65
View File
@@ -0,0 +1,65 @@
name: 🚀 pkg-pr-new
on:
push:
paths:
- "packages/**"
pull_request:
paths:
- "packages/**"
permissions:
contents: read
concurrency:
group: ${{ github.repository }}-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
NX_VERBOSE_LOGGING: true
NX_CI_EXECUTION_ID: ${{ github.head_ref }}-${{ github.sha }}-${{ github.run_attempt }}
NX_CI_EXECUTION_ENV: "Publish Commit"
jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 15
# NOTE: deliberately NOT in the `npm` environment. pkg-pr-new publishes to
# pkg.pr.new (not the npm registry) and uses no environment-scoped secrets,
# and this job runs on every push/PR touching packages/** — the npm
# environment's deployment-branch policy (main, canary/*,
# release/publish/*) would block it.
permissions:
contents: read
# pkg-pr-new posts snapshot comments on the PR using the workflow token
pull-requests: write
steps:
# persist-credentials required: pkg-pr-new uses repo token to post snapshot comments on the PR
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- name: Install pnpm
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- run: corepack enable
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version-file: "package.json"
# setup-node built-in cache is fork-safe (fork PRs can't write to base repo cache)
cache: "pnpm"
cache-dependency-path: "**/pnpm-lock.yaml"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Configure Nx Cloud environment
run: |
echo "NX_CI_EXECUTION_ID=${{ github.run_id }}-${{ github.run_attempt }}-pkg-pr-new" >> $GITHUB_ENV
echo "NX_CLOUD_NO_TIMEOUTS=true" >> $GITHUB_ENV
echo "NX_CLOUD_DISTRIBUTED_EXECUTION=false" >> $GITHUB_ENV
echo "NX_NO_CLOUD=true" >> $GITHUB_ENV
- name: Build
run: pnpm run build
- run: npx pkg-pr-new publish --pnpm --packageManager pnpm "./packages/*"
+873
View File
@@ -0,0 +1,873 @@
# release / publish
#
# Single npm OIDC entry point for both stable releases and prerelease canaries.
# npm trusted publisher records for the monorepo packages plus independently
# scoped @copilotkit packages are registered against THIS workflow file.
# Matching happens on the OIDC token's `workflow_ref` claim, which is always
# publish-release.yml when this workflow is the entry point.
#
# Triggers:
# - pull_request: closed on a release/publish/<scope>/v<X.Y.Z> branch → stable
# release of <scope> at version <X.Y.Z> (the normal flow).
# - workflow_dispatch with mode=stable → manual retrigger of a failed stable
# release. Republishes from the latest commit on main. Only use this when
# the normal flow failed BEFORE npm publish succeeded.
# - workflow_dispatch with mode=prerelease → canary publish. Bumps versions
# in the build job to <X.Y.Z>-canary.<suffix>, publishes with --tag canary,
# skips tag push + GH Release + Notion notification.
name: release / publish
# This workflow handles two independent release lanes:
#
# 1. npm (TypeScript) — fires on merged release/publish/* PRs or manual dispatch.
# Build → publish via nx release + OIDC trusted publishers.
#
# 2. PyPI (Python SDK) — fires on any merged PR that bumps sdk-python/pyproject.toml.
# Detects version change vs PyPI registry, builds with poetry, publishes with uv.
# Ported from ag-ui's publish-release.yml Python lane.
on:
pull_request:
types: [closed]
branches: [main]
workflow_dispatch:
inputs:
scope:
description: "What to release"
required: true
type: choice
options:
- monorepo
- angular
- channels
- channels-discord
- channels-intelligence
- channels-slack
- channels-teams
- channels-telegram
- channels-whatsapp
mode:
description: "Release mode: stable (full release with tag + GH Release) or prerelease (canary, no tag/release)"
required: false
default: stable
type: choice
options:
- stable
- prerelease
suffix:
description: "Canary suffix (only used when mode=prerelease). Falls back to timestamp if empty. Allowed: [a-zA-Z0-9._-]+"
required: false
type: string
default: ""
dry-run:
description: "Dry run (skip publish step)"
required: false
default: false
type: boolean
python_publish:
description: "Run the Python publish lane regardless of which files changed. Still no-ops if sdk-python's version already matches PyPI."
required: false
default: false
type: boolean
permissions:
contents: read
concurrency:
# Scope the lock to the package being released so a `monorepo` publish and an
# `angular` publish run in independent lanes instead of queuing behind each
# other. On the manual path `inputs.scope` carries the target; on the merged
# release-PR path inputs are empty, but the PR branch is
# `release/publish/<scope>/v<version>`, so `github.head_ref` already encodes
# the scope. Same-scope runs still serialize (cancel-in-progress: false),
# which is what protects the tag push / npm publish.
group: publish-release-${{ inputs.scope || github.head_ref || github.ref }}
cancel-in-progress: false
env:
NX_VERBOSE_LOGGING: true
jobs:
build:
# Run on a merged release PR (normal flow), a stable manual dispatch from
# main (retry escape hatch), or a prerelease manual dispatch from any
# selected branch. Canary publishes are intentionally branch-scoped so
# maintainers can push a button on feature work without merging first.
if: >
(github.event_name == 'workflow_dispatch' &&
(inputs.mode == 'prerelease' || github.ref == 'refs/heads/main')) ||
(github.event.pull_request.merged == true &&
startsWith(github.event.pull_request.head.ref, 'release/publish/'))
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: read
steps:
- name: Determine scope and mode
id: meta
env:
PR_HEAD_REF: ${{ github.event.pull_request.head.ref }}
INPUT_SCOPE: ${{ inputs.scope }}
INPUT_MODE: ${{ inputs.mode }}
run: |
set -euo pipefail
if [ -n "$INPUT_SCOPE" ]; then
SCOPE="$INPUT_SCOPE"
else
# Branch format: release/publish/<scope>/v<version>
SCOPE="${PR_HEAD_REF#release/publish/}"
SCOPE="${SCOPE%%/v*}"
fi
MODE="${INPUT_MODE:-stable}"
echo "scope=$SCOPE" >> "$GITHUB_OUTPUT"
echo "mode=$MODE" >> "$GITHUB_OUTPUT"
echo "Detected scope: $SCOPE, mode: $MODE"
# No token/credential persistence: the publish job sets up its own
# `git config insteadOf` with secrets.GITHUB_TOKEN before pushing tags,
# so this checkout doesn't need write access. Critically, the
# subsequent `Upload workspace` step packs the entire checkout
# (including .git/config) into an artifact — persisting credentials
# here would leak a workflow-scoped token to anyone with actions:read.
- name: Checkout Repo
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
fetch-depth: 0
persist-credentials: false
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Setup Node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
- name: Install Dependencies
run: pnpm install --frozen-lockfile
# Validate user-supplied suffix against npm-safe charset before passing
# to bump-prerelease.ts. Empty suffix → omit the --suffix flag entirely
# so the script applies its timestamp fallback (passing an empty string
# would produce a version like "X.Y.Z-canary." with a trailing dot).
- name: Bump prerelease versions
if: ${{ steps.meta.outputs.mode == 'prerelease' }}
env:
INPUT_SCOPE: ${{ inputs.scope }}
INPUT_SUFFIX: ${{ inputs.suffix }}
run: |
set -euo pipefail
if [ -n "$INPUT_SUFFIX" ]; then
if ! [[ "$INPUT_SUFFIX" =~ ^[a-zA-Z0-9._-]+$ ]]; then
echo "::error::Invalid suffix '$INPUT_SUFFIX'. Allowed: [a-zA-Z0-9._-]+"
exit 1
fi
pnpm tsx scripts/release/bump-prerelease.ts --scope "$INPUT_SCOPE" --suffix "$INPUT_SUFFIX"
else
pnpm tsx scripts/release/bump-prerelease.ts --scope "$INPUT_SCOPE"
fi
- name: Build packages
run: pnpm run build
# Strip caches and pack the workspace into a single tarball before
# upload. upload-artifact's path filters are post-walk: it still
# descends into every node_modules and stats every file (~6.4M for
# this monorepo with pnpm's .pnpm/ symlink farm) before applying
# negations, which is the actual bottleneck. Removing the dirs and
# uploading one file collapses that to a single fast step.
- name: Pack workspace
run: |
find . -type d \( -name node_modules -o -name .nx -o -name .turbo -o -name .next \) -prune -exec rm -rf {} +
tar -czf /tmp/workspace.tgz .
- name: Upload workspace
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: workspace
path: /tmp/workspace.tgz
retention-days: 1
outputs:
scope: ${{ steps.meta.outputs.scope }}
mode: ${{ steps.meta.outputs.mode }}
publish:
needs: build
if: ${{ !cancelled() && needs.build.result == 'success' }}
runs-on: ubuntu-latest
timeout-minutes: 20
# npm trusted publishing is bound to this environment. Its deployment
# branch policy must allow prerelease workflow_dispatch refs; stable
# releases remain main-only via the build job guard.
environment: npm
permissions:
contents: write
id-token: write
steps:
- name: Determine scope and mode
id: meta
env:
PR_HEAD_REF: ${{ github.event.pull_request.head.ref }}
INPUT_SCOPE: ${{ inputs.scope }}
INPUT_MODE: ${{ inputs.mode }}
run: |
set -euo pipefail
if [ -n "$INPUT_SCOPE" ]; then
SCOPE="$INPUT_SCOPE"
else
# Branch format: release/publish/<scope>/v<version>
SCOPE="${PR_HEAD_REF#release/publish/}"
SCOPE="${SCOPE%%/v*}"
fi
if [ -z "$SCOPE" ]; then
echo "::error::Failed to resolve scope (input=$INPUT_SCOPE, ref=$PR_HEAD_REF)"
exit 1
fi
MODE="${INPUT_MODE:-stable}"
echo "scope=$SCOPE" >> "$GITHUB_OUTPUT"
echo "mode=$MODE" >> "$GITHUB_OUTPUT"
- name: Download workspace
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: workspace
- name: Unpack workspace
run: |
tar -xzf workspace.tgz
rm workspace.tgz
- name: Configure git credentials
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
git config --local url."https://x-access-token:${GH_TOKEN}@github.com/".insteadOf "https://github.com/"
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Setup Node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
registry-url: https://registry.npmjs.org
# Restore node_modules — the build job excludes them from the
# uploaded workspace artifact (see "Upload workspace" above). Uses
# pnpm-lock.yaml from the artifact, so this is a deterministic
# restore of exactly what the build job ran with.
- name: Install Dependencies
run: pnpm install --frozen-lockfile
- name: Dry-run notice
if: ${{ inputs.dry-run == true }}
run: |
{
echo "## Dry Run"
echo ""
echo "DRY RUN — skipping publish step. Scope: ${{ steps.meta.outputs.scope }}, mode: ${{ steps.meta.outputs.mode }}."
} >> "$GITHUB_STEP_SUMMARY"
- name: Publish to npm
id: publish
if: ${{ inputs.dry-run != true }}
env:
NODE_AUTH_TOKEN: ""
NOTION_API_KEY: ${{ steps.meta.outputs.mode == 'stable' && secrets.NOTION_API_KEY || '' }}
PUBLISH_SCRIPT: ${{ steps.meta.outputs.mode == 'prerelease' && 'prerelease.ts' || 'publish-release.ts' }}
SCOPE: ${{ steps.meta.outputs.scope }}
run: pnpm tsx "scripts/release/$PUBLISH_SCRIPT" --scope "$SCOPE"
- name: Verify publish step emitted version
if: ${{ success() && inputs.dry-run != true }}
env:
MODE: ${{ steps.meta.outputs.mode }}
VERSION: ${{ steps.publish.outputs.version }}
run: |
set -euo pipefail
if [ -z "$VERSION" ]; then
if [ "$MODE" = "prerelease" ]; then
echo "::error::prerelease.ts did not emit 'version' output to GITHUB_OUTPUT. The Prerelease summary would render a blank Version field; aborting."
else
echo "::error::publish-release.ts did not emit 'version' output to GITHUB_OUTPUT. Tag/release creation would produce malformed artifacts; aborting."
fi
exit 1
fi
echo "VERSION=$VERSION confirmed (mode=$MODE)"
- name: Configure git user
if: ${{ success() && inputs.dry-run != true && steps.meta.outputs.mode != 'prerelease' }}
run: |
git config --local user.email "github-actions[bot]@users.noreply.github.com"
git config --local user.name "github-actions[bot]"
- name: Check for pre-existing tags
if: ${{ success() && inputs.dry-run != true && steps.meta.outputs.mode != 'prerelease' }}
env:
SCOPE: ${{ steps.meta.outputs.scope }}
VERSION: ${{ steps.publish.outputs.version }}
run: |
if [ "$SCOPE" == "monorepo" ]; then
TAG="v${VERSION}"
else
TAG="${SCOPE}/v${VERSION}"
fi
if git rev-parse "$TAG" >/dev/null 2>&1; then
echo "ERROR: Tag $TAG already exists" >&2
exit 1
fi
- name: Create and push git tag
if: ${{ success() && inputs.dry-run != true && steps.meta.outputs.mode != 'prerelease' }}
env:
SCOPE: ${{ steps.meta.outputs.scope }}
VERSION: ${{ steps.publish.outputs.version }}
run: |
if [ "$SCOPE" == "monorepo" ]; then
TAG="v${VERSION}"
else
TAG="${SCOPE}/v${VERSION}"
fi
git tag -a "$TAG" -m "Release ${SCOPE} ${VERSION}"
git push origin "$TAG"
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
id: tag
- name: Create GitHub Release
if: ${{ success() && inputs.dry-run != true && steps.meta.outputs.mode != 'prerelease' }}
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
env:
RELEASE_TAG: ${{ steps.tag.outputs.tag }}
RELEASE_SCOPE: ${{ steps.meta.outputs.scope }}
RELEASE_VERSION: ${{ steps.publish.outputs.version }}
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require("fs");
const { owner, repo } = context.repo;
const tag = process.env.RELEASE_TAG;
const scope = process.env.RELEASE_SCOPE;
const version = process.env.RELEASE_VERSION;
const name = scope === "monorepo" ? `v${version}` : `${scope}/v${version}`;
let body = "";
try {
body = fs.readFileSync("./release-notes.md", "utf8");
} catch {
body = `Release ${name}`;
}
try {
const existing = await github.rest.repos.getReleaseByTag({ owner, repo, tag });
await github.rest.repos.updateRelease({
owner, repo,
release_id: existing.data.id,
tag_name: tag, name, body,
draft: false, prerelease: false,
});
} catch (error) {
if (error.status !== 404) throw error;
await github.rest.repos.createRelease({
owner, repo,
tag_name: tag, name, body,
draft: false, prerelease: false,
});
}
- name: Release summary (stable)
if: ${{ success() && inputs.dry-run != true && steps.meta.outputs.mode != 'prerelease' }}
run: |
{
echo "## Release Published"
echo ""
echo "**Scope:** ${{ steps.meta.outputs.scope }}"
echo "**Mode:** ${{ steps.meta.outputs.mode }}"
echo "**Version:** ${{ steps.publish.outputs.version }}"
echo "**Tag:** ${{ steps.tag.outputs.tag }}"
} >> "$GITHUB_STEP_SUMMARY"
- name: Prerelease summary
if: ${{ success() && inputs.dry-run != true && steps.meta.outputs.mode == 'prerelease' }}
run: |
{
echo "## Prerelease Published"
echo ""
echo "**Scope:** ${{ steps.meta.outputs.scope }}"
echo "**Version:** ${{ steps.publish.outputs.version }}"
echo "**Tag:** (prerelease — no tag created)"
} >> "$GITHUB_STEP_SUMMARY"
- name: Dry-run summary
if: ${{ success() && inputs.dry-run == true }}
run: |
{
echo "## Dry Run Completed"
echo ""
echo "**Scope:** ${{ steps.meta.outputs.scope }}"
echo "**Mode:** ${{ steps.meta.outputs.mode }}"
echo "- Publish step was skipped; no npm publish, no git tag, no GitHub Release."
} >> "$GITHUB_STEP_SUMMARY"
# Populated only on a stable, non-dry-run success (the publish/tag steps are
# gated on mode != prerelease && dry-run != true). On prerelease, dry-run, or
# failure these are empty — the notify job gates on that emptiness.
outputs:
version: ${{ steps.publish.outputs.version }}
tag: ${{ steps.tag.outputs.tag }}
# ===========================================================================
# Python SDK publish lane
#
# Fires independently of the npm lane. Detects whether sdk-python/pyproject.toml
# has a version newer than what's on PyPI, builds with poetry, publishes with uv.
#
# SECURITY: Same build/publish separation as the npm lane — PYPI_API_TOKEN is
# only available in the publish-python job, never where poetry install runs.
# ===========================================================================
build-python:
# Fires when:
# 1. A PR merging to main touched sdk-python/pyproject.toml (version bump), OR
# 2. Manual dispatch with python_publish=true
if: >
(github.event_name == 'workflow_dispatch' && inputs.python_publish == true) ||
(github.event_name == 'pull_request' && github.event.pull_request.merged == true)
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
outputs:
should_publish: ${{ steps.detect.outputs.should_publish }}
version: ${{ steps.detect.outputs.version }}
name: ${{ steps.detect.outputs.name }}
# Earliest Python-release intent signal: emitted by the `changed` step
# BEFORE any failure-prone step (setup-python, detect, build). The notify
# job gates the PyPI FAILURE alert on this (not should_publish, which is
# emitted only at the END of detect) so a build-python failure at/before
# detect on a genuine release still pages instead of being silently
# swallowed.
pyproject_changed: ${{ steps.changed.outputs.pyproject_changed }}
steps:
- name: Checkout merged main
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
fetch-depth: 0
ref: main
persist-credentials: false
# For PRs, skip early if this PR didn't touch pyproject.toml. Manual
# dispatch always continues (the user explicitly asked for it).
- name: Check if pyproject.toml changed in this PR
if: github.event_name == 'pull_request'
id: changed
env:
PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
PR_HEAD_SHA: ${{ github.event.pull_request.merge_commit_sha }}
run: |
set -euo pipefail
if [ -z "$PR_BASE_SHA" ]; then
echo "::error::PR_BASE_SHA is empty — cannot determine PR base for diff. Refusing to silently skip Python publish."
exit 1
fi
if [ -z "$PR_HEAD_SHA" ]; then
echo "::error::PR_HEAD_SHA (merge_commit_sha) is empty — GitHub may not have computed the merge commit yet. Refusing to silently skip Python publish; rerun the workflow."
exit 1
fi
# Capture diff FIRST so a git failure trips set -e and fails loudly,
# rather than producing an empty pipe that grep silently routes to
# "not changed" — that path masked real version bumps before.
CHANGED="$(git diff --name-only "$PR_BASE_SHA" "$PR_HEAD_SHA")"
# grep -q exits 1 on legitimate no-match; guard with `if` so set -e
# doesn't kill the step on that expected case.
if printf '%s\n' "$CHANGED" | grep -q '^sdk-python/pyproject.toml$'; then
echo "pyproject_changed=true" >> "$GITHUB_OUTPUT"
else
echo "pyproject_changed=false" >> "$GITHUB_OUTPUT"
echo "sdk-python/pyproject.toml not changed in this PR — skipping Python publish"
fi
- name: Set up Python
if: github.event_name == 'workflow_dispatch' || steps.changed.outputs.pyproject_changed == 'true'
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with:
python-version: "3.12"
- name: Detect version change
if: github.event_name == 'workflow_dispatch' || steps.changed.outputs.pyproject_changed == 'true'
id: detect
run: ./scripts/release/detect-py-version-changes.sh
- name: Install Poetry
if: steps.detect.outputs.should_publish == 'true'
uses: snok/install-poetry@a783c322200f0519c7926aa6faa857c4e23e9263 # v1.4.2
with:
version: latest
virtualenvs-create: true
virtualenvs-in-project: true
- name: Build Python package
if: steps.detect.outputs.should_publish == 'true'
working-directory: sdk-python
run: poetry build
- name: Upload Python build artifacts
if: steps.detect.outputs.should_publish == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: py-build-artifacts
path: sdk-python/dist/
retention-days: 1
- name: Nothing to publish
if: steps.detect.outputs.should_publish != 'true'
run: |
{
echo "## Python SDK"
echo ""
echo "No version change detected — nothing to publish."
} >> "$GITHUB_STEP_SUMMARY"
# WARNING: PyPI trusted-publisher binding pins to:
# repository: CopilotKit/CopilotKit
# workflow_file: publish-release.yml
# environment: pypi
# Renaming this file, changing this job's `environment:` value, or moving the
# publish step into another workflow breaks PyPI publishing with HTTP 422
# until the Trusted Publisher record on pypi.org is updated to match.
publish-python:
needs: build-python
if: ${{ !cancelled() && needs.build-python.result == 'success' && needs.build-python.outputs.should_publish == 'true' && inputs.dry-run != true }}
runs-on: ubuntu-latest
timeout-minutes: 10
environment: pypi
permissions:
contents: write
id-token: write
steps:
- name: Checkout merged main
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
fetch-depth: 0
ref: main
persist-credentials: false
- name: Install uv
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
with:
version: ">=0.8.0"
- name: Download Python build artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: py-build-artifacts
path: sdk-python/dist
- name: Publish to PyPI (OIDC trusted publishing)
run: |
set -euo pipefail
shopt -s nullglob
files=(sdk-python/dist/*)
if [ ${#files[@]} -eq 0 ]; then
echo "::error::no build artifacts in sdk-python/dist — nothing to publish"
exit 1
fi
uv publish --trusted-publishing always "${files[@]}"
- name: Verify version is live on PyPI
env:
NAME: ${{ needs.build-python.outputs.name }}
VERSION: ${{ needs.build-python.outputs.version }}
run: |
set -euo pipefail
for i in $(seq 1 18); do
if curl -fsS "https://pypi.org/pypi/${NAME}/${VERSION}/json" >/dev/null 2>&1; then
echo "Confirmed ${NAME}==${VERSION} on PyPI"; exit 0
fi
echo "Attempt ${i}: ${NAME}==${VERSION} not visible yet; retrying in 10s..."
sleep 10
done
echo "::error::${NAME}==${VERSION} did not appear on PyPI within 180s"
echo "Last curl response:"
curl -sS "https://pypi.org/pypi/${NAME}/${VERSION}/json" 2>&1 | tail -n 5 || true
exit 1
- name: Configure git
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
git config --local user.email "github-actions[bot]@users.noreply.github.com"
git config --local user.name "github-actions[bot]"
git config --local url."https://x-access-token:${GH_TOKEN}@github.com/".insteadOf "https://github.com/"
- name: Create and push git tag
id: tag
env:
PY_VERSION: ${{ needs.build-python.outputs.version }}
PY_NAME: ${{ needs.build-python.outputs.name }}
run: |
set -euo pipefail
TAG="python-sdk/v${PY_VERSION}"
if git rev-parse "$TAG" >/dev/null 2>&1; then
echo "Tag $TAG already exists — skipping"
else
git tag -a "$TAG" -m "Release ${PY_NAME} ${PY_VERSION}"
git push origin "$TAG"
fi
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
- name: Create GitHub Release
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
env:
RELEASE_TAG: ${{ steps.tag.outputs.tag }}
RELEASE_VERSION: ${{ needs.build-python.outputs.version }}
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { owner, repo } = context.repo;
const tag = process.env.RELEASE_TAG;
const version = process.env.RELEASE_VERSION;
const name = `python-sdk/v${version}`;
const body = `Python SDK release: copilotkit ${version}\n\nhttps://pypi.org/project/copilotkit/${version}/`;
try {
const existing = await github.rest.repos.getReleaseByTag({ owner, repo, tag });
await github.rest.repos.updateRelease({
owner, repo,
release_id: existing.data.id,
tag_name: tag, name, body,
draft: false, prerelease: false,
});
} catch (error) {
if (error.status !== 404) throw error;
await github.rest.repos.createRelease({
owner, repo,
tag_name: tag, name, body,
draft: false, prerelease: false,
});
}
- name: Release summary
env:
PY_VERSION: ${{ needs.build-python.outputs.version }}
run: |
{
echo "## Python SDK Published"
echo ""
echo "- \`copilotkit@${PY_VERSION}\`"
echo "- https://pypi.org/project/copilotkit/${PY_VERSION}/"
} >> "$GITHUB_STEP_SUMMARY"
# ===========================================================================
# Slack #engr notification
#
# A single concise post to #engr when a release publishes (or fails).
# Runs after both lanes regardless of their outcome (`if: always()`). The
# load-bearing truth table lives in the unit-tested pure builder at
# scripts/release/lib/build-release-notification.ts; this job only feeds it the
# needs.* signals and posts what it returns. Suppressed entirely for canaries
# (mode=prerelease) and dry-runs — the builder returns should_post=false there.
# Webhook empty-guard mirrors showcase_validate.yml so an unset
# SLACK_WEBHOOK_ENGR secret does not break the shell or red this step.
# ===========================================================================
notify:
needs: [build, publish, build-python, publish-python]
# always() so we still report on a failed lane, but guard on a real release
# context: a workflow_dispatch (manual release) OR a *merged* PR. A
# closed-unmerged PR is not a release attempt and must not notify.
if: >
always() &&
(github.event_name == 'workflow_dispatch' ||
github.event.pull_request.merged == true)
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
env:
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_ENGR }}
steps:
# Determine release intent IN THIS JOB, from the github.event payload +
# the PR changed-files API — NOT from needs.build*/needs.build-python
# outputs. The build jobs emit their intent signals (should_publish,
# pyproject_changed) AFTER failure-prone steps (SHA guards, setup-python,
# the PyPI version-compare), so a build job that dies before emitting them
# on a REAL release would leave the intent empty → no alert. Computing
# intent here, independent of whether the build jobs ran at all, closes
# that silent-swallow class. The builder gates the npm/PyPI FAILURE arms on
# these signals.
#
# This step runs FIRST — before Checkout/Setup/Install — on purpose:
# computing intent before any infra step means steps.intent.outputs.* are
# always populated even if a later infra step (the dependency install,
# checkout, or setup) fails. The failure() self-alert below gates its
# best-effort Slack post on these outputs, so running intent first
# guarantees that gate can still fire when the notify job dies during
# install — exactly the silent-swallow the self-watchdog exists to prevent.
# It needs only `gh api` (preinstalled), the github.event context, and
# GITHUB_TOKEN (in its own env), so it has no dependency on checkout/deps.
- name: Determine release intent
id: intent
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
# npm intent: a manual dispatch, or a merged release/publish/* PR.
# Computed purely from event-context expressions (no API needed).
NPM_INTENDED="${{ (github.event_name == 'workflow_dispatch' || (github.event.pull_request.merged == true && startsWith(github.event.pull_request.head.ref, 'release/publish/'))) && 'true' || 'false' }}"
echo "npm_intended=$NPM_INTENDED" >> "$GITHUB_OUTPUT"
# Python intent: a python_publish dispatch, OR a merged PR that changed
# sdk-python/pyproject.toml (per the GitHub PR changed-files API —
# robust to an uncomputed local merge_commit_sha). Default false.
PY_INTENDED="false"
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
if [ "${{ inputs.python_publish }}" = "true" ]; then PY_INTENDED="true"; fi
elif [ "${{ github.event.pull_request.merged }}" = "true" ]; then
# Query the merged PR's changed files. Fail TOWARD paging: if the API
# call fails on a merged PR, default PY_INTENDED=true (never toward
# silence) and emit a ::warning::.
if FILES="$(gh api "repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/files" --paginate --jq '.[].filename' 2>/dev/null)"; then
if printf '%s\n' "$FILES" | grep -qx 'sdk-python/pyproject.toml'; then PY_INTENDED="true"; fi
else
echo "::warning::Could not list changed files for PR #${{ github.event.pull_request.number }} via the GitHub API — defaulting py_intended=true (fail toward paging, never toward silence)."
PY_INTENDED="true"
fi
fi
echo "py_intended=$PY_INTENDED" >> "$GITHUB_OUTPUT"
echo "npm_intended=$NPM_INTENDED py_intended=$PY_INTENDED"
- name: Checkout Repo
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Setup pnpm
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Setup Node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
# Restore node_modules so `pnpm tsx` (and the release-config import the
# notifier does) resolves. The build/publish jobs install before running
# tsx for the same reason; without this the notify step fails and NO
# release notification can ever post.
- name: Install Dependencies
run: pnpm install --frozen-lockfile
# Compute the scope-correct npm URL: the monorepo packages live under the
# @copilotkit org page, while single-package scopes link to their package.
- name: Resolve npm URL for scope
id: npmurl
env:
SCOPE: ${{ needs.build.outputs.scope }}
run: |
set -euo pipefail
case "$SCOPE" in
angular)
NPM_URL="https://www.npmjs.com/package/@copilotkit/angular"
;;
*)
NPM_URL="https://www.npmjs.com/org/copilotkit"
;;
esac
echo "npm_url=$NPM_URL" >> "$GITHUB_OUTPUT"
- name: Build notification message
id: build
env:
MODE: ${{ needs.build.outputs.mode }}
NPM_RESULT: ${{ needs.publish.result }}
NPM_VER: ${{ needs.publish.outputs.version }}
BUILD_RESULT: ${{ needs.build.result }}
# Event-derived release intent computed in the `intent` step above
# (independent of the build jobs). The builder gates the npm/PyPI
# FAILURE arms on these so a build-job failure on a genuine release
# always pages, even if the build jobs emitted no usable outputs.
NPM_INTENDED: ${{ steps.intent.outputs.npm_intended }}
PY_INTENDED: ${{ steps.intent.outputs.py_intended }}
# should_publish still legitimately gates the PyPI SUCCESS arm (a real
# success means detect ran and emitted it).
PY_PUB: ${{ needs.build-python.outputs.should_publish }}
PY_RESULT: ${{ needs.publish-python.result }}
PY_BUILD_RESULT: ${{ needs.build-python.result }}
PY_VER: ${{ needs.build-python.outputs.version }}
SCOPE: ${{ needs.build.outputs.scope }}
DRY_RUN: ${{ inputs.dry-run }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
# Empty when no tag was created (non-stable / dry-run). The builder's
# empty-releaseUrl guard that consumes this is retained as
# DEFENSE-IN-DEPTH: the empty-releaseUrl-on-SUCCESS state is NOT
# currently reachable — the tag step is `if: success()`, so a tag-step
# failure flips the publish JOB to `failure` and routes to the failure
# arm rather than rendering an empty link. The guard protects against a
# FUTURE change making the tag step continue-on-error (publish success
# + empty tag output), which would otherwise render a broken empty
# "<|Release notes>" / "/releases/tag/" link — do NOT remove it.
RELEASE_URL: ${{ needs.publish.outputs.tag && format('{0}/{1}/releases/tag/{2}', github.server_url, github.repository, needs.publish.outputs.tag) || '' }}
NPM_URL: ${{ steps.npmurl.outputs.npm_url }}
# Empty-version guard, mirroring RELEASE_URL above: with no version
# the per-version PyPI URL would be a broken ".../copilotkit//" link,
# so fall back to the project root page.
PY_URL: ${{ needs.build-python.outputs.version && format('https://pypi.org/project/copilotkit/{0}/', needs.build-python.outputs.version) || 'https://pypi.org/project/copilotkit/' }}
run: pnpm tsx scripts/release/build-release-notification.ts
- name: Post to #engr
if: ${{ steps.build.outputs.should_post == 'true' && env.SLACK_WEBHOOK != '' && inputs.dry-run != true }}
uses: slackapi/slack-github-action@0d95c9a7becc1e6e297d76df9bc735c44f4cbcbc # v3.0.5
with:
webhook: ${{ secrets.SLACK_WEBHOOK_ENGR }}
webhook-type: incoming-webhook
payload: |
{
"text": ${{ toJSON(steps.build.outputs.message) }}
}
- name: Log (no Slack — webhook unset)
if: ${{ steps.build.outputs.should_post == 'true' && env.SLACK_WEBHOOK == '' }}
run: |
echo "::warning::A release notification was ready to post but SLACK_WEBHOOK_ENGR is not set; no Slack notification sent."
# Self-watchdog: if any earlier step in THIS job failed (e.g. the
# dependency install or the builder crashed), the notifier itself is the
# thing that broke — so a real release alert could be silently swallowed.
# Emit a ::error:: and, when the webhook is configured, a minimal
# best-effort Slack post so the failure isn't completely invisible.
- name: Notifier failed — self-alert
if: ${{ failure() }}
run: |
echo "::error::The release notify job failed before it could post — a release alert may have been swallowed. Check this run: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
- name: Notifier failed — best-effort Slack
# Guard on dry-run: the self-watchdog Slack post must honor the same
# silence invariant as the real notification — a dry-run is silent
# EVERYWHERE, so a notify-job failure during one must not post a false
# red page.
#
# ALSO guard on real-release-context via the robust, build-job-INDEPENDENT
# intent computed in the `intent` step: the notify job runs on EVERY
# merged PR (always()), so a routine non-release merge whose notify job
# hits a transient install/builder flake would otherwise self-page even
# though no release was attempted. Only self-alert when a release was
# actually in flight (npm_intended OR py_intended). This replaces the old
# npm-biased `mode != 'prerelease'` + should_publish/pyproject_changed
# heuristic — which both relied on the build jobs' outputs (the very
# signals that may be empty if a build job died) AND would have suppressed
# a python_publish self-alert under a prerelease-mode dispatch. The intent
# gate is correct and robust. (The ::error:: echo step above stays
# unconditional — only the Slack POST needs these guards.)
if: ${{ failure() && env.SLACK_WEBHOOK != '' && inputs.dry-run != true && (steps.intent.outputs.npm_intended == 'true' || steps.intent.outputs.py_intended == 'true') }}
uses: slackapi/slack-github-action@0d95c9a7becc1e6e297d76df9bc735c44f4cbcbc # v3.0.5
with:
webhook: ${{ secrets.SLACK_WEBHOOK_ENGR }}
webhook-type: incoming-webhook
payload: |
{
"text": ${{ toJSON(format('🔴 *CopilotKit release notifier failed* — a release alert may have been swallowed · <{0}/{1}/actions/runs/{2}|View run>', github.server_url, github.repository, github.run_id)) }}
}
@@ -0,0 +1,106 @@
name: "Security: Fork PR Alert"
on:
pull_request:
types: [opened, synchronize, closed, reopened]
permissions:
pull-requests: write
contents: read
jobs:
fork-pr-monitor:
if: github.event.pull_request.head.repo.full_name != github.repository
runs-on: ubuntu-latest
steps:
- name: Check for suspicious patterns
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
const pr = context.payload.pull_request;
const alerts = [];
// 1. Check for [skip ci] in commit messages from fork PRs
if (context.payload.action === 'opened' || context.payload.action === 'synchronize') {
const commits = await github.rest.pulls.listCommits({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: pr.number,
per_page: 100
});
const skipCiCommits = commits.data.filter(c =>
/\[skip ci\]|\[ci skip\]|\[no ci\]/i.test(c.commit.message)
);
if (skipCiCommits.length > 0) {
alerts.push(`⚠️ **[skip ci] detected in fork PR** — ${skipCiCommits.length} commit(s) contain CI skip directives. Commits: ${skipCiCommits.map(c => c.sha.substring(0, 7)).join(', ')}`);
}
}
// 2. Check for force-push that reduces changed files to 0 (evidence cleanup)
if (context.payload.action === 'synchronize') {
const prDetails = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: pr.number
});
if (prDetails.data.changed_files === 0) {
alerts.push(`🚨 **Zero-file fork PR after force-push** — PR was force-pushed to show 0 changed files. This matches the TanStack attack cleanup pattern.`);
}
}
// 3. Check for rapid open-then-close (PR used only to trigger CI)
if (context.payload.action === 'closed' && !pr.merged) {
const created = new Date(pr.created_at);
const closed = new Date(pr.closed_at);
const minutesOpen = (closed - created) / (1000 * 60);
if (minutesOpen < 30) {
alerts.push(`🚨 **Fork PR closed rapidly** — opened and closed within ${Math.round(minutesOpen)} minutes without merging. May indicate a CI-trigger-only attack.`);
}
}
// 4. Check for large bundled files (>5000 lines) added by the PR
if (context.payload.action === 'opened' || context.payload.action === 'synchronize') {
const files = await github.rest.pulls.listFiles({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: pr.number,
per_page: 100
});
const largeNewFiles = files.data.filter(f =>
f.status === 'added' && f.additions > 5000
);
if (largeNewFiles.length > 0) {
alerts.push(`⚠️ **Large files added by fork PR** — ${largeNewFiles.map(f => '`' + f.filename + '` (' + f.additions + ' lines)').join(', ')}. Bundled payloads are a common supply-chain attack vector.`);
}
}
// Report alerts
if (alerts.length > 0) {
const body = [
'## 🔒 Supply Chain Security Alert',
'',
'This fork PR triggered the following security alerts:',
'',
alerts.join('\n\n'),
'',
'---',
'_Automated by supply-chain security monitor. See [TanStack incident](https://socket.dev/blog/tanstack-npm-packages-compromised-mini-shai-hulud-supply-chain-attack) for context._'
].join('\n');
// Post as PR comment
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number,
body: body
});
// Also set the action as failed annotation
core.warning(alerts.join(' | '));
}
+60
View File
@@ -0,0 +1,60 @@
name: security / zizmor
# zizmor runs static analysis over every workflow under .github/workflows
# looking for the well-known classes of GitHub Actions footguns: template
# injection from untrusted input, dangerous triggers like
# `pull_request_target`, unpinned `uses:` refs, excessive token scopes,
# secret exfil via job outputs, and a long tail of others.
#
# Findings at `low` confidence and above fail the job, so this acts as a
# blocking PR check. To deliberately allow a finding, suppress it in
# `.github/zizmor.yml` with a justification — never silently ignore.
on:
push:
branches: [main]
paths:
- ".github/workflows/**"
- ".github/actions/**"
- ".github/zizmor.yml"
pull_request:
paths:
- ".github/workflows/**"
- ".github/actions/**"
- ".github/zizmor.yml"
schedule:
# Catch findings introduced by newly-published advisories even when no
# workflow file changed this week.
- cron: "0 9 * * 1"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
zizmor:
name: Static analysis (zizmor)
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
# SARIF upload requires `security-events: write`, but we currently
# rely on the action's own annotations. Keep contents:read only.
contents: read
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Run zizmor
# `min-severity: low` blocks PRs on the broadest set of findings
# without flagging hypothetical-only `informational` notes. Drop
# to `medium` if low-severity churn becomes a problem.
uses: zizmorcore/zizmor-action@192e21d79ab29983730a13d1382995c2307fbcaa # v0.5.7
with:
min-severity: low
advanced-security: false
config: .github/zizmor.yml
+998
View File
@@ -0,0 +1,998 @@
name: "Showcase: Build & Push"
# Decoupled from the old "Build & Deploy" workflow. This workflow builds
# Docker images, pushes them to GHCR, and triggers Railway to redeploy.
# The separate "Showcase: Verify Deploy" workflow (showcase_deploy.yml)
# handles health verification.
#
# Critical design property: NO concurrency group with cancel-in-progress.
# Every push to main runs to completion so that rapid-fire PR merges never
# cancel in-flight builds. This was the #1 operational pain point with the
# old combined workflow.
on:
push:
branches: [main]
paths:
- "showcase/**"
- "examples/integrations/**"
- ".github/workflows/showcase_build.yml"
- ".github/workflows/showcase_build_check.yml"
workflow_dispatch:
inputs:
service:
description: "Service to build"
required: false
default: "all"
type: choice
options:
- all
- shell
- langgraph-python
- mastra
- crewai-crews
- pydantic-ai
- google-adk
- ag2
- agno
- llamaindex
- langgraph-fastapi
- langgraph-typescript
- langroid
- spring-ai
- strands
- strands-typescript
- ms-agent-python
- claude-sdk-typescript
- ms-agent-dotnet
- ms-agent-harness-dotnet
- claude-sdk-python
- built-in-agent
- shell-dojo
- shell-dashboard
- shell-docs
- showcase-harness
- showcase-aimock
- showcase-pocketbase
- webhooks
# Per-starter image builds (model B, §b stage 1). These map to the
# build-starters job below, NOT the showcase `build` job. "all"
# builds the full showcase fleet AND all 12 starters; a specific
# starter slug narrows to that one starter via STARTER_DISPATCH.
- starter-langgraph-python
- starter-mastra
- starter-langgraph-js
- starter-crewai-crews
- starter-pydantic-ai
- starter-adk
- starter-agno
- starter-llamaindex
- starter-langgraph-fastapi
- starter-strands-python
- starter-ms-agent-framework-python
- starter-ms-agent-framework-dotnet
# No top-level concurrency group. Every build run completes. This is the
# whole point of the decoupling: rapid pushes to main no longer cancel
# in-flight builds.
# Top-level env intentionally empty: env IDs are resolved inside
# showcase/scripts/redeploy-env.ts from the showcase/scripts/railway-envs.ts
# SSOT (and its emitted railway-envs.generated.json), not by this workflow.
permissions:
contents: read
jobs:
detect-changes:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
matrix: ${{ steps.build-matrix.outputs.matrix }}
has_changes: ${{ steps.build-matrix.outputs.has_changes }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Detect changed paths
uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
id: filter
with:
# All filter values use list form for consistency. `shell`
# genuinely needs multiple paths; the others could collapse to
# single-line strings, but mixing styles (list vs. string)
# in the same filters block is easy to misread during review.
filters: |
workflow_config:
- '.github/workflows/showcase_build.yml'
- '.github/workflows/showcase_build_check.yml'
shell:
- 'showcase/shell/**'
- 'showcase/shared/**'
- 'showcase/scripts/**'
- 'showcase/integrations/*/manifest.yaml'
langgraph_python:
- 'showcase/integrations/langgraph-python/**'
mastra:
- 'showcase/integrations/mastra/**'
crewai_crews:
- 'showcase/integrations/crewai-crews/**'
pydantic_ai:
- 'showcase/integrations/pydantic-ai/**'
google_adk:
- 'showcase/integrations/google-adk/**'
ag2:
- 'showcase/integrations/ag2/**'
agno:
- 'showcase/integrations/agno/**'
llamaindex:
- 'showcase/integrations/llamaindex/**'
langgraph_fastapi:
- 'showcase/integrations/langgraph-fastapi/**'
langgraph_typescript:
- 'showcase/integrations/langgraph-typescript/**'
langroid:
- 'showcase/integrations/langroid/**'
spring_ai:
- 'showcase/integrations/spring-ai/**'
strands:
- 'showcase/integrations/strands/**'
strands_typescript:
- 'showcase/integrations/strands-typescript/**'
ms_agent_python:
- 'showcase/integrations/ms-agent-python/**'
claude_sdk_typescript:
- 'showcase/integrations/claude-sdk-typescript/**'
ms_agent_dotnet:
- 'showcase/integrations/ms-agent-dotnet/**'
ms_agent_harness_dotnet:
- 'showcase/integrations/ms-agent-harness-dotnet/**'
claude_sdk_python:
- 'showcase/integrations/claude-sdk-python/**'
built_in_agent:
- 'showcase/integrations/built-in-agent/**'
shell_dojo:
- 'showcase/shell-dojo/**'
- 'showcase/shared/**'
- 'showcase/scripts/**'
- 'showcase/integrations/*/manifest.yaml'
shell_dashboard:
- 'showcase/shell-dashboard/**'
- 'showcase/shared/**'
- 'showcase/scripts/**'
- 'showcase/integrations/*/manifest.yaml'
shell_docs:
- 'showcase/shell-docs/**'
- 'showcase/shared/**'
- 'showcase/scripts/**'
- 'showcase/integrations/*/manifest.yaml'
- 'showcase/integrations/*/docs-links.json'
- 'showcase/integrations/*/docs/setup/**'
- 'showcase/integrations/*/src/**'
showcase_harness:
- 'showcase/harness/**'
- 'showcase/shared/**'
- 'showcase/scripts/**'
- 'showcase/integrations/*/manifest.yaml'
showcase_aimock:
- 'showcase/aimock/**'
pocketbase:
# PB image is self-contained: PB binary + pb_migrations +
# pb_hooks + Dockerfile. No shared-module copy, so the slot
# is gated purely to its own subtree — it does NOT rebuild
# on every showcase push, only when migrations/hooks/Dockerfile
# change.
- 'showcase/pocketbase/**'
webhooks:
# Sentinel pattern that cannot match any in-tree path.
# The webhooks GHCR image is built by the showcase-eval-webhook
# repo's own release workflow, so we never want a push-driven
# build run to include it. workflow_dispatch can still target
# webhooks explicitly via the service input — that path skips
# paths-filter entirely.
- 'showcase/__no_match_webhooks_built_out_of_band__'
- name: Build service matrix
id: build-matrix
env:
DISPATCH_SERVICE: ${{ github.event.inputs.service }}
GITHUB_SHA_ENV: ${{ github.sha }}
GITHUB_REF_NAME_ENV: ${{ github.ref_name }}
FILTER_CHANGES: ${{ steps.filter.outputs.changes }}
run: |
# Full service config as JSON
# Fields: dispatch_name, filter_key, context, image, railway_id, timeout, lfs, build_args, build_args_sha, build_args_branch, dockerfile, health_path, skip_build
# skip_build (optional, boolean): when true, the Docker build step
# is skipped for this slot (image is built out-of-band by another
# workflow/repo). Currently used by `webhooks` (built by the
# showcase-eval-webhook repo's own release workflow).
# health_path: historical field retained in the matrix for human
# reference. The actual verify probe is driven by per-service
# drivers in verify-deploy.ts (showcase/scripts/verify-deploy.ts),
# NOT by this field — it is informational only at this layer.
ALL_SERVICES='[
{"dispatch_name":"shell","filter_key":"shell","context":".","image":"showcase-shell","railway_id":"40eea0da-6071-4ea8-bdb9-39afb19225ec","timeout":10,"lfs":true,"build_args_sha":"__GH_SHA__","build_args_branch":"__GH_REF_NAME__","dockerfile":"showcase/shell/Dockerfile","health_path":"/"},
{"dispatch_name":"langgraph-python","filter_key":"langgraph_python","context":"showcase/integrations/langgraph-python","image":"showcase-langgraph-python","railway_id":"90d03214-4569-41b0-b4c1-6438a8a7b203","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"mastra","filter_key":"mastra","context":"showcase/integrations/mastra","image":"showcase-mastra","railway_id":"d7979eb7-2405-4aab-ad21-438f4a1b08af","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"crewai-crews","filter_key":"crewai_crews","context":"showcase/integrations/crewai-crews","image":"showcase-crewai-crews","railway_id":"0e9c284d-8d87-4fcf-9f82-6b704d7e4bd4","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"pydantic-ai","filter_key":"pydantic_ai","context":"showcase/integrations/pydantic-ai","image":"showcase-pydantic-ai","railway_id":"0a106173-2282-4887-a994-0ca276a99d69","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"google-adk","filter_key":"google_adk","context":"showcase/integrations/google-adk","image":"showcase-google-adk","railway_id":"87f60507-5a3d-4b8a-9e23-2b1de85d939c","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"ag2","filter_key":"ag2","context":"showcase/integrations/ag2","image":"showcase-ag2","railway_id":"4a37481b-f264-4eb7-a9cd-0a9ebb9ac05c","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"agno","filter_key":"agno","context":"showcase/integrations/agno","image":"showcase-agno","railway_id":"32cab80b-e329-45bd-9c73-c4e1ddc94305","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"llamaindex","filter_key":"llamaindex","context":"showcase/integrations/llamaindex","image":"showcase-llamaindex","railway_id":"285386e8-492d-4cb8-b632-0a7d4607378f","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"langgraph-fastapi","filter_key":"langgraph_fastapi","context":"showcase/integrations/langgraph-fastapi","image":"showcase-langgraph-fastapi","railway_id":"06cccb5c-59f4-46b5-8adc-7113e77011a4","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"langgraph-typescript","filter_key":"langgraph_typescript","context":"showcase/integrations/langgraph-typescript","image":"showcase-langgraph-typescript","railway_id":"66246d3b-a18e-46f0-be51-5f3ff7a36e5a","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"langroid","filter_key":"langroid","context":"showcase/integrations/langroid","image":"showcase-langroid","railway_id":"6dd9cb0a-66cc-46f1-972e-7cd74756157d","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"spring-ai","filter_key":"spring_ai","context":"showcase/integrations/spring-ai","image":"showcase-spring-ai","railway_id":"eed5d041-91be-4282-b414-beea00843401","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"strands","filter_key":"strands","context":"showcase/integrations/strands","image":"showcase-strands","railway_id":"92e1cfad-ad53-403f-ab2b-5ab380832232","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"strands-typescript","filter_key":"strands_typescript","context":"showcase/integrations/strands-typescript","image":"showcase-strands-typescript","railway_id":"d6f47c8c-a0a1-4dbe-991c-50f8463fd68d","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"ms-agent-python","filter_key":"ms_agent_python","context":"showcase/integrations/ms-agent-python","image":"showcase-ms-agent-python","railway_id":"655db75a-af8d-427d-a4f9-441570ae5003","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"claude-sdk-typescript","filter_key":"claude_sdk_typescript","context":"showcase/integrations/claude-sdk-typescript","image":"showcase-claude-sdk-typescript","railway_id":"18a98727-5700-44aa-b497-b60795dbbd6a","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"ms-agent-dotnet","filter_key":"ms_agent_dotnet","context":"showcase/integrations/ms-agent-dotnet","image":"showcase-ms-agent-dotnet","railway_id":"beeb2dd6-87a4-4599-aa07-0578f7bd6519","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"ms-agent-harness-dotnet","filter_key":"ms_agent_harness_dotnet","context":"showcase/integrations/ms-agent-harness-dotnet","image":"showcase-ms-agent-harness-dotnet","railway_id":"6343d7f9-6c3f-4c8d-9a6e-79f03d2f1e37","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"claude-sdk-python","filter_key":"claude_sdk_python","context":"showcase/integrations/claude-sdk-python","image":"showcase-claude-sdk-python","railway_id":"b122ab65-9854-4cb2-a68e-b50ff13f7481","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"built-in-agent","filter_key":"built_in_agent","context":"showcase/integrations/built-in-agent","image":"showcase-built-in-agent","railway_id":"f4f8371a-bc46-45b2-b6d4-9c9af608bdbf","timeout":15,"lfs":false,"build_args":"","dockerfile":"","health_path":"/api/health"},
{"dispatch_name":"shell-dojo","filter_key":"shell_dojo","context":".","image":"showcase-shell-dojo","railway_id":"7ad1ece7-2228-49cd-8a78-bddf30322907","timeout":10,"lfs":false,"build_args":"","dockerfile":"showcase/shell-dojo/Dockerfile","health_path":"/"},
{"dispatch_name":"shell-dashboard","filter_key":"shell_dashboard","context":".","image":"showcase-shell-dashboard","railway_id":"4d5dfd74-be61-40b2-8564-b53b7dd4c15b","timeout":10,"lfs":true,"build_args_sha":"__GH_SHA__","build_args_branch":"__GH_REF_NAME__","dockerfile":"showcase/shell-dashboard/Dockerfile","health_path":"/"},
{"dispatch_name":"shell-docs","filter_key":"shell_docs","context":".","image":"showcase-shell-docs","railway_id":"7badfb8d-4228-414c-9145-b4026803714f","timeout":10,"lfs":true,"build_args_sha":"__GH_SHA__","build_args_branch":"__GH_REF_NAME__","dockerfile":"showcase/shell-docs/Dockerfile","health_path":"/"},
{"dispatch_name":"showcase-harness","filter_key":"showcase_harness","context":".","image":"showcase-harness","railway_id":"3a14bfed-0537-4d71-897b-7c593dca161d","timeout":20,"lfs":false,"build_args":"","dockerfile":"showcase/harness/Dockerfile","health_path":"/health"},
{"dispatch_name":"showcase-aimock","filter_key":"showcase_aimock","context":"showcase/aimock","image":"showcase-aimock","railway_id":"0fa0435d-8a66-46f0-84fd-e4250b580013","timeout":5,"lfs":false,"build_args":"","dockerfile":"showcase/aimock/Dockerfile","health_path":"/health"},
{"dispatch_name":"showcase-pocketbase","filter_key":"pocketbase","context":"showcase/pocketbase","image":"showcase-pocketbase","railway_id":"ba11e854-d695-4738-9a45-2b0776788824","timeout":10,"lfs":false,"build_args":"","dockerfile":"showcase/pocketbase/Dockerfile","health_path":"/api/health"},
{"dispatch_name":"webhooks","filter_key":"webhooks","context":".","image":"showcase-eval-webhook","railway_id":"ba6acc13-7585-41fe-a5ee-585b34a58fcd","timeout":5,"lfs":false,"build_args":"","dockerfile":"","health_path":"/health","skip_build":true}
]'
DISPATCH="$DISPATCH_SERVICE"
CHANGES="${FILTER_CHANGES:-[]}"
# Filter services based on three dispatch modes:
# dispatch == "all": manual "deploy all" — include every service unconditionally.
# This re-pulls :latest for every matrix slot (~38 services); intentional for
# drift-rebuild runs and full-fleet restarts. Do NOT try to short-circuit
# unchanged services here — operators invoke "all" precisely when they want
# the fleet re-deployed regardless of git state (cache poisoning, base-image CVE).
# (paths-filter is unreliable on workflow_dispatch because there is no 'before' SHA,
# so we must NOT consult $changes here — doing so silently produces an empty matrix).
# dispatch == <specific service>: narrow to that service only (skips paths-filter so
# drift-rebuild and manual single-service dispatches work regardless of $changes).
# dispatch == "": push event — include services whose filter_key appears in paths-filter CHANGES.
MATRIX=$(echo "$ALL_SERVICES" | jq -c --arg dispatch "$DISPATCH" --argjson changes "$CHANGES" --arg sha "$GITHUB_SHA_ENV" --arg ref "$GITHUB_REF_NAME_ENV" '
[.[] |
if .build_args_sha == "__GH_SHA__" then .build_args_sha = $sha else . end |
if .build_args_branch == "__GH_REF_NAME__" then .build_args_branch = $ref else . end |
(.filter_key as $fk | select(
$dispatch == "all" or
($dispatch != "" and $dispatch != "all" and $dispatch == .dispatch_name) or
($dispatch == "" and (($changes | index("workflow_config") != null) or ($changes | index($fk) != null)))
))]
')
# Fail loudly on typo'd workflow_dispatch inputs. If a user types a
# service name that doesn't exist in ALL_SERVICES, the jq filter
# silently produces [] and the run shows green with zero work
# done — a common "did my dispatch deploy?" footgun. The `all` and
# empty (push-event) modes legitimately produce [] when there are
# no changes and must still succeed.
#
# The `starter-*` namespace is OWNED by the detect-starter-changes
# job (which scopes its OWN fail-loud to `starter-*`). A
# `service=starter-<slug>` dispatch legitimately yields [] here
# (no showcase service is named `starter-*`), so it must SKIP — an
# empty showcase matrix, NOT a fail-loud exit 1. Mirrors how the
# starter job scopes its fail-loud to the `starter-*` namespace.
case "$DISPATCH" in
starter-*) ;; # starter namespace → empty showcase matrix + skip
"" | "all") ;; # push / full-fleet → [] is legitimate
*)
if [ "$MATRIX" = "[]" ]; then
echo "::error::workflow_dispatch service='$DISPATCH' did not match any entry in ALL_SERVICES — check the dispatch_name spelling"
exit 1
fi
;;
esac
echo "matrix=$MATRIX" >> $GITHUB_OUTPUT
if [ "$MATRIX" = "[]" ]; then
echo "has_changes=false" >> $GITHUB_OUTPUT
else
echo "has_changes=true" >> $GITHUB_OUTPUT
fi
check-lockfile:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (enforced via corepack).
# Earlier revisions hard-pinned `version: 10.13.1` which silently
# drifted from package.json whenever the repo bumped pnpm —
# resulting in lockfile-vs-engine mismatches that only surfaced on
# the slow `--frozen-lockfile` path.
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
- run: pnpm install --frozen-lockfile --ignore-scripts
verify-image-refs:
needs: [detect-changes]
if: needs.detect-changes.outputs.has_changes == 'true'
runs-on: ubuntu-latest
timeout-minutes: 3
permissions:
contents: read
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
- name: Verify Railway image refs
env:
RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}
run: npx tsx showcase/scripts/verify-railway-image-refs.ts
build:
needs: [detect-changes, check-lockfile, verify-image-refs]
if: needs.detect-changes.outputs.has_changes == 'true'
runs-on: depot-ubuntu-24.04-4
timeout-minutes: ${{ fromJSON(matrix.service.timeout) }}
permissions:
id-token: write
contents: read
packages: write
strategy:
fail-fast: false
matrix:
service: ${{ fromJSON(needs.detect-changes.outputs.matrix) }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
# Always pull LFS. Nearly every integration ships LFS-tracked demo
# assets under public/ (public/demo-files/*.png|*.pdf,
# public/demo-audio/*.wav per root .gitattributes). With lfs:false
# these check out as 130-byte LFS pointer stubs and get COPYed into
# the image as text — the deployed image then serves the pointer
# with HTTP 200 and the frontend's magic-bytes guard rejects it,
# breaking the multimodal test pill. The per-slot matrix.service.lfs
# flag was true only for shell/shell-dashboard/shell-docs, leaving
# every framework integration shipping pointer stubs. Uniform
# lfs:true is the least-error-prone fix: a new integration that adds
# demo assets is covered automatically, with no matrix flag to
# forget.
lfs: true
persist-credentials: false
- name: Setup Depot
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1
- name: Login to GHCR
uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Prepare build args
id: build-args
env:
BUILD_ARGS_SHA: ${{ matrix.service.build_args_sha }}
BUILD_ARGS_BRANCH: ${{ matrix.service.build_args_branch }}
run: |
# set -euo pipefail: without `-e`, a transient `$GITHUB_OUTPUT`
# write failure (disk pressure, ENOSPC) could silently produce
# empty build-args and we'd ship an image without COMMIT_SHA /
# BRANCH baked in — invisible drift between build label and
# what's actually running.
set -euo pipefail
ARGS=""
if [ -n "$BUILD_ARGS_SHA" ]; then
ARGS="COMMIT_SHA=${BUILD_ARGS_SHA}"
ARGS="${ARGS}"$'\n'"BRANCH=${BUILD_ARGS_BRANCH}"
fi
# Use delimiter to safely pass multiline value
echo "args<<BUILDARGS_EOF" >> $GITHUB_OUTPUT
echo "$ARGS" >> $GITHUB_OUTPUT
echo "BUILDARGS_EOF" >> $GITHUB_OUTPUT
- name: Copy shared modules into build context
run: |
set -euo pipefail
CONTEXT="${{ matrix.service.context }}"
# Idempotent copy: if a stale `shared_python`/`shared_typescript`
# already exists (previous failed run on the same runner, or a
# checkout artifact), remove it first. `cp -r src dst` into an
# existing directory nests source-inside-destination, which
# would silently produce a broken build context.
if [ -d "showcase/shared/python" ] && [ -d "$CONTEXT" ]; then
rm -rf "$CONTEXT/shared_python"
cp -r showcase/shared/python "$CONTEXT/shared_python"
fi
if [ -d "showcase/shared/typescript/tools" ] && [ -d "$CONTEXT" ]; then
rm -rf "$CONTEXT/shared_typescript"
mkdir -p "$CONTEXT/shared_typescript"
cp -r showcase/shared/typescript/tools "$CONTEXT/shared_typescript/tools"
fi
# Dereference tools/, shared-tools/, and _shared/ symlinks for the
# Docker context. Integration directories use symlinks pointing to
# ../../shared/python/tools etc., and _shared -> ../_shared for the
# CVDIAG bootstrap modules. Docker cannot follow symlinks outside
# the build context (buildkit fails the checksum with "too many
# symlinks: /_shared"), so we replace each symlink with a real copy
# of its target. Mirrors stage_shared() in
# showcase/scripts/cli/_common.sh (the local bin/showcase path).
for link_name in tools shared-tools _shared; do
link_path="$CONTEXT/$link_name"
if [ -L "$link_path" ]; then
target="$(readlink -f "$link_path")"
if [ -d "$target" ]; then
rm "$link_path"
cp -r "$target" "$link_path"
fi
fi
done
- name: Build and push
if: ${{ matrix.service.skip_build != true }}
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.18.0
with:
project: m2kw2wmmcp
context: ${{ matrix.service.context }}
file: ${{ matrix.service.dockerfile != '' && matrix.service.dockerfile || format('{0}/Dockerfile', matrix.service.context) }}
# Pin amd64: Railway and GHCR serve x86 hosts. An arm64-only
# image crashes on pull with "does not have a linux/amd64
# variant available".
platforms: linux/amd64
push: true
tags: |
ghcr.io/copilotkit/${{ matrix.service.image }}:latest
ghcr.io/copilotkit/${{ matrix.service.image }}:${{ github.sha }}
build-args: ${{ steps.build-args.outputs.args }}
- name: Write per-slot build result
if: always()
env:
SERVICE: ${{ matrix.service.dispatch_name }}
BUILD_STATUS: ${{ job.status }}
run: |
# job.status is one of: success, failure, cancelled. Normalize
# cancelled→skipped to match the BuildOutcome contract in
# showcase/scripts/lib/build-outputs.ts. We deliberately do NOT
# write to $GITHUB_OUTPUT — matrix-slot outputs are not
# aggregable across slots in GitHub Actions, so we publish the
# per-slot result as an artifact instead. The downstream
# aggregator job downloads every `build-result-*` artifact.
case "$BUILD_STATUS" in
success) STATUS=success ;;
failure) STATUS=failure ;;
*) STATUS=skipped ;;
esac
mkdir -p "$RUNNER_TEMP/build-result"
printf '{"service":"%s","status":"%s"}\n' "$SERVICE" "$STATUS" \
> "$RUNNER_TEMP/build-result/result.json"
- name: Upload per-slot build-result artifact
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
# Canonical per-slot name (see buildResultArtifactName in
# showcase/scripts/lib/build-outputs.ts). The aggregator
# downloads every artifact matching `build-result-*`.
name: build-result-${{ matrix.service.dispatch_name }}
path: ${{ runner.temp }}/build-result/result.json
if-no-files-found: error
retention-days: 7
# ────────────────────────────────────────────────────────────────────
# Per-starter image publish (model B, §b stage 1 / Phase 1).
#
# Fully decoupled from the showcase `build`→`aggregate`→`redeploy` chain
# above: starters are self-contained npm projects (no monorepo-source
# build, no shared-module copy) and build from their own root Dockerfile
# at examples/integrations/<slug>/Dockerfile (the single-image deployable:
# Next.js frontend + agent, EXPOSE 3000, CMD entrypoint.sh — distinct from
# the docker/Dockerfile.app + docker/Dockerfile.agent split stack used by
# docker-compose.test.yml). Published to ghcr.io/copilotkit/starter-<slug>
# (the `starter-` prefix is disjoint from `showcase-*`; S2's harness
# discovery filters on namePrefix "starter-"). Railway deploy is the
# gated S5 — NOT done here.
#
# The 12 starter slugs are the matrix source of truth (the smoke matrix in
# test_smoke-starter.yml and STARTER_TO_COLUMN in
# showcase/harness/src/probes/helpers/starter-mapping.ts). The dashboard
# column remap lives in the harness (§a), so this layer uses raw starter
# slugs.
detect-starter-changes:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
matrix: ${{ steps.starter-matrix.outputs.matrix }}
has_changes: ${{ steps.starter-matrix.outputs.has_changes }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Detect changed starter paths
uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
id: filter
with:
filters: |
workflow_config:
- '.github/workflows/showcase_build.yml'
langgraph_python:
- 'examples/integrations/langgraph-python/**'
mastra:
- 'examples/integrations/mastra/**'
langgraph_js:
- 'examples/integrations/langgraph-js/**'
crewai_crews:
- 'examples/integrations/crewai-crews/**'
pydantic_ai:
- 'examples/integrations/pydantic-ai/**'
adk:
- 'examples/integrations/adk/**'
agno:
- 'examples/integrations/agno/**'
llamaindex:
- 'examples/integrations/llamaindex/**'
langgraph_fastapi:
- 'examples/integrations/langgraph-fastapi/**'
strands_python:
- 'examples/integrations/strands-python/**'
ms_agent_framework_python:
- 'examples/integrations/ms-agent-framework-python/**'
ms_agent_framework_dotnet:
- 'examples/integrations/ms-agent-framework-dotnet/**'
- name: Build starter matrix
id: starter-matrix
env:
DISPATCH_SERVICE: ${{ github.event.inputs.service }}
FILTER_CHANGES: ${{ steps.filter.outputs.changes }}
run: |
set -euo pipefail
# One slot per starter. `slug` is the examples/integrations/<slug>
# directory name; `image` is the published GHCR repo (starter-<slug>);
# `filter_key` matches the paths-filter key above.
ALL_STARTERS='[
{"slug":"langgraph-python","image":"starter-langgraph-python","filter_key":"langgraph_python"},
{"slug":"mastra","image":"starter-mastra","filter_key":"mastra"},
{"slug":"langgraph-js","image":"starter-langgraph-js","filter_key":"langgraph_js"},
{"slug":"crewai-crews","image":"starter-crewai-crews","filter_key":"crewai_crews"},
{"slug":"pydantic-ai","image":"starter-pydantic-ai","filter_key":"pydantic_ai"},
{"slug":"adk","image":"starter-adk","filter_key":"adk"},
{"slug":"agno","image":"starter-agno","filter_key":"agno"},
{"slug":"llamaindex","image":"starter-llamaindex","filter_key":"llamaindex"},
{"slug":"langgraph-fastapi","image":"starter-langgraph-fastapi","filter_key":"langgraph_fastapi"},
{"slug":"strands-python","image":"starter-strands-python","filter_key":"strands_python"},
{"slug":"ms-agent-framework-python","image":"starter-ms-agent-framework-python","filter_key":"ms_agent_framework_python"},
{"slug":"ms-agent-framework-dotnet","image":"starter-ms-agent-framework-dotnet","filter_key":"ms_agent_framework_dotnet"}
]'
# Dispatch modes (mirror the showcase detect-changes job):
# "all" → every starter (full-fleet rebuild).
# "starter-<slug>" → that one starter (strip "starter-" prefix to match .image).
# "<showcase service slug>" → no starters (this is a showcase-only dispatch).
# "" (push event) → starters whose filter_key appears in CHANGES
# (or workflow_config touched → rebuild all).
DISPATCH="${DISPATCH_SERVICE:-}"
CHANGES="${FILTER_CHANGES:-[]}"
# `.image` is exactly "starter-<slug>", which is also the
# workflow_dispatch choice value, so a specific-starter dispatch
# matches `$dispatch == .image` directly.
MATRIX=$(echo "$ALL_STARTERS" | jq -c --arg dispatch "$DISPATCH" --argjson changes "$CHANGES" '
[.[] |
(.filter_key as $fk | select(
$dispatch == "all" or
($dispatch != "" and $dispatch != "all" and $dispatch == .image) or
($dispatch == "" and (($changes | index("workflow_config") != null) or ($changes | index($fk) != null)))
))]
')
# Fail loudly on a typo'd starter dispatch (mirrors showcase job).
# Only applies to the starter-* dispatch namespace; a showcase
# service slug or "all" legitimately yields [] here.
case "$DISPATCH" in
starter-*)
if [ "$MATRIX" = "[]" ]; then
echo "::error::workflow_dispatch service='$DISPATCH' did not match any starter — check the slug"
exit 1
fi
;;
esac
echo "matrix=$MATRIX" >> "$GITHUB_OUTPUT"
if [ "$MATRIX" = "[]" ]; then
echo "has_changes=false" >> "$GITHUB_OUTPUT"
else
echo "has_changes=true" >> "$GITHUB_OUTPUT"
fi
build-starters:
needs: [detect-starter-changes]
if: needs.detect-starter-changes.outputs.has_changes == 'true'
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 20
permissions:
id-token: write
contents: read
packages: write
strategy:
fail-fast: false
matrix:
starter: ${{ fromJSON(needs.detect-starter-changes.outputs.matrix) }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Setup Depot
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1
- name: Login to GHCR
uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push starter image
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.18.0
with:
project: m2kw2wmmcp
context: examples/integrations/${{ matrix.starter.slug }}
file: examples/integrations/${{ matrix.starter.slug }}/Dockerfile
# Pin amd64: Railway and GHCR serve x86 hosts. Depot defaults to
# the runner's native arch, so an arm64-only image would crash on
# pull with "does not have a linux/amd64 variant available".
platforms: linux/amd64
push: true
tags: |
ghcr.io/copilotkit/${{ matrix.starter.image }}:latest
ghcr.io/copilotkit/${{ matrix.starter.image }}:${{ github.sha }}
aggregate-build-results:
name: Aggregate build results
needs: [detect-changes, build]
if: ${{ !cancelled() && needs.detect-changes.outputs.has_changes == 'true' }}
runs-on: ubuntu-latest
timeout-minutes: 3
permissions:
contents: read
outputs:
results: ${{ steps.collect.outputs.results }}
any_success: ${{ steps.collect.outputs.any_success }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Setup pnpm
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Setup Node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
cache: pnpm
- name: Install
run: pnpm install --frozen-lockfile --ignore-scripts
- name: Download all per-slot build-result artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
# `pattern` matches every per-slot artifact emitted by the
# build matrix. `merge-multiple: false` keeps each artifact
# in its own subdirectory so we can iterate them deterministically.
pattern: build-result-*
path: ${{ runner.temp }}/build-results-in
merge-multiple: false
- name: Collect per-service build outcomes
id: collect
env:
INPUT_DIR: ${{ runner.temp }}/build-results-in
OUTPUT_DIR: ${{ runner.temp }}/build-results-out
run: |
set -euo pipefail
mkdir -p "$OUTPUT_DIR"
# Each per-slot artifact extracts to
# $INPUT_DIR/build-result-<dispatch_name>/result.json
# The aggregator script (showcase/scripts/aggregate-build-results.ts)
# reads them, merges via the shared helper (mergeBuildResultFiles),
# writes $OUTPUT_DIR/results.json, and appends `results` +
# `any_success` to $GITHUB_OUTPUT. The contract (service +
# status enum) is enforced in one place (build-outputs.ts).
npx tsx showcase/scripts/aggregate-build-results.ts
- name: Upload aggregated build-results artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: build-results
path: ${{ runner.temp }}/build-results-out/results.json
if-no-files-found: error
retention-days: 7
redeploy-staging:
name: Trigger Railway staging redeploy
needs: [detect-changes, build, aggregate-build-results]
# Run if at least one service was in the matrix AND the build job
# was not outright cancelled or skipped AND at least one slot
# succeeded (per aggregate-build-results.outputs.any_success). The
# any_success guard is the explicit "do not redeploy when nothing
# was pushed" check — without it, an all-failed build run would
# still kick a redeploy that just re-pulls the stale :latest and
# silently looks healthy. The skipped/cancelled checks on the build
# job still cover the "verify-image-refs gate blocked the build job
# entirely" path. Partial build failures (fail-fast: false) still
# surface as needs.build.result == 'failure' with any_success ==
# 'true', so redeploying what did get pushed is preserved.
if: >-
${{ !cancelled()
&& needs.detect-changes.outputs.has_changes == 'true'
&& needs.build.result != 'skipped'
&& needs.build.result != 'cancelled'
&& needs.aggregate-build-results.outputs.any_success == 'true' }}
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
- name: Compute changed-service list from build matrix
id: changed
env:
MATRIX_JSON: ${{ needs.detect-changes.outputs.matrix }}
BUILD_RESULTS_JSON: ${{ needs.aggregate-build-results.outputs.results }}
run: |
# We feed the redeploy script with the INTERSECTION of:
# (a) the scheduled build matrix (detect-changes.outputs.matrix —
# JSON array of objects with `dispatch_name`), and
# (b) the SUCCESS set from the aggregator
# (aggregate-build-results.outputs.results — JSON array of
# `{service: <dispatch_name>, status: success|failure|skipped}`;
# the `service` field is the dispatch_name; shape defined in
# showcase/scripts/lib/build-outputs.ts).
# Without this intersection a service whose Docker build FAILED
# would still be in the redeploy CSV, Railway would re-pull its
# stale `:latest`, and the verify workflow would report it as a
# fresh, healthy deploy — a false green. Skipped slots are also
# excluded (only `status == "success"` qualifies).
set -euo pipefail
matrix_names="$(echo "$MATRIX_JSON" | jq -r '[.[] | .dispatch_name]')"
success_names="$(echo "$BUILD_RESULTS_JSON" | jq -r '[.[] | select(.status == "success") | .service]')"
csv="$(jq -rn --argjson m "$matrix_names" --argjson s "$success_names" \
'($m | map(select(. as $n | $s | index($n)))) | join(",")')"
if [ -z "$csv" ]; then
echo "No services in matrix ∩ success-set — skipping redeploy."
echo "services=" >> "$GITHUB_OUTPUT"
else
echo "services=$csv" >> "$GITHUB_OUTPUT"
fi
echo "Computed services CSV (matrix ∩ build-success): $csv"
- name: Redeploy changed services in staging
if: steps.changed.outputs.services != ''
env:
RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}
SERVICES_CSV: ${{ steps.changed.outputs.services }}
# Bridge the per-service redeploy summary to showcase_deploy.yml's
# `enforce-redeploy-gate` (consumed via the `redeploy-summary`
# artifact, extracted to `.redeploy/summary.json`). redeploy-env.ts
# writes this path atomically (.tmp → rename) but does NOT create
# parent dirs, so the step below mkdir's `.redeploy` first.
REDEPLOY_SUMMARY_JSON: .redeploy/summary.json
run: |
# Staging is non-blocking by design: the script always exits 0
# and writes per-service failures into $GITHUB_STEP_SUMMARY. The
# verify-deploy workflow is the real release gate.
mkdir -p .redeploy
npx tsx showcase/scripts/redeploy-env.ts staging --services "$SERVICES_CSV"
- name: Upload redeploy summary
# Upload is MANDATORY whenever a redeploy was attempted (services
# != ''). Both failure modes red the build — no false-green path:
#
# (A) HARD crash inside redeploy-env.ts BEFORE summary.json is
# written. redeploy-env.ts writes the summary atomically
# (.tmp → rename) AFTER the per-service loop completes, so
# a crash leaves no file. The redeploy step itself exits
# non-zero on that crash and fails the redeploy-staging
# job; this upload step is then skipped entirely by
# step-failure propagation. Build → red.
#
# (B) redeploy step exits 0 but summary.json is absent (e.g. a
# logic bug skipped the write). `if-no-files-found: error`
# reds this step → reds the redeploy-staging job → reds the
# build. The deploy workflow's resolve-matrix.if
# (workflow_run.conclusion == 'success') then blocks the
# deploy run from starting at all.
#
# Do NOT add hashFiles() guards here: that would silently skip
# the upload on (B), the deploy workflow would see "artifact
# absent" via check-redeploy-summary, treat it as "nothing
# redeployed", skip the gate, and ship a false-green.
# The legitimate "services == '' → nothing redeployed → no upload"
# path is preserved by the services != '' guard.
#
# If a future change ever switches this step to `if: always()`,
# `if-no-files-found: error` STILL reds path (A): the redeploy
# step's non-zero exit on a HARD crash is independent of upload
# gating, and `if-no-files-found: error` on `always()` then trips
# because summary.json was never written. So neither relaxation
# alone opens a false-green window.
#
# However, swapping the guard to `if: always()` would ALSO red the
# legitimate `services == ''` (nothing-to-redeploy) path — no
# summary.json is written there either, so `if-no-files-found:
# error` would trip on every push that didn't redeploy anything.
# Net effect: trades the (already-closed) false-green risk for a
# false-red on every non-buildable push. Don't do it.
if: steps.changed.outputs.services != ''
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
# Artifact name MUST stay `redeploy-summary`: showcase_deploy.yml's
# `resolve-matrix` job downloads it by this exact name and reads
# `.redeploy/summary.json` inside.
name: redeploy-summary
path: .redeploy/summary.json
if-no-files-found: error
retention-days: 7
notify-all-builds-failed:
name: Notify all builds failed (staging unchanged)
needs: [detect-changes, build, aggregate-build-results]
# Explicit "everything failed; nothing redeployed" signal — distinct
# from the `notify:` job below which fires on any build-job failure
# (some slots may still have succeeded in that case). Both jobs can
# fire; that's intentional and matches the Slack alert SOP.
if: >-
${{ !cancelled()
&& needs.detect-changes.outputs.has_changes == 'true'
&& needs.build.result == 'failure'
&& needs.aggregate-build-results.outputs.any_success == 'false' }}
# The `needs.build.result == 'failure'` clause guards against the case
# where the build job itself was SKIPPED (e.g. verify-image-refs failed
# upstream, so the matrix never executed). Without it, the aggregator
# would still report `any_success=false` and we'd Slack-spam "all
# builds failed" even though builds never ran — a misleading alert.
runs-on: ubuntu-latest
timeout-minutes: 3
permissions:
contents: read
env:
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_OSS_ALERTS }}
steps:
- name: Mark workflow red (no service succeeded)
run: |
echo "::error::All builds failed for this run; staging redeploy skipped; :latest unchanged."
exit 1
- name: Slack #oss-alerts
if: always() && env.SLACK_WEBHOOK != ''
uses: slackapi/slack-github-action@0d95c9a7becc1e6e297d76df9bc735c44f4cbcbc # v3.0.5
with:
webhook: ${{ secrets.SLACK_WEBHOOK_OSS_ALERTS }}
webhook-type: incoming-webhook
# NOTE: `\n` is NOT an escape sequence in GitHub Actions expression
# string literals — `format()` would emit the two literal characters
# backslash+n, which `toJSON` then encodes as `\\n`, so Slack renders
# a literal "\n". Inject real newlines via `fromJSON('"\n"')` ({4}) so
# `toJSON` encodes them as a single `\n` that Slack honors.
payload: |
{ "text": ${{ toJSON(format(':x: *Showcase: all builds failed*{4}staging unchanged (no redeploy){4}Commit: `{0}` by {1}{4}<https://github.com/{2}/actions/runs/{3}|View run>', github.sha, github.actor, github.repository, github.run_id, fromJSON('"\n"'))) }} }
notify:
name: Notify on failure
# Cover the whole detect→build→aggregate→redeploy pipeline. `needs: [build]`
# alone meant a failure in aggregate-build-results or redeploy-staging
# produced ZERO Slack signal (verify just never ran). Likewise, a red
# in detect-changes, check-lockfile, or verify-image-refs would skip
# `build` (skipped != failure) so the original `needs: [build]` form
# also missed those pre-build red paths. Adding the early-stage jobs
# to `needs` extends the alert surface to the full workflow, matching
# the workflow-level notify pattern in showcase_promote.yml.
# `if: failure()` already skips when none of the needs failed — so
# this still no-ops for the "no changes → build skipped" path, since
# skipped != failure. If `notify-all-builds-failed` also fires
# (genuine all-failed case), both alerts firing for the same event is
# acceptable per the alert SOP.
needs:
[
detect-changes,
check-lockfile,
verify-image-refs,
build,
aggregate-build-results,
redeploy-staging,
]
if: failure()
runs-on: ubuntu-latest
permissions:
pull-requests: write
issues: write
timeout-minutes: 5
env:
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_OSS_ALERTS }}
steps:
- name: Slack alert
if: env.SLACK_WEBHOOK != ''
uses: slackapi/slack-github-action@0d95c9a7becc1e6e297d76df9bc735c44f4cbcbc # v3.0.5
with:
webhook: ${{ secrets.SLACK_WEBHOOK_OSS_ALERTS }}
webhook-type: incoming-webhook
# NOTE: `\n` is NOT an escape sequence in GitHub Actions expression
# string literals — `format()` would emit the two literal characters
# backslash+n, which `toJSON` then encodes as `\\n`, so Slack renders
# a literal "\n". Inject real newlines via `fromJSON('"\n"')` ({4}) so
# `toJSON` encodes them as a single `\n` that Slack honors.
payload: |
{ "text": ${{ toJSON(format(':x: *Showcase Build Failed*{4}Commit: `{0}` by {1}{4}<https://github.com/{2}/actions/runs/{3}|View run>', github.sha, github.actor, github.repository, github.run_id, fromJSON('"\n"'))) }} }
- name: Comment on PR
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
const { data: prs } = await github.rest.repos.listPullRequestsAssociatedWithCommit({
owner: context.repo.owner,
repo: context.repo.repo,
commit_sha: context.sha,
});
const merged = prs.find(pr => pr.merged_at);
if (!merged) {
console.log('No merged PR found for this commit — skipping comment');
return;
}
const marker = '<!-- showcase-build-failure -->';
const body = [
marker,
`### :x: Showcase Build Failed`,
``,
`The Docker build triggered by this PR's merge failed.`,
``,
`**Run:** ${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`,
`**Commit:** \`${context.sha.slice(0, 8)}\``,
``,
`@${merged.user?.login ?? 'unknown'} — please check the build logs and fix the issue.`,
].join('\n');
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: merged.number,
});
const existing = comments.find(c => c.body?.includes(marker));
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: merged.number,
body,
});
}
+262
View File
@@ -0,0 +1,262 @@
name: "Showcase: Build Check (PR)"
# Pre-merge Docker build check. Runs the same Depot Docker builds as the
# post-merge showcase_build.yml pipeline but with push: false, so
# Dockerfile-specific failures (missing deps that exist in the monorepo
# but not in the isolated Docker context) are caught before merge.
on:
pull_request:
paths:
- "showcase/**"
- ".github/workflows/showcase_build.yml"
- ".github/workflows/showcase_build_check.yml"
concurrency:
group: showcase-build-check-${{ github.event.pull_request.number }}
cancel-in-progress: true
permissions:
contents: read
jobs:
detect-changes:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
matrix: ${{ steps.build-matrix.outputs.matrix }}
has_changes: ${{ steps.build-matrix.outputs.has_changes }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Detect changed paths
uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
id: filter
with:
filters: |
workflow_config:
- '.github/workflows/showcase_build.yml'
- '.github/workflows/showcase_build_check.yml'
shell:
- 'showcase/shell/**'
- 'showcase/shared/**'
- 'showcase/scripts/**'
- 'showcase/integrations/*/manifest.yaml'
langgraph_python:
- 'showcase/integrations/langgraph-python/**'
mastra:
- 'showcase/integrations/mastra/**'
crewai_crews:
- 'showcase/integrations/crewai-crews/**'
pydantic_ai:
- 'showcase/integrations/pydantic-ai/**'
google_adk:
- 'showcase/integrations/google-adk/**'
ag2:
- 'showcase/integrations/ag2/**'
agno:
- 'showcase/integrations/agno/**'
llamaindex:
- 'showcase/integrations/llamaindex/**'
langgraph_fastapi:
- 'showcase/integrations/langgraph-fastapi/**'
langgraph_typescript:
- 'showcase/integrations/langgraph-typescript/**'
langroid:
- 'showcase/integrations/langroid/**'
spring_ai:
- 'showcase/integrations/spring-ai/**'
strands:
- 'showcase/integrations/strands/**'
strands_typescript:
- 'showcase/integrations/strands-typescript/**'
ms_agent_python:
- 'showcase/integrations/ms-agent-python/**'
claude_sdk_typescript:
- 'showcase/integrations/claude-sdk-typescript/**'
ms_agent_dotnet:
- 'showcase/integrations/ms-agent-dotnet/**'
ms_agent_harness_dotnet:
- 'showcase/integrations/ms-agent-harness-dotnet/**'
claude_sdk_python:
- 'showcase/integrations/claude-sdk-python/**'
built_in_agent:
- 'showcase/integrations/built-in-agent/**'
shell_dojo:
- 'showcase/shell-dojo/**'
- 'showcase/shared/**'
- 'showcase/scripts/**'
- 'showcase/integrations/*/manifest.yaml'
shell_dashboard:
- 'showcase/shell-dashboard/**'
- 'showcase/shared/**'
- 'showcase/scripts/**'
- 'showcase/integrations/*/manifest.yaml'
shell_docs:
- 'showcase/shell-docs/**'
- 'showcase/shared/**'
- 'showcase/scripts/**'
- 'showcase/integrations/*/manifest.yaml'
- 'showcase/integrations/*/docs-links.json'
- 'showcase/integrations/*/docs/setup/**'
- 'showcase/integrations/*/src/**'
showcase_harness:
- 'showcase/harness/**'
- 'showcase/shared/**'
- 'showcase/scripts/**'
- 'showcase/integrations/*/manifest.yaml'
showcase_aimock:
- 'showcase/aimock/**'
- name: Build service matrix
id: build-matrix
env:
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
PR_HEAD_REF: ${{ github.head_ref }}
FILTER_CHANGES: ${{ steps.filter.outputs.changes }}
run: |
# Mirror of the ALL_SERVICES definition from showcase_build.yml.
# Keep in sync — the matrix here must match the production build
# workflow so that every service buildable post-merge is also
# checked pre-merge.
#
# Fields carried over: dispatch_name, filter_key, context, image,
# timeout, lfs, build_args_*, dockerfile.
# Fields omitted (not needed for build-only): railway_id, health_path.
ALL_SERVICES='[
{"dispatch_name":"shell","filter_key":"shell","context":".","image":"showcase-shell","timeout":10,"lfs":true,"build_args_sha":"__GH_SHA__","build_args_branch":"__GH_REF_NAME__","dockerfile":"showcase/shell/Dockerfile"},
{"dispatch_name":"langgraph-python","filter_key":"langgraph_python","context":"showcase/integrations/langgraph-python","image":"showcase-langgraph-python","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"mastra","filter_key":"mastra","context":"showcase/integrations/mastra","image":"showcase-mastra","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"crewai-crews","filter_key":"crewai_crews","context":"showcase/integrations/crewai-crews","image":"showcase-crewai-crews","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"pydantic-ai","filter_key":"pydantic_ai","context":"showcase/integrations/pydantic-ai","image":"showcase-pydantic-ai","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"google-adk","filter_key":"google_adk","context":"showcase/integrations/google-adk","image":"showcase-google-adk","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"ag2","filter_key":"ag2","context":"showcase/integrations/ag2","image":"showcase-ag2","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"agno","filter_key":"agno","context":"showcase/integrations/agno","image":"showcase-agno","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"llamaindex","filter_key":"llamaindex","context":"showcase/integrations/llamaindex","image":"showcase-llamaindex","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"langgraph-fastapi","filter_key":"langgraph_fastapi","context":"showcase/integrations/langgraph-fastapi","image":"showcase-langgraph-fastapi","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"langgraph-typescript","filter_key":"langgraph_typescript","context":"showcase/integrations/langgraph-typescript","image":"showcase-langgraph-typescript","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"langroid","filter_key":"langroid","context":"showcase/integrations/langroid","image":"showcase-langroid","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"spring-ai","filter_key":"spring_ai","context":"showcase/integrations/spring-ai","image":"showcase-spring-ai","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"strands","filter_key":"strands","context":"showcase/integrations/strands","image":"showcase-strands","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"strands-typescript","filter_key":"strands_typescript","context":"showcase/integrations/strands-typescript","image":"showcase-strands-typescript","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"ms-agent-python","filter_key":"ms_agent_python","context":"showcase/integrations/ms-agent-python","image":"showcase-ms-agent-python","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"claude-sdk-typescript","filter_key":"claude_sdk_typescript","context":"showcase/integrations/claude-sdk-typescript","image":"showcase-claude-sdk-typescript","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"ms-agent-dotnet","filter_key":"ms_agent_dotnet","context":"showcase/integrations/ms-agent-dotnet","image":"showcase-ms-agent-dotnet","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"ms-agent-harness-dotnet","filter_key":"ms_agent_harness_dotnet","context":"showcase/integrations/ms-agent-harness-dotnet","image":"showcase-ms-agent-harness-dotnet","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"claude-sdk-python","filter_key":"claude_sdk_python","context":"showcase/integrations/claude-sdk-python","image":"showcase-claude-sdk-python","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"built-in-agent","filter_key":"built_in_agent","context":"showcase/integrations/built-in-agent","image":"showcase-built-in-agent","timeout":15,"lfs":false,"build_args":"","dockerfile":""},
{"dispatch_name":"shell-dojo","filter_key":"shell_dojo","context":".","image":"showcase-shell-dojo","timeout":10,"lfs":false,"build_args":"","dockerfile":"showcase/shell-dojo/Dockerfile"},
{"dispatch_name":"shell-dashboard","filter_key":"shell_dashboard","context":".","image":"showcase-shell-dashboard","timeout":10,"lfs":true,"build_args_sha":"__GH_SHA__","build_args_branch":"__GH_REF_NAME__","dockerfile":"showcase/shell-dashboard/Dockerfile"},
{"dispatch_name":"shell-docs","filter_key":"shell_docs","context":".","image":"showcase-shell-docs","timeout":10,"lfs":true,"build_args_sha":"__GH_SHA__","build_args_branch":"__GH_REF_NAME__","dockerfile":"showcase/shell-docs/Dockerfile"},
{"dispatch_name":"showcase-harness","filter_key":"showcase_harness","context":".","image":"showcase-harness","timeout":20,"lfs":false,"build_args":"","dockerfile":"showcase/harness/Dockerfile"},
{"dispatch_name":"showcase-aimock","filter_key":"showcase_aimock","context":"showcase/aimock","image":"showcase-aimock","timeout":5,"lfs":false,"build_args":"","dockerfile":"showcase/aimock/Dockerfile"}
]'
CHANGES="${FILTER_CHANGES:-[]}"
# PR event only — filter to changed services. Use jq --arg for
# untrusted PR metadata so branch names are encoded as JSON data,
# not interpolated into shell source.
MATRIX=$(echo "$ALL_SERVICES" | jq -c --argjson changes "$CHANGES" --arg sha "$PR_HEAD_SHA" --arg ref "$PR_HEAD_REF" '
[.[] |
if .build_args_sha == "__GH_SHA__" then .build_args_sha = $sha else . end |
if .build_args_branch == "__GH_REF_NAME__" then .build_args_branch = $ref else . end |
(.filter_key as $fk | select(
($changes | index("workflow_config") != null) or
($changes | index($fk) != null)
))]
')
echo "matrix=$MATRIX" >> $GITHUB_OUTPUT
if [ "$MATRIX" = "[]" ]; then
echo "has_changes=false" >> $GITHUB_OUTPUT
else
echo "has_changes=true" >> $GITHUB_OUTPUT
fi
build-check:
needs: [detect-changes]
if: needs.detect-changes.outputs.has_changes == 'true'
runs-on: depot-ubuntu-24.04-4
timeout-minutes: ${{ fromJSON(matrix.service.timeout) }}
permissions:
id-token: write
contents: read
strategy:
fail-fast: false
matrix:
service: ${{ fromJSON(needs.detect-changes.outputs.matrix) }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
lfs: ${{ matrix.service.lfs }}
persist-credentials: false
- name: Setup Depot
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1
- name: Prepare build args
id: build-args
env:
BUILD_ARGS_SHA: ${{ matrix.service.build_args_sha }}
BUILD_ARGS_BRANCH: ${{ matrix.service.build_args_branch }}
run: |
ARGS=""
if [ -n "$BUILD_ARGS_SHA" ]; then
ARGS="COMMIT_SHA=${BUILD_ARGS_SHA}"
ARGS="${ARGS}"$'\n'"BRANCH=${BUILD_ARGS_BRANCH}"
fi
# Use delimiter to safely pass multiline value
echo "args<<BUILDARGS_EOF" >> $GITHUB_OUTPUT
echo "$ARGS" >> $GITHUB_OUTPUT
echo "BUILDARGS_EOF" >> $GITHUB_OUTPUT
- name: Copy shared modules into build context
run: |
set -euo pipefail
CONTEXT="${{ matrix.service.context }}"
if [ -d "showcase/shared/python" ] && [ -d "$CONTEXT" ]; then
rm -rf "$CONTEXT/shared_python"
cp -r showcase/shared/python "$CONTEXT/shared_python"
fi
if [ -d "showcase/shared/typescript/tools" ] && [ -d "$CONTEXT" ]; then
rm -rf "$CONTEXT/shared_typescript"
mkdir -p "$CONTEXT/shared_typescript"
cp -r showcase/shared/typescript/tools "$CONTEXT/shared_typescript/tools"
fi
# Dereference tools/, shared-tools/, and _shared/ symlinks for the
# Docker context. Integration directories use symlinks pointing
# outside the build context (tools -> ../../shared/python/tools,
# _shared -> ../_shared for the CVDIAG bootstrap modules). Docker
# build contexts cannot follow a symlink whose target escapes the
# context root — buildkit fails the checksum with "too many
# symlinks: /_shared" — so each symlink is replaced with a real
# copy of its target. Mirrors stage_shared() in
# showcase/scripts/cli/_common.sh (the local bin/showcase path).
for link_name in tools shared-tools _shared; do
link_path="$CONTEXT/$link_name"
if [ -L "$link_path" ]; then
target="$(readlink -f "$link_path")"
if [ -d "$target" ]; then
rm "$link_path"
cp -r "$target" "$link_path"
fi
fi
done
- name: Build Docker image (no push)
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.18.0
with:
project: m2kw2wmmcp
context: ${{ matrix.service.context }}
file: ${{ matrix.service.dockerfile != '' && matrix.service.dockerfile || format('{0}/Dockerfile', matrix.service.context) }}
platforms: linux/amd64
push: false
build-args: ${{ steps.build-args.outputs.args }}
@@ -0,0 +1,280 @@
name: "Showcase: Capture Previews"
on:
workflow_run:
workflows: ["Showcase: Build & Push"]
types: [completed]
branches: [main]
workflow_dispatch:
inputs:
slug:
description: "Integration slug to capture (leave empty for all)"
type: string
default: ""
demo:
description: "Demo ID to capture (leave empty for all)"
type: string
default: ""
push:
branches: [main]
paths:
- "showcase/integrations/*/src/app/demos/**"
- "showcase/integrations/*/manifest.yaml"
concurrency:
group: showcase-capture-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: write
jobs:
capture:
name: Capture Preview MP4s
# Hoist the Slack webhook into an env var so step-level `if:`
# expressions can reference it — `secrets.*` is not a valid
# named-value inside `if:` and causes a workflow startup failure.
env:
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_OSS_ALERTS }}
runs-on: ubuntu-latest
# Captures demo previews as MP4 and uploads them to a GitHub release.
# Loop prevention: capture commits registry.json with the message below,
# which could re-trigger via push or the completed-deploy workflow_run.
# Skip when the triggering commit came from this workflow itself.
if: >-
(github.event_name != 'workflow_run' ||
!startsWith(github.event.workflow_run.head_commit.message, 'Update preview URLs in registry')) &&
(github.event_name != 'push' ||
!startsWith(github.event.head_commit.message, 'Update preview URLs in registry'))
timeout-minutes: 30
steps:
- name: Mint devops-bot token
# PROTECT_OUR_MAIN requires a PR for pushes to main; the default
# GITHUB_TOKEN fails with GH013 on the `Commit registry updates`
# step below. devops-bot (app-id 1108748) is a configured bypass
# actor on that ruleset — mint its installation token here and use
# it for checkout + push. Mirrors the pattern in
# CopilotKit/internal-skills's sync-versions.yml.
id: app-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: "1108748"
private-key: ${{ secrets.DEVOPS_BOT_PRIVATE_KEY }}
permission-contents: write
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
lfs: true
ref: ${{ github.event.workflow_run.head_branch || github.ref }}
token: ${{ steps.app-token.outputs.token }}
persist-credentials: false
- name: Ensure preview release exists
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh release view showcase-previews --repo ${{ github.repository }} >/dev/null 2>&1 || \
gh release create showcase-previews --repo ${{ github.repository }} \
--title "Showcase Preview Videos" \
--notes "Auto-generated demo preview videos for the showcase platform. Updated by CI." \
--latest=false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20
- name: Setup pnpm
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Install ffmpeg
run: sudo apt-get update && sudo apt-get install -y ffmpeg
- name: Install Playwright
run: npx playwright install chromium --with-deps
- name: Install script dependencies
working-directory: showcase/scripts
run: pnpm install --ignore-scripts || true
- name: Detect changed packages
id: detect
if: github.event_name == 'push'
run: |
# Find which package slugs had demo changes
CHANGED=$(git diff --name-only HEAD~1 HEAD -- 'showcase/integrations/*/src/app/demos/' 'showcase/integrations/*/manifest.yaml' | \
sed -n 's|showcase/integrations/\([^/]*\)/.*|\1|p' | sort -u | tr '\n' ',' | sed 's/,$//')
echo "slugs=$CHANGED" >> $GITHUB_OUTPUT
echo "Changed packages: $CHANGED"
- name: Generate registry
run: |
cd showcase/scripts
npm ci --silent
npx tsx generate-registry.ts
- name: Capture previews
env:
INPUT_SLUG: ${{ inputs.slug }}
INPUT_DEMO: ${{ inputs.demo }}
EVENT_NAME: ${{ github.event_name }}
DETECTED_SLUGS: ${{ steps.detect.outputs.slugs }}
run: |
# Build the args as an array (not a space-joined string) so a
# slug or demo value containing whitespace or shell metacharacters
# stays a single argument rather than being re-tokenized by the shell.
ARGS=()
# Use input slug/demo if provided (manual dispatch)
if [ -n "$INPUT_SLUG" ]; then
ARGS+=(--slug "$INPUT_SLUG")
fi
if [ -n "$INPUT_DEMO" ]; then
ARGS+=(--demo "$INPUT_DEMO")
fi
# Use detected slugs for push events (capture only changed)
if [ "$EVENT_NAME" = "push" ] && [ -n "$DETECTED_SLUGS" ]; then
# Capture each changed slug
IFS=',' read -ra SLUGS <<< "$DETECTED_SLUGS"
for slug in "${SLUGS[@]}"; do
npx tsx showcase/scripts/capture-previews.ts --slug "$slug" || true
done
else
npx tsx showcase/scripts/capture-previews.ts "${ARGS[@]}" || true
fi
- name: Configure git for push
run: |
git config user.name "devops-bot[bot]"
git config user.email "1108748+devops-bot[bot]@users.noreply.github.com"
git config --local url."https://x-access-token:${TOKEN}@github.com/".insteadOf "https://github.com/"
env:
TOKEN: ${{ steps.app-token.outputs.token }}
- name: Commit registry updates
run: |
cd showcase/shell/src/data
if git diff --quiet registry.json; then
echo "No registry changes"
exit 0
fi
git add -f registry.json
git commit -m "Update preview URLs in registry"
# Rebase-and-retry loop: the capture job can take ~30 minutes,
# during which other commits routinely land on main. Without
# this loop, `git push` loses the race and fails with "fetch
# first" (observed in runs 24799601181 and 24809331709 on
# 2026-04-22). Rebase our single registry commit onto the
# latest main and retry; bounded attempts so a persistent
# failure still surfaces rather than looping forever.
attempts=0
max_attempts=5
until git push; do
attempts=$((attempts + 1))
if [ "$attempts" -ge "$max_attempts" ]; then
echo "::error::push failed after $max_attempts rebase attempts"
exit 1
fi
echo "push rejected — rebasing onto latest main (attempt $attempts/$max_attempts)"
git pull --rebase origin "$(git rev-parse --abbrev-ref HEAD)"
done
# Slack failure alert. This workflow only runs on main-branch pushes,
# workflow_run completions, and manual dispatch — all of which
# constitute "production" events where silent failures (e.g. the
# GH013 PROTECT_OUR_MAIN regression that motivated PR #4159) must
# surface in #oss-alerts. Extracts failed step name + first
# meaningful error line so the payload is triage-ready rather than
# forcing a click-through, per the oss-alerts detail policy.
#
# Must NEVER fail the job (runs on failure() already; a crash here
# would compound the original failure and could block the notify
# step). All extraction uses best-effort fallbacks so a malformed
# jobs response or truncated log still yields sane defaults.
- name: Extract failure details for Slack
id: extract
if: failure() && env.SLACK_WEBHOOK != ''
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_REPO: ${{ github.repository }}
RUN_ID: ${{ github.run_id }}
run: |
set +e # best-effort: never block the notify step below
# --- Find the currently-running job and its first failed step ---
# The jobs API returns every job in the run. Match by job name
# first (mirrors `jobs.capture.name`); fall back to first job
# with a failed step if name match misses (e.g. future rename).
jobs_json=$(gh api "/repos/${GH_REPO}/actions/runs/${RUN_ID}/jobs" --paginate 2>/dev/null)
job_id=$(printf '%s' "$jobs_json" | jq -r '
.jobs // []
| map(select(.name == "Capture Preview MP4s"))
| (.[0].id // empty)
' 2>/dev/null)
if [ -z "$job_id" ]; then
job_id=$(printf '%s' "$jobs_json" | jq -r '
.jobs // []
| map(select(.steps // [] | map(.conclusion) | index("failure")))
| (.[0].id // empty)
' 2>/dev/null)
fi
failed_step=$(printf '%s' "$jobs_json" | jq -r --arg id "$job_id" '
.jobs // []
| map(select((.id|tostring) == $id))
| (.[0].steps // [])
| map(select(.conclusion == "failure"))
| (.[0].name // "unknown step")
' 2>/dev/null)
[ -z "$failed_step" ] && failed_step="unknown step"
# --- Pull log and extract first meaningful error line ------------
# `gh run view --log-failed` output is TSV: job\tstep\ttimestamp +
# content. Strip the three leading columns, strip ANSI escape
# codes + BOM, skip runner/group/env header noise, grab first
# line matching a recognised error marker. Truncate to ~300
# chars so the Slack payload stays under the 800-char budget.
error_excerpt="see workflow run for details"
if [ -n "$job_id" ]; then
log_excerpt=$(gh run view "$RUN_ID" --repo "$GH_REPO" --log-failed --job="$job_id" 2>/dev/null \
| awk -F'\t' 'NF>=3 { sub(/^[\xEF\xBB\xBF]?[0-9T:.\-Z ]+/, "", $3); print $3 }' \
| sed 's/\x1b\[[0-9;]*[a-zA-Z]//g' \
| grep -vE '^(##\[|shell: |env: |Run |[[:space:]]*$)' \
| grep -m1 -E '^\[(FAIL|ERROR)\]|^Error:|^error:|^::error' \
| head -c 300)
if [ -n "$log_excerpt" ]; then
error_excerpt="$log_excerpt"
fi
fi
# --- Emit to $GITHUB_ENV using heredoc delimiter -----------------
# Heredoc delimiter protects against values with `=` or newlines
# breaking the KEY=VALUE format.
{
echo "failed_step<<EOF_FAILED_STEP_b3f2"
printf '%s\n' "$failed_step"
echo "EOF_FAILED_STEP_b3f2"
echo "error_excerpt<<EOF_ERROR_EXCERPT_b3f2"
printf '%s\n' "$error_excerpt"
echo "EOF_ERROR_EXCERPT_b3f2"
} >> "$GITHUB_ENV"
exit 0 # belt-and-suspenders: never propagate a failure
- name: Notify Slack (failure)
if: failure() && env.SLACK_WEBHOOK != ''
uses: slackapi/slack-github-action@0d95c9a7becc1e6e297d76df9bc735c44f4cbcbc # v3.0.5
with:
webhook: ${{ secrets.SLACK_WEBHOOK_OSS_ALERTS }}
webhook-type: incoming-webhook
# Defensive: wrap dynamic values via toJSON(format(...)) so that
# if github.repository or extracted failed_step / error_excerpt
# contain characters that would break the JSON payload (quotes,
# backslashes, newlines), the value is safely JSON-encoded.
# Matches the pattern used in showcase_validate.yml.
payload: |
{ "text": ${{ toJSON(format(':x: *Showcase: Capture Previews*: failed — {0}: {1} | <https://github.com/{2}/actions/runs/{3}|View run>', env.failed_step, env.error_excerpt, github.repository, github.run_id)) }} }
- name: Log (no Slack — webhook unset)
if: failure() && env.SLACK_WEBHOOK == ''
run: |
echo "::warning::showcase_capture-previews failed but SLACK_WEBHOOK_OSS_ALERTS is not set; no Slack notification sent."
+370
View File
@@ -0,0 +1,370 @@
name: "Showcase: Verify Deploy"
# Triggered after "Showcase: Build & Push" completes. Verifies that the
# STAGING redeploy from the build workflow actually produced healthy
# services. Push-to-main redeploys staging only. This workflow is the
# staging gate; verify-deploy.ts is the parameterized probe driven off
# showcase/scripts/railway-envs.ts (the SSOT).
on:
workflow_run:
workflows: ["Showcase: Build & Push"]
types: [completed]
branches: [main]
workflow_dispatch:
inputs:
service:
description: "Service to verify (SSOT key or dispatch_name; 'all' = everything probe-eligible)"
required: false
default: "all"
type: string
concurrency:
group: showcase-verify-deploy
cancel-in-progress: true
permissions:
contents: read
jobs:
resolve-matrix:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
actions: read
if: >-
github.event_name == 'workflow_dispatch' ||
github.event.workflow_run.conclusion == 'success'
outputs:
services_csv: ${{ steps.matrix.outputs.services_csv }}
has_services: ${{ steps.matrix.outputs.has_services }}
build_run_id: ${{ github.event.workflow_run.id }}
build_run_url: ${{ github.event.workflow_run.html_url }}
redeploy_red: ${{ steps.redeploy-gate.outputs.redeploy_red }}
ok_services: ${{ steps.redeploy-gate.outputs.ok_services }}
failed_services: ${{ steps.redeploy-gate.outputs.failed_services }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
- name: Check whether build uploaded a redeploy-summary artifact
id: check-redeploy-summary
# `actions/download-artifact@v4` with a `name:` HARD-FAILS when the
# named artifact does not exist. The artifact legitimately does
# NOT exist whenever the upstream build ran but redeployed nothing
# — e.g. a push touching `showcase/**` that fires the build's
# `paths:` filter, but `detect-changes` finds no buildable service
# changed, so `redeploy-staging` is skipped and never uploads.
# The build still concludes `success`, so this workflow fires on
# `workflow_run` and `resolve-matrix` runs. Without this pre-check
# the unguarded download would fail the job and (via
# `enforce-redeploy-gate` tripping on `result == 'failure'`) flip
# the whole deploy workflow RED — a false-red on a routine
# showcase-docs/script-only change. Listing artifacts via the API
# only requires `actions: read`, which `resolve-matrix` already has.
if: github.event_name == 'workflow_run'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
// Fail loud on API error: github-script propagates unhandled
// rejections, which fails this step and (via the
// resolve-matrix.result == 'failure' clause on
// enforce-redeploy-gate) reds the workflow. Do NOT wrap this
// in try/catch — silently defaulting summary_present=false on
// a 5xx/403 would open the gate (skip verify) on what is
// actually a transient API failure, hiding a broken pipeline.
//
// Use the `name` query-param on listWorkflowRunArtifacts to
// ask the API to return only the redeploy-summary artifact.
// This makes the lookup robust to the build run uploading
// many artifacts (per-slot build-result-* + build-results +
// redeploy-summary — already ~28 today, well within per_page
// 100, but a future expansion past 100 would otherwise risk a
// false "absent" if redeploy-summary fell off the first page).
// The endpoint accepts `name` for an exact-match filter; we
// still paginate defensively in case the API returns multiple
// rows (e.g. an artifact with the same name re-uploaded).
const runId = context.payload.workflow_run.id;
const iterator = github.paginate.iterator(
github.rest.actions.listWorkflowRunArtifacts,
{
owner: context.repo.owner,
repo: context.repo.repo,
run_id: runId,
name: "redeploy-summary",
per_page: 100,
},
);
let present = false;
for await (const page of iterator) {
if ((page.data || []).some((a) => a.name === "redeploy-summary")) {
present = true;
break;
}
}
core.setOutput("summary_present", present ? "true" : "false");
core.info(`redeploy-summary present for run ${runId}: ${present}`);
- name: Download redeploy summary from build workflow
# Three cases now handled distinctly:
# (a) workflow_dispatch — no `workflow_run` payload exists; the
# download is skipped and the bash `[ ! -f "$SUMMARY" ]`
# branch below treats it as "nothing to gate" (correct: a
# manual dispatch is not gated by a build's per-service set).
# (b) workflow_run + artifact absent — the upstream build
# redeployed nothing (e.g. no service had buildable
# changes); the precheck reports `summary_present=false`,
# the download is skipped, and the bash branch no-ops the
# gate. The deploy workflow should NOT red here — there is
# nothing to gate.
# (c) workflow_run + artifact PRESENT — we always attempt the
# download. We intentionally do NOT set
# `continue-on-error: true`: if the artifact exists but the
# download genuinely fails (network/permission), silently
# opening the gate would let verify probe the FULL service
# set against stale `:latest` and mask a broken redeploy as
# a green deploy. Fail loud instead.
if: >-
github.event_name == 'workflow_run' &&
steps.check-redeploy-summary.outputs.summary_present == 'true'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: redeploy-summary
path: .redeploy
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: Gate on staging redeploy errors
id: redeploy-gate
run: |
set -euo pipefail
SUMMARY=".redeploy/summary.json"
if [ ! -f "$SUMMARY" ]; then
echo "No redeploy summary found (workflow_dispatch path or build did not upload). Skipping gate."
{
echo "redeploy_red=false"
echo "ok_services="
echo "failed_services="
} >> "$GITHUB_OUTPUT"
exit 0
fi
# Shape guard: redeploy-env.ts writes per-entry `status` of
# exactly "ok" or "error". If the schema ever drifts (e.g.
# `status`→`state`, `ok`→`success`) every `select(.status==...)`
# silently yields empty → redeploy_red=false AND ok_services=""
# → resolve-verify-matrix skips verify on a real unverified
# redeploy = green CI on a broken release. Refuse the ambiguity:
# if the file has entries but ANY entry is missing a valid
# ok|error status (partial drift — some rows on the legacy schema,
# some on the new one), fail loud here so enforce-redeploy-gate
# reds the workflow (resolve-matrix.result == 'failure' fans into
# the gate). The previous TOTAL>0 && WITH_STATUS==0 check was
# all-or-nothing and silently dropped the drifted rows on a mixed
# summary.
TOTAL=$(jq 'length' "$SUMMARY")
WITH_STATUS=$(jq '[.[] | select(.status == "ok" or .status == "error")] | length' "$SUMMARY")
if [ "$TOTAL" -gt 0 ] && [ "$WITH_STATUS" -lt "$TOTAL" ]; then
echo "::error::summary.json shape drift: $WITH_STATUS of $TOTAL entries have status ok|error"
exit 1
fi
# Per spec §3: the workflow MUST turn red on any staging
# status:"error", while verify still runs against the success-set.
ERRORS=$(jq -c '[.[] | select(.status == "error")]' "$SUMMARY")
ERROR_COUNT=$(echo "$ERRORS" | jq 'length')
OK=$(jq -r '[.[] | select(.status == "ok") | .service] | join(",")' "$SUMMARY")
FAILED=$(jq -r '[.[] | select(.status == "error") | .service] | join(",")' "$SUMMARY")
echo "ok_services=$OK" >> "$GITHUB_OUTPUT"
echo "failed_services=$FAILED" >> "$GITHUB_OUTPUT"
if [ "$ERROR_COUNT" -gt 0 ]; then
echo "::error::Staging redeploy reported $ERROR_COUNT per-service error(s):"
echo "$ERRORS" | jq -r '.[] | " - \(.service): \(.error)"'
echo "redeploy_red=true" >> "$GITHUB_OUTPUT"
else
echo "redeploy_red=false" >> "$GITHUB_OUTPUT"
fi
- name: Build verify matrix from SSOT
id: matrix
env:
DISPATCH_SERVICE: ${{ github.event.inputs.service }}
OK_FROM_REDEPLOY: ${{ steps.redeploy-gate.outputs.ok_services }}
EVENT_NAME: ${{ github.event_name }}
SUMMARY_PRESENT: ${{ steps.check-redeploy-summary.outputs.summary_present }}
# The decision-table that picks the verify matrix lives in
# showcase/scripts/resolve-verify-matrix.ts (pure function +
# unit tests). Summary of cases:
# - workflow_dispatch + 'all'/empty → full probe-eligible set.
# - workflow_dispatch + specific svc → just that service
# (unknown name → error exit).
# - workflow_run + summary_present=false → has_services=false
# (build redeployed nothing).
# - workflow_run + summary_present=true + ok empty
# → has_services=false (success-set empty; verify is skipped).
# In practice redeploy-env.ts only emits status ok|error,
# so this branch implies redeploy_red=true and
# enforce-redeploy-gate reds the workflow independently —
# skipping verify here is correct (no ok services left to
# probe; the gate has already turned the workflow red).
# - workflow_run + summary_present=true + ok non-empty
# → intersect ok_services (SSOT key OR dispatchName aliases)
# with probe.staging-eligible SSOT services. has_services
# reflects CSV emptiness. When the intersection collapses
# to empty, verify is skipped; if there were per-service
# errors, enforce-redeploy-gate reds the workflow — otherwise
# the run is correctly green (every redeploy succeeded, just
# none probe-eligible).
run: npx tsx showcase/scripts/resolve-verify-matrix.ts
verify:
needs: [resolve-matrix]
if: needs.resolve-matrix.outputs.has_services == 'true'
runs-on: ubuntu-latest
timeout-minutes: 20
environment: railway
permissions:
contents: read
actions: read
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
- name: Install
working-directory: showcase/scripts
run: npm ci
- name: Run verify-deploy --env staging
env:
RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}
SERVICES_CSV: ${{ needs.resolve-matrix.outputs.services_csv }}
run: |
if [ -z "$RAILWAY_TOKEN" ]; then
echo "::error::RAILWAY_TOKEN is not set"
exit 1
fi
npx tsx showcase/scripts/verify-deploy.ts --env staging --services "$SERVICES_CSV"
enforce-redeploy-gate:
# Spec §3: workflow turns red on any per-service redeploy error, even
# if verify against the success-set passes. This job is independent of
# verify so the user sees both signals (what was redeployed badly,
# what was redeployed and is unhealthy) rather than one masking the other.
needs: [resolve-matrix]
# Trip the gate on EITHER a per-service redeploy error OR a complete
# resolve-matrix failure. A resolve-matrix failure leaves the
# `redeploy_red` output empty (jobs that fail mid-step don't publish
# outputs reliably), which would otherwise let an upstream crash slip
# past as "not red" — a silent bypass of the gate.
if: always() && (needs.resolve-matrix.outputs.redeploy_red == 'true' || needs.resolve-matrix.result == 'failure')
runs-on: ubuntu-latest
timeout-minutes: 2
permissions:
contents: read
steps:
- name: Fail workflow on staging redeploy errors
run: |
echo "::error::One or more staging services reported status:error in the redeploy summary."
echo "See the resolve-matrix job's 'Gate on staging redeploy errors' step for details."
exit 1
notify-harness:
needs: [resolve-matrix, verify, enforce-redeploy-gate]
if: always() && needs.resolve-matrix.outputs.has_services == 'true'
permissions:
contents: read
actions: read
runs-on: ubuntu-latest
timeout-minutes: 3
steps:
- name: Compute deploy-result payload
id: payload
env:
VERIFY_RESULT: ${{ needs.verify.result }}
OK_SERVICES: ${{ needs.resolve-matrix.outputs.ok_services }}
FAILED_SERVICES: ${{ needs.resolve-matrix.outputs.failed_services }}
BUILD_RUN_ID: ${{ needs.resolve-matrix.outputs.build_run_id }}
BUILD_RUN_URL: ${{ needs.resolve-matrix.outputs.build_run_url }}
RUN_ID: ${{ github.run_id }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
# Build the payload to match the harness ingest schema at
# showcase/harness/src/http/webhooks/deploy.ts (`.strict()`):
# {runId, runUrl, buildRunId?, buildRunUrl?, services[], succeeded[], failed[], cancelled}
# No `state` key — the schema rejects unknown fields. State is
# derived downstream from (failed.length, cancelled, succeeded).
# - succeeded = ok_services from the redeploy gate
# - failed = failed_services from the redeploy gate
# - services = union (full attempted set, per deploy.ts:307)
# - cancelled = verify job conclusion == "cancelled"
# jq -R/split handles empty CSV → [] safely.
run: |
set -euo pipefail
# `echo` (not printf '%s') so empty CSV → newline → jq -R reads
# an empty record → split produces [""] → filter to []. Without
# the trailing newline jq sees no records and --argjson barfs
# on the empty string.
SUCCEEDED=$(echo "${OK_SERVICES:-}" | jq -R 'split(",") | map(select(length>0))')
FAILED=$(echo "${FAILED_SERVICES:-}" | jq -R 'split(",") | map(select(length>0))')
if [ "$VERIFY_RESULT" = "cancelled" ]; then
CANCELLED="true"
else
CANCELLED="false"
fi
# buildRunId/buildRunUrl are .url()-validated in the harness
# Zod schema, so we must omit them entirely when empty rather
# than emit "" (which fails .url()). Same for runUrl: only
# emit it if non-empty (it's optional in the schema). The base
# object always carries runId + succeeded/failed/services/cancelled;
# we conditionally splice in the optional keys.
PAYLOAD=$(jq -cn \
--arg runId "$RUN_ID" --arg runUrl "${RUN_URL:-}" \
--arg buildRunId "${BUILD_RUN_ID:-}" --arg buildRunUrl "${BUILD_RUN_URL:-}" \
--argjson succeeded "$SUCCEEDED" \
--argjson failed "$FAILED" \
--argjson cancelled "$CANCELLED" \
'
{
runId: $runId,
services: ($succeeded + $failed | unique),
succeeded: $succeeded,
failed: $failed,
cancelled: $cancelled
}
+ (if $runUrl == "" then {} else {runUrl: $runUrl} end)
+ (if $buildRunId == "" then {} else {buildRunId: $buildRunId} end)
+ (if $buildRunUrl == "" then {} else {buildRunUrl: $buildRunUrl} end)
')
{
echo "payload<<EOF_PAYLOAD"
echo "$PAYLOAD"
echo "EOF_PAYLOAD"
} >> "$GITHUB_OUTPUT"
- name: POST deploy result to showcase-harness
env:
SHOWCASE_HARNESS_URL: ${{ secrets.SHOWCASE_HARNESS_URL }}
SHARED_SECRET: ${{ secrets.SHOWCASE_HARNESS_SHARED_SECRET }}
PAYLOAD: ${{ steps.payload.outputs.payload }}
run: |
set -euo pipefail
if [ -z "${SHOWCASE_HARNESS_URL:-}" ] || [ -z "${SHARED_SECRET:-}" ]; then
echo "::warning::SHOWCASE_HARNESS_URL or SHOWCASE_HARNESS_SHARED_SECRET not set; skipping webhook"
exit 0
fi
TS=$(date +%s)
BODY_SHA=$(printf '%s' "$PAYLOAD" | openssl dgst -sha256 -hex | awk '{print $2}')
CANONICAL="POST|/webhooks/deploy|${TS}|${BODY_SHA}"
SIG=$(printf '%s' "$CANONICAL" | openssl dgst -sha256 -hmac "$SHARED_SECRET" -hex | awk '{print $2}')
curl -sS --fail-with-body \
-X POST "${SHOWCASE_HARNESS_URL%/}/webhooks/deploy" \
-H 'content-type: application/json' \
-H "X-Ops-Timestamp: ${TS}" \
-H "X-Ops-Signature: sha256=${SIG}" \
--data-raw "$PAYLOAD"
@@ -0,0 +1,37 @@
name: "showcase / eval-webhook / build"
on:
push:
branches: [main]
paths:
- "showcase/eval-webhook/**"
workflow_dispatch:
permissions:
contents: read
packages: write
jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Log in to GHCR
uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7
with:
context: showcase/eval-webhook
push: true
tags: |
ghcr.io/copilotkit/showcase-eval-webhook:latest
ghcr.io/copilotkit/showcase-eval-webhook:${{ github.sha }}
+621
View File
@@ -0,0 +1,621 @@
name: "showcase / eval"
# SECURITY — residual trust model (read before editing):
#
# This workflow executes `showcase/bin/showcase eval` against PR-HEAD code.
# Hardening layers mirror test_e2e-showcase-on-demand.yml:
# - `getCollaboratorPermissionLevel` gate limits the trigger to users with
# write (or higher) access — third-party commenters cannot spawn runs.
# - workflow-level `permissions: contents: read` means the eval job's
# GITHUB_TOKEN cannot mutate the repo; the `post-result` job gets write
# perms scoped to just the final PR comment.
# - `persist-credentials: false` on `actions/checkout` prevents the token
# from leaking to PR-HEAD build hooks.
# - `env:`-based pattern for UNTRUSTED values (comment body) prevents shell
# injection.
# - Slug whitelist (`^[a-z0-9-]+$`) prevents path traversal.
#
# Known TOCTOU — comment-trigger vs resolved HEAD SHA:
# Same gap as test_e2e-showcase-on-demand.yml. The `pulls.get` call resolves
# whatever HEAD is current at job start, not at comment time. The permission
# gate + code-review social contract are the mitigations.
on:
issue_comment:
types: [created]
workflow_dispatch:
inputs:
pr_number:
description: "PR number to evaluate"
required: true
type: string
check_run_id:
description: "Check Run ID to update with results"
required: false
type: string
level:
description: "Eval depth level"
required: false
default: "d5"
type: string
concurrency:
group: showcase-eval-${{ github.event.inputs.pr_number || github.event.issue.number || github.run_id }}
cancel-in-progress: true
permissions:
contents: read
jobs:
gate:
if: >
github.event.issue.pull_request
&& startsWith(github.event.comment.body, '/eval')
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
pr_sha: ${{ steps.pr-ref.outputs.sha }}
pr_number: ${{ steps.pr-ref.outputs.pr_number }}
level: ${{ steps.parse.outputs.level }}
scope_flag: ${{ steps.parse.outputs.scope_flag }}
scope_display: ${{ steps.parse.outputs.scope_display }}
permissions:
contents: read
pull-requests: write
issues: write
steps:
- name: Check commenter has write access
id: auth
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
const { data: perm } = await github.rest.repos.getCollaboratorPermissionLevel({
owner: context.repo.owner,
repo: context.repo.repo,
username: context.payload.comment.user.login,
});
const level = perm.permission;
if (!['admin', 'write'].includes(level)) {
core.setFailed(`User ${context.payload.comment.user.login} has '${level}' access — write access required to trigger /eval.`);
return;
}
core.info(`User ${context.payload.comment.user.login} has '${level}' access — authorized.`);
- name: Parse /eval command
id: parse
env:
COMMENT_BODY: ${{ github.event.comment.body }}
run: |
set -euo pipefail
# Extract the first line of the comment to parse the command.
FIRST_LINE=$(printf '%s' "$COMMENT_BODY" | head -n1)
# Parse: /eval → d5 affected
# /eval d5 → d5 affected
# /eval d5 all → d5 all
# /eval d5 mastra,agno → d5 specific slugs
ARGS=$(printf '%s' "$FIRST_LINE" | sed 's|^/eval[[:space:]]*||')
# Default level
LEVEL="d5"
SCOPE=""
SCOPE_FLAG=""
SCOPE_DISPLAY=""
if [ -z "$ARGS" ]; then
# Bare /eval — d5 affected
SCOPE_FLAG="--scope affected"
SCOPE_DISPLAY="affected integrations"
else
# First token is the level (only d5 supported for now)
LEVEL_TOKEN=$(printf '%s' "$ARGS" | awk '{print $1}')
REST=$(printf '%s' "$ARGS" | sed "s|^${LEVEL_TOKEN}[[:space:]]*||")
# Validate level
case "$LEVEL_TOKEN" in
d5) LEVEL="d5" ;;
*)
echo "::error::Unknown eval level '$LEVEL_TOKEN'. Supported: d5"
exit 1
;;
esac
if [ -z "$REST" ]; then
# /eval d5 — affected
SCOPE_FLAG="--scope affected"
SCOPE_DISPLAY="affected integrations"
elif [ "$REST" = "all" ]; then
# /eval d5 all
SCOPE_FLAG="--scope all"
SCOPE_DISPLAY="all integrations"
else
# /eval d5 mastra,agno → specific slugs
# Validate each slug against ^[a-z0-9-]+$ to prevent injection
IFS=',' read -ra SLUGS <<< "$REST"
for s in "${SLUGS[@]}"; do
s=$(printf '%s' "$s" | xargs) # trim whitespace
case "$s" in
''|*[!a-z0-9-]*)
echo "::error::Invalid slug '$s' — must match ^[a-z0-9-]+$"
exit 1
;;
esac
done
# Reassemble validated slugs into a clean comma-separated string
# (trims whitespace the user may have typed, e.g. "mastra, agno")
CLEAN_REST=$(printf '%s' "$REST" | tr -d ' ')
SCOPE_FLAG="--slug $CLEAN_REST"
SCOPE_DISPLAY="$CLEAN_REST"
fi
fi
echo "level=$LEVEL" >> "$GITHUB_OUTPUT"
echo "scope_flag=$SCOPE_FLAG" >> "$GITHUB_OUTPUT"
echo "scope_display=$SCOPE_DISPLAY" >> "$GITHUB_OUTPUT"
- name: Resolve PR HEAD ref
id: pr-ref
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
const { data: pr } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.issue.number,
});
if (pr.state !== 'open') {
core.setFailed(`PR #${pr.number} is ${pr.state} (not open). Refusing to run eval on a non-open PR.`);
return;
}
core.setOutput('sha', pr.head.sha);
core.setOutput('pr_number', pr.number);
- name: React with rocket emoji
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
await github.rest.reactions.createForIssueComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: context.payload.comment.id,
content: 'rocket',
});
- name: Post running status comment
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
env:
LEVEL: ${{ steps.parse.outputs.level }}
SCOPE_DISPLAY: ${{ steps.parse.outputs.scope_display }}
with:
script: |
const level = process.env.LEVEL;
const scope = process.env.SCOPE_DISPLAY;
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: [
`<!-- showcase-eval-status -->`,
`### Showcase Eval`,
``,
`| | |`,
`|---|---|`,
`| **Status** | Running... |`,
`| **Level** | \`${level}\` |`,
`| **Scope** | ${scope} |`,
`| **Run** | [View workflow](${runUrl}) |`,
].join('\n'),
});
dispatch-gate:
if: github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
timeout-minutes: 2
permissions:
contents: read
pull-requests: read
outputs:
pr_sha: ${{ steps.resolve.outputs.sha }}
pr_number: ${{ github.event.inputs.pr_number }}
level: ${{ github.event.inputs.level || 'd5' }}
scope_flag: "--scope affected"
scope_display: "affected"
check_run_id: ${{ github.event.inputs.check_run_id }}
steps:
- name: Resolve PR HEAD SHA
id: resolve
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
const pr = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: Number(process.env.PR_NUMBER),
});
if (pr.data.state !== 'open') {
core.setFailed(`PR #${process.env.PR_NUMBER} is not open`);
return;
}
core.setOutput('sha', pr.data.head.sha);
env:
PR_NUMBER: ${{ github.event.inputs.pr_number }}
eval:
needs: [gate, dispatch-gate]
if: always() && (needs.gate.result == 'success' || needs.dispatch-gate.result == 'success')
runs-on: depot-ubuntu-24.04-16
timeout-minutes: 45
permissions:
contents: read
env:
PR_SHA: ${{ needs.gate.outputs.pr_sha || needs.dispatch-gate.outputs.pr_sha }}
PR_NUMBER: ${{ needs.gate.outputs.pr_number || needs.dispatch-gate.outputs.pr_number }}
EVAL_LEVEL: ${{ needs.gate.outputs.level || needs.dispatch-gate.outputs.level || 'd5' }}
EVAL_SCOPE_FLAG: ${{ needs.gate.outputs.scope_flag || needs.dispatch-gate.outputs.scope_flag }}
CHECK_RUN_ID: ${{ needs.dispatch-gate.outputs.check_run_id || '' }}
outputs:
result_json: ${{ steps.run-eval.outputs.result_json }}
exit_code: ${{ steps.run-eval.outputs.exit_code }}
stderr_excerpt: ${{ steps.run-eval.outputs.stderr_excerpt }}
steps:
- name: Checkout PR HEAD
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
ref: ${{ env.PR_SHA }}
fetch-depth: 0
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Install dependencies
run: pnpm install --ignore-scripts
- name: Install Playwright chromium
run: npx playwright install chromium --with-deps
- name: Run showcase eval
id: run-eval
run: |
set -o pipefail
# Build the command. EVAL_SCOPE_FLAG may contain spaces (e.g. "--slug mastra,agno")
# so we intentionally leave it unquoted for word splitting.
# shellcheck disable=SC2086
CMD="showcase/bin/showcase eval --${EVAL_LEVEL} ${EVAL_SCOPE_FLAG} --parallel 8 --json --baseline compare --timeout 60000 --ci"
echo "::group::Running: $CMD"
EXIT_CODE=0
# Capture both stdout (JSON results) and stderr separately.
# Tee stderr to a file for excerpt extraction on failure.
$CMD > eval-results.json 2> eval-stderr.log || EXIT_CODE=$?
echo "::endgroup::"
echo "exit_code=$EXIT_CODE" >> "$GITHUB_OUTPUT"
if [ -f eval-results.json ] && [ -s eval-results.json ]; then
# GitHub outputs have a 1MB limit; truncate if needed
RESULT_SIZE=$(wc -c < eval-results.json)
if [ "$RESULT_SIZE" -gt 900000 ]; then
echo "::warning::eval-results.json exceeds 900KB ($RESULT_SIZE bytes), truncating for output"
head -c 900000 eval-results.json > eval-results-truncated.json
echo "result_json<<GHEOF" >> "$GITHUB_OUTPUT"
cat eval-results-truncated.json >> "$GITHUB_OUTPUT"
echo "GHEOF" >> "$GITHUB_OUTPUT"
else
echo "result_json<<GHEOF" >> "$GITHUB_OUTPUT"
cat eval-results.json >> "$GITHUB_OUTPUT"
echo "GHEOF" >> "$GITHUB_OUTPUT"
fi
else
echo 'result_json={}' >> "$GITHUB_OUTPUT"
fi
# Capture last 50 lines of stderr for failure reporting
if [ -f eval-stderr.log ] && [ -s eval-stderr.log ]; then
echo "stderr_excerpt<<GHEOF" >> "$GITHUB_OUTPUT"
tail -n 50 eval-stderr.log >> "$GITHUB_OUTPUT"
echo "GHEOF" >> "$GITHUB_OUTPUT"
else
echo "stderr_excerpt=" >> "$GITHUB_OUTPUT"
fi
# Propagate the exit code so the job status reflects eval outcome
exit $EXIT_CODE
- name: Upload eval artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: showcase-eval-results
path: |
eval-results.json
eval-stderr.log
retention-days: 14
if-no-files-found: ignore
post-result:
needs: [gate, dispatch-gate, eval]
if: always() && (needs.gate.result == 'success' || needs.dispatch-gate.result == 'success')
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
pull-requests: write
issues: write
checks: write
steps:
- name: Post eval results to PR
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
env:
EVAL_STATUS: ${{ needs.eval.result }}
RESULT_JSON: ${{ needs.eval.outputs.result_json }}
STDERR_EXCERPT: ${{ needs.eval.outputs.stderr_excerpt }}
EXIT_CODE: ${{ needs.eval.outputs.exit_code }}
LEVEL: ${{ needs.gate.outputs.level || needs.dispatch-gate.outputs.level || 'd5' }}
SCOPE_DISPLAY: ${{ needs.gate.outputs.scope_display || needs.dispatch-gate.outputs.scope_display || 'affected' }}
PR_NUMBER: ${{ needs.gate.outputs.pr_number || needs.dispatch-gate.outputs.pr_number }}
with:
script: |
const evalStatus = process.env.EVAL_STATUS;
const resultJson = process.env.RESULT_JSON || '{}';
const stderrExcerpt = process.env.STDERR_EXCERPT || '';
const exitCode = process.env.EXIT_CODE || 'unknown';
const level = process.env.LEVEL;
const scope = process.env.SCOPE_DISPLAY;
const prNumber = parseInt(process.env.PR_NUMBER, 10);
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
let body = '';
if (evalStatus === 'success') {
// Parse JSON results and build markdown table
let results;
try {
results = JSON.parse(resultJson);
} catch (e) {
// JSON parse failed — report raw
body = [
`<!-- showcase-eval-result -->`,
`### Showcase Eval Results`,
``,
`| | |`,
`|---|---|`,
`| **Verdict** | :warning: PARSE ERROR |`,
`| **Level** | \`${level}\` |`,
`| **Scope** | ${scope} |`,
`| **Run** | [View workflow](${runUrl}) |`,
``,
`Could not parse eval JSON output:`,
'```',
e.message,
'```',
``,
`<details><summary>Raw output</summary>`,
``,
'```json',
resultJson.substring(0, 50000),
'```',
``,
`</details>`,
].join('\n');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
body,
});
return;
}
// Build results table from the JSON.
// Expected shape: { summary: { total, pass, fail, skip, duration_ms },
// results: { slug: { testName: { status, duration_ms, error? } } } }
const summary = results.summary || {};
const resultsMap = results.results || {};
const total = summary.total || 0;
const passed = summary.pass || 0;
const failed = summary.fail || 0;
const skipped = summary.skip || 0;
const verdict = failed === 0
? ':white_check_mark: **SAFE TO MERGE**'
: `:x: **FAILURES DETECTED** (${failed}/${total} failed)`;
// Build per-integration results table from nested object
let tableRows = '';
const rows = [];
for (const [slug, tests] of Object.entries(resultsMap)) {
for (const [testName, r] of Object.entries(tests)) {
const icon = r.status === 'pass' ? ':white_check_mark:'
: r.status === 'fail' ? ':x:'
: r.status === 'skip' ? ':fast_forward:'
: r.status === 'error' ? ':boom:'
: r.status === 'build_failed' ? ':hammer:'
: r.status === 'unhealthy' ? ':warning:'
: ':question:';
const duration = r.duration_ms ? `${(r.duration_ms / 1000).toFixed(1)}s` : '-';
const detail = r.error ? r.error.substring(0, 120) : '-';
rows.push(`| ${icon} | \`${slug}\` | ${testName} | ${r.status || 'unknown'} | ${duration} | ${detail} |`);
}
}
if (rows.length > 0) {
tableRows = rows.join('\n');
}
body = [
`<!-- showcase-eval-result -->`,
`### Showcase Eval Results`,
``,
`| | |`,
`|---|---|`,
`| **Verdict** | ${verdict} |`,
`| **Level** | \`${level}\` |`,
`| **Scope** | ${scope} |`,
`| **Total** | ${total} |`,
`| **Passed** | ${passed} |`,
`| **Failed** | ${failed} |`,
`| **Skipped** | ${skipped} |`,
`| **Run** | [View workflow](${runUrl}) |`,
``,
].join('\n');
if (tableRows) {
body += [
`#### Per-Integration Results`,
``,
`| | Integration | Test | Status | Duration | Details |`,
`|---|---|---|---|---|---|`,
tableRows,
``,
].join('\n');
}
// Collapsible full JSON
body += [
`<details><summary>Full JSON details</summary>`,
``,
'```json',
JSON.stringify(results, null, 2).substring(0, 60000),
'```',
``,
`</details>`,
].join('\n');
} else {
// Eval failed — post error with stderr excerpt
body = [
`<!-- showcase-eval-result -->`,
`### Showcase Eval Results`,
``,
`| | |`,
`|---|---|`,
`| **Verdict** | :x: **EVAL FAILED** (exit code: ${exitCode}) |`,
`| **Level** | \`${level}\` |`,
`| **Scope** | ${scope} |`,
`| **Run** | [View workflow](${runUrl}) |`,
``,
].join('\n');
if (stderrExcerpt) {
body += [
`<details><summary>Error output (last 50 lines)</summary>`,
``,
'```',
stderrExcerpt.substring(0, 30000),
'```',
``,
`</details>`,
``,
].join('\n');
}
// If we got partial JSON, include it
if (resultJson && resultJson !== '{}') {
body += [
`<details><summary>Partial JSON output</summary>`,
``,
'```json',
resultJson.substring(0, 30000),
'```',
``,
`</details>`,
].join('\n');
}
}
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
body,
});
- name: Generate devops-bot token
id: bot-token
if: needs.dispatch-gate.outputs.check_run_id != ''
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: 1108748
private-key: ${{ secrets.DEVOPS_BOT_PRIVATE_KEY }}
permission-checks: write
- name: Update Check Run with results
if: needs.dispatch-gate.outputs.check_run_id != ''
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
github-token: ${{ steps.bot-token.outputs.token }}
script: |
const checkRunId = Number(process.env.CHECK_RUN_ID);
const resultJson = process.env.RESULT_JSON || '{}';
const evalStatus = '${{ needs.eval.result }}';
let conclusion = 'failure';
let title = 'Showcase Eval — error';
let summary = 'The evaluation encountered an error.';
try {
const results = JSON.parse(resultJson);
const s = results.summary || {};
if (evalStatus === 'success' && s.fail === 0) {
conclusion = 'success';
title = `${s.pass}/${s.total} passed (${(s.duration_ms / 1000).toFixed(1)}s)`;
} else if (s.total === 0) {
conclusion = 'neutral';
title = 'No showcase integrations affected';
} else {
conclusion = 'failure';
title = `${s.fail} failed, ${s.pass} passed`;
}
const lines = ['## Eval Results\n'];
lines.push('| Integration | Status |');
lines.push('|-------------|--------|');
if (results.results) {
for (const [slug, tests] of Object.entries(results.results)) {
const statuses = Object.values(tests);
const pass = statuses.filter(t => t.status === 'pass').length;
const total = statuses.length;
const icon = pass === total ? '✅' : '❌';
lines.push(`| ${slug} | ${icon} ${pass}/${total} |`);
}
}
lines.push(`\n**Total:** ${s.pass} passed, ${s.fail} failed, ${s.skip} skipped (${(s.duration_ms / 1000).toFixed(1)}s)`);
summary = lines.join('\n');
} catch (e) {
summary = `Parse error: ${e.message}`;
}
await github.rest.checks.update({
owner: context.repo.owner,
repo: context.repo.repo,
check_run_id: checkRunId,
status: 'completed',
conclusion,
output: { title, summary },
actions: [{
label: 'Re-run Eval',
description: 'Run D5 evaluation',
identifier: 'run-eval',
}],
});
env:
CHECK_RUN_ID: ${{ needs.dispatch-gate.outputs.check_run_id }}
RESULT_JSON: ${{ needs.eval.outputs.result_json }}
+108
View File
@@ -0,0 +1,108 @@
name: "Showcase: Eval Check"
on:
pull_request:
types: [opened, synchronize, reopened]
permissions:
contents: read
jobs:
create-check:
if: false
runs-on: ubuntu-latest
timeout-minutes: 2
permissions:
contents: read
pull-requests: write
issues: write
steps:
- name: Generate devops-bot token
id: bot-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: 1108748
private-key: ${{ secrets.DEVOPS_BOT_PRIVATE_KEY }}
permission-checks: write
permission-issues: write
permission-pull-requests: write
- name: Create Check Run
id: check-run
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
github-token: ${{ steps.bot-token.outputs.token }}
script: |
const result = await github.rest.checks.create({
owner: context.repo.owner,
repo: context.repo.repo,
name: "Showcase Eval",
head_sha: context.payload.pull_request.head.sha,
status: "completed",
conclusion: "neutral",
external_id: String(context.payload.pull_request.number),
output: {
title: "Showcase Eval — click to run",
summary: "Evaluates this PR against the showcase integration matrix.\n\n_For targeted runs, comment `/eval d5 slug1,slug2` on the PR._",
},
actions: [
{
label: "Run Showcase Eval",
description: "Run D5 evaluation",
identifier: "run-eval",
},
],
});
core.setOutput('check_run_id', result.data.id);
- name: Post or update bot comment with trigger link
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
github-token: ${{ steps.bot-token.outputs.token }}
script: |
const crypto = require('crypto');
const pr = String(context.payload.pull_request.number);
const checkRunId = String(${{ steps.check-run.outputs.check_run_id }});
const secret = process.env.WEBHOOK_SECRET;
const sig = crypto
.createHmac('sha256', secret)
.update(`${pr}:${checkRunId}`)
.digest('hex');
const baseUrl = 'https://hooks.showcase.copilotkit.ai';
const triggerUrl = `${baseUrl}/trigger/eval?pr=${pr}&check_run_id=${checkRunId}&sig=${sig}`;
const marker = '<!-- showcase-eval-button -->';
const body = `${marker}\n\n` +
`### 🔬 Showcase Eval\n\n` +
`Evaluate this PR against the showcase integration matrix.\n\n` +
`[**▶ Run Evaluation**](${triggerUrl})\n\n` +
`_For targeted runs: \`/eval d5 slug1,slug2\`_`;
// Find existing bot comment
const comments = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.pull_request.number,
per_page: 100,
});
const existing = comments.data.find(c => c.body?.includes(marker));
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.pull_request.number,
body,
});
}
env:
WEBHOOK_SECRET: ${{ secrets.EVAL_WEBHOOK_SECRET }}
+57
View File
@@ -0,0 +1,57 @@
# INTERIM — superseded by showcase-harness smoke cadence (see Notion).
#
# Goal: curl /api/health on each showcase service every 5 min so the agent's
# in-memory state (JIT, module cache, connection pools) stays warm. Railway Pro
# tier doesn't sleep containers, but warm-state decays over idle.
#
# This workflow is intentionally minimal — no Slack, no metrics, no state.
# This is strictly keep-alive; alerting lives with the validation/deploy
# workflows.
name: "Showcase: Keep-Alive"
on:
schedule:
- cron: "*/5 * * * *"
workflow_dispatch: {}
permissions:
contents: read
jobs:
ping:
permissions:
contents: read
name: Ping ${{ matrix.slug }}
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
timeout-minutes: 2
continue-on-error: true
strategy:
fail-fast: false
matrix:
slug:
- ag2
- agno
- claude-sdk-python
- claude-sdk-typescript
- crewai-crews
- google-adk
- langgraph-fastapi
- langgraph-python
- langgraph-typescript
- langroid
- llamaindex
- mastra
- ms-agent-dotnet
- ms-agent-harness-dotnet
- ms-agent-python
- pydantic-ai
- spring-ai
- strands
steps:
- name: Ping /api/health
run: |
curl -fsS --max-time 15 \
"https://showcase-${{ matrix.slug }}-production.up.railway.app/api/health" \
> /dev/null
+173
View File
@@ -0,0 +1,173 @@
name: "Showcase: lint-prod (digest pinning)"
# Runs `bin/railway lint-prod` on every PR that touches showcase/.
#
# Currently ADVISORY: the `--exit-zero` flag makes the step exit 0 even when
# findings exist, so this workflow will not block PRs while we soak. Findings
# still print to the step log so we can monitor drift. Once we have confidence
# the findings are clean, remove `--exit-zero` to flip this to enforcing.
#
# Long-term contract: production must always be reproducible from a snapshot
# (every service pinned to `ghcr.io/...@sha256:...`).
#
# Visibility surfaces:
# - $GITHUB_STEP_SUMMARY: structured markdown table rendered at the top of
# the workflow run page (every run, push or pull_request).
# - Sticky PR comment: a single comment per PR, found+updated via an HTML
# marker (<!-- lint-prod-sticky-comment -->). Only posted on
# pull_request events; push events on main only write the step summary.
on:
pull_request:
paths:
- "showcase/**"
- ".github/workflows/showcase_lint_prod.yml"
workflow_dispatch:
concurrency:
group: showcase-lint-prod-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
pull-requests: write
jobs:
lint-prod:
name: Lint production pinning (advisory)
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Set up Ruby
uses: ruby/setup-ruby@d45b1a4e94b71acab930e56e79c6aa188764e7f9 # v1.316.0
with:
ruby-version: "3.2"
- name: Lint production pinning (advisory)
id: lint
env:
RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}
run: |
set -uo pipefail
if [ -z "${RAILWAY_TOKEN:-}" ]; then
echo "::warning::RAILWAY_TOKEN secret not configured; skipping lint-prod."
echo "skipped=true" >> "$GITHUB_OUTPUT"
echo "findings=0" >> "$GITHUB_OUTPUT"
echo "errored=false" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "skipped=false" >> "$GITHUB_OUTPUT"
# Advisory mode: --exit-zero so findings don't block the PR while we soak.
# Flip to enforcing by removing --exit-zero once findings are clean.
# Also emit machine-readable JSON for the visibility renderer.
#
# We capture stderr + exit code so a snapshot/GraphQL error doesn't
# abort the workflow before we can render a "audit unavailable" block.
# The intent of advisory mode is "never block a PR on this check".
rc=0
ruby showcase/bin/railway lint-prod --exit-zero --format json \
> lint-prod.json 2> lint-prod.err || rc=$?
# Mirror to the job log for humans (best-effort; ignore errors).
ruby showcase/bin/railway lint-prod --exit-zero || true
if [ "${rc}" -ne 0 ] || [ ! -s lint-prod.json ]; then
echo "::warning::lint-prod failed (rc=${rc}); rendering audit-unavailable block."
echo "--- lint-prod stderr ---"
cat lint-prod.err || true
echo "errored=true" >> "$GITHUB_OUTPUT"
echo "findings=0" >> "$GITHUB_OUTPUT"
# Synthesize an empty payload so the renderer has something to chew on.
ruby -rjson -rtime -e '
err = File.exist?("lint-prod.err") ? File.read("lint-prod.err").strip : ""
puts JSON.generate({"services" => [], "findings" => 0,
"timestamp" => Time.now.utc.iso8601,
"error" => err})
' > lint-prod.json
else
echo "errored=false" >> "$GITHUB_OUTPUT"
findings=$(ruby -rjson -e 'puts JSON.parse(File.read("lint-prod.json"))["findings"]')
echo "findings=${findings}" >> "$GITHUB_OUTPUT"
fi
- name: Render audit summary
if: always() && steps.lint.outputs.skipped != 'true'
id: render
run: |
set -uo pipefail
# Render in America/Los_Angeles so the timestamp respects DST.
export TZ="America/Los_Angeles"
ruby <<'RUBY' > audit.md
require "json"
require "time"
data = JSON.parse(File.read("lint-prod.json"))
services = data["services"] || []
findings = (data["findings"] || 0).to_i
ts_utc = Time.parse(data["timestamp"] || Time.now.utc.iso8601)
ts_pt = ts_utc.getlocal
total = services.size
err = data["error"].to_s
out = +""
out << "## Production Digest-Pinning Audit\n\n"
if !err.empty?
# Audit failed to run (e.g. snapshot/GraphQL error). Render a fallback
# block instead of leaving the surface blank.
out << "Audit could not run this round.\n\n"
out << "<details><summary>Error</summary>\n\n```\n#{err}\n```\n\n</details>\n\n"
elsif findings.zero?
out << "All #{total} services digest-pinned.\n\n"
else
out << "#{findings} of #{total} service(s) not digest-pinned.\n\n"
out << "| Service | Source.image | Status |\n"
out << "|---|---|---|\n"
services.each do |s|
next if s["status"] == "pinned"
src = s["source"].to_s.empty? ? "_(unset)_" : "`#{s['source']}`"
out << "| #{s['name']} | #{src} | mutable-tag |\n"
end
out << "\n"
end
out << "_Run #{ts_pt.strftime('%Y-%m-%d %H:%M:%S %Z')} — #{findings} finding(s)._\n"
puts out
RUBY
# Write to step summary (top of run page).
cat audit.md >> "$GITHUB_STEP_SUMMARY"
# Save body (with sticky marker) for the comment step.
{
echo "<!-- lint-prod-sticky-comment -->"
cat audit.md
} > comment.md
- name: Post/update sticky PR comment
if: always() && github.event_name == 'pull_request' && steps.lint.outputs.skipped != 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: |
set -uo pipefail
MARKER="<!-- lint-prod-sticky-comment -->"
# Find existing sticky comment (returns id only).
existing_id=$(gh api \
"/repos/${REPO}/issues/${PR_NUMBER}/comments?per_page=100" \
--jq ".[] | select(.body | contains(\"${MARKER}\")) | .id" \
| head -1)
# Build a JSON body file so we can PATCH/POST a multiline body safely.
ruby -rjson -e 'puts JSON.generate({body: File.read("comment.md")})' \
> comment.json
if [ -n "${existing_id:-}" ]; then
echo "Updating existing sticky comment id=${existing_id}"
gh api -X PATCH \
"/repos/${REPO}/issues/comments/${existing_id}" \
--input comment.json > /dev/null
else
echo "Creating new sticky comment"
gh pr comment "${PR_NUMBER}" --repo "${REPO}" --body-file comment.md
fi
- name: Run bin/railway tests
run: |
ruby showcase/bin/spec/all_tests.rb
+634
View File
@@ -0,0 +1,634 @@
name: "Showcase: Promote (staging → prod)"
# Promotes the staging-tested digest of one or more services to prod.
# Workflow_dispatch only. Humans trigger. No automatic prod promotes.
#
# Order:
# 0. resolve-targets → expand the workflow_dispatch `service`
# input (SSOT key, dispatch_name, or
# 'all') into the canonical services_csv
# consumed by every downstream job.
# 1. verify-staging-precondition → live-probe staging for the target service(s).
# Refuse on red (matches bin/railway
# --require-staging-green default).
# 2. promote → bin/railway promote <service>; runs the
# spec §7 hardening (P1..P6).
# 3. verify-prod → verify-deploy.ts --env prod for the
# target service(s). Feature-level probes,
# not naked 200. Then the prod equivalence
# re-sweep gate (§6.2) and the pinned-ness
# gate (lint-prod, §8.2 — every prod service
# must be on an immutable @sha256: digest).
# 4. notify → Slack #oss-alerts on any red. Never #engr.
# success → #team-showcase.
on:
workflow_dispatch:
inputs:
service:
description: "Service to promote (dispatch_name or SSOT key). 'all' = whole fleet. Leave the placeholder to abort."
required: true
type: choice
# >>> BEGIN GENERATED service options (showcase/scripts/sync-promote-service-options.ts) — DO NOT EDIT
default: __select_a_service__
options:
- __select_a_service__
- all
- ag2
- agno
- built-in-agent
- claude-sdk-python
- claude-sdk-typescript
- crewai-crews
- google-adk
- langgraph-fastapi
- langgraph-python
- langgraph-typescript
- langroid
- llamaindex
- mastra
- ms-agent-dotnet
- ms-agent-harness-dotnet
- ms-agent-python
- pydantic-ai
- shell
- shell-dashboard
- shell-docs
- shell-dojo
- showcase-aimock
- showcase-harness
- showcase-pocketbase
- spring-ai
- starter-adk
- starter-agno
- starter-crewai-crews
- starter-langgraph-fastapi
- starter-langgraph-js
- starter-langgraph-python
- starter-llamaindex
- starter-mastra
- starter-ms-agent-framework-dotnet
- starter-ms-agent-framework-python
- starter-pydantic-ai
- starter-strands-python
- strands
- strands-typescript
- webhooks
# <<< END GENERATED service options
digest:
description: "Optional digest override (default: snapshot from staging)"
required: false
type: string
# Serialize ALL promotes on a single input-agnostic group so concurrent runs
# (e.g. `all` + a single-service promote) can't pin the same Railway service
# at once; `cancel-in-progress: false` ensures an in-flight prod promote is
# never cancelled by a newer run queued behind it.
concurrency:
group: showcase-promote
cancel-in-progress: false
permissions:
contents: read
jobs:
resolve-targets:
runs-on: ubuntu-latest
timeout-minutes: 3
permissions:
contents: read
outputs:
# Leaf-set CSV — the BACKWARD-COMPAT promote target the promote job still
# drives off (Phase 1 behavior is unchanged).
services_csv: ${{ steps.resolve.outputs.services_csv }}
# Tier-ordered promote closure (requested transitive runtimeDeps
# Tier-1 verification, §4.2). Phase 1 SURFACES it (operators see the
# closure in the step summary) but the actual promote still uses
# services_csv; U4 enforces tier ordering / dependent-gating off these.
closure_csv: ${{ steps.resolve.outputs.closure_csv }}
# Machine-readable `tier:name` form of the closure for U4 to consume.
closure_plan: ${{ steps.resolve.outputs.closure_plan }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
- name: Generate SSOT artifact
working-directory: showcase/scripts
# The emitted JSON is EPHEMERAL here: resolve-promote-targets.sh reads it
# with jq to resolve which service(s) to promote and it is NEVER
# committed, so oxfmt-canonical formatting is irrelevant. This job's
# `npm ci` runs in showcase/scripts only and does NOT install the
# repo-root oxfmt binary the committed-artifact path shells out to, so
# leave it canonical-free via EMIT_SKIP_OXFMT (else spawnSync ENOENT
# fails the step and blocks EVERY promote). The committed-artifact path
# (static_quality.yml) keeps oxfmt REQUIRED + fail-loud.
env:
EMIT_SKIP_OXFMT: "1"
run: |
npm ci
npx tsx emit-railway-envs-json.ts
- name: Resolve target service set
id: resolve
# Resolution + the tiered-closure derivation live in the bats-tested
# showcase/scripts/resolve-promote-targets.sh so they can't drift from
# their test (see __tests__/resolve-promote-targets.bats), mirroring how
# promote-fleet.sh / verify-prod-display.sh were extracted from this same
# workflow. The script preserves the existing guards (--digest+all
# reject, empty-`all` fail-loud, unknown/ambiguous/not-prod-eligible) and
# emits services_csv (leaf, backward-compat) + closure_csv/closure_plan
# (tiered closure) into $GITHUB_OUTPUT, plus the closure plan + skips into
# $GITHUB_STEP_SUMMARY.
env:
INPUT: ${{ inputs.service }}
DIGEST: ${{ inputs.digest }}
GENERATED: showcase/scripts/railway-envs.generated.json
run: showcase/scripts/resolve-promote-targets.sh
verify-staging-precondition:
needs: [resolve-targets]
runs-on: ubuntu-latest
timeout-minutes: 15
environment: railway
permissions:
contents: read
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
- working-directory: showcase/scripts
run: npm ci
- name: Live-probe staging for promote precondition
working-directory: showcase/scripts
env:
RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}
SERVICES_CSV: ${{ needs.resolve-targets.outputs.services_csv }}
run: |
if [ -z "$RAILWAY_TOKEN" ]; then
echo "::error::RAILWAY_TOKEN is not set"
exit 1
fi
# Spec §7.2 P3: the live staging probe at promote time is
# authoritative. CI verify history is a leading indicator only.
# Run from showcase/scripts (where `npm ci` installed tsx) so npx
# uses the local install instead of network-fetching it.
#
# --skip-ineligible: SERVICES_CSV is the PROMOTE target set, which
# for `service=all` legitimately includes services that are
# promotable but NOT staging-probe-eligible (the starter-* fleet
# carries probe.staging=false in the SSOT). Those are an expected
# state here, not a fault — verify-deploy SKIPS them with a clear
# status line and probes only the eligible services, rather than
# hard-crashing the whole precondition on the first starter-*. This
# flag does NOT relax green for eligible services, nor does it
# tolerate an unknown (non-SSOT) name — a typo is still a hard error.
npx tsx verify-deploy.ts --env staging --services "$SERVICES_CSV" --skip-ineligible
promote:
needs: [resolve-targets, verify-staging-precondition]
# Do NOT hard-depend on verify-staging-precondition's RESULT. That job
# probes the FULL requested set all-or-nothing, so a single staging-red
# service (e.g. a chronically-broken integration in `all`) fails it and
# — under the default `if: success()` — would SKIP promote entirely,
# re-blocking the whole fleet. bin/railway enforces staging-green
# PER-SERVICE (spec §7 P2 = latest staging deploy must be SUCCESS, P3 =
# live staging probe, default-on), so a red service is refused on its own
# and lands in promote-fleet.sh's failed set (reds the run) without
# taking the greens down with it. verify-staging-precondition therefore
# stays as an advisory early signal surfaced in the notify payload, not a
# hard blocker. `!cancelled()` keeps a human-cancelled run from promoting;
# mirrors the verify-prod gate idiom below.
if: ${{ !cancelled() && needs.resolve-targets.result == 'success' }}
runs-on: ubuntu-latest
# promote-fleet.sh now fans out promotes WITHIN a tier (PROMOTE_FANOUT, cap
# 5) instead of running the whole fleet serially. Even so, the largest tier
# (~30 integration leaves) at cap 5 with bin/railway's ~300s/service
# verify_serving_digest! is ~6 batches * ~5 min ≈ 30 min, so 20 min cancelled
# a `service=all` promote mid-fleet. 35 min leaves headroom above the
# worst-case tier without masking a genuinely stuck promote.
timeout-minutes: 35
environment: railway
permissions:
contents: read
packages: read
outputs:
# CSV of the services that ACTUALLY promoted (best-effort: a partial
# failure still exposes the succeeded subset). verify-prod scopes its
# prod verification to exactly this set so a single failed service can't
# red the verification of the services that did promote.
succeeded_csv: ${{ steps.promote.outputs.succeeded_csv }}
# Aggregated staging-drift payload: non-empty when any promoted service's
# staging RUNNING digest differed from the current GHCR :latest (i.e.
# staging was NOT serving :latest). Folded into the Slack notify payload
# so operators see "what shipped to prod is not current :latest".
staging_drift: ${{ steps.promote.outputs.staging_drift }}
# Base64-encoded results JSON (schema_version=1) carrying BOTH the
# succeeded[] and failed[] sets, consumed by the three-variant Slack
# renderer (showcase_promote_notify.yml). promote-fleet.sh is the SSOT
# for the result set; the notify job enriches this blob with run-context
# (run_id, trigger, operator, elapsed, pre_staging) before dispatch.
# Empty when the promote step did not run (the notify job synthesizes a
# total-failure blob in that case).
results_b64: ${{ steps.promote.outputs.results_b64 }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- uses: ruby/setup-ruby@d45b1a4e94b71acab930e56e79c6aa188764e7f9 # v1.316.0
with:
ruby-version: "3.3"
bundler-cache: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
- working-directory: showcase/scripts
# Same EPHEMERAL regeneration as resolve-targets: bin/railway reads this
# JSON to derive EXPECTED_DOMAINS and it is never committed, so skip the
# oxfmt-canonical pass (repo-root oxfmt is not installed by this job's
# showcase/scripts-scoped `npm ci`; an ENOENT here would abort promote).
env:
EMIT_SKIP_OXFMT: "1"
run: |
npm ci
npx tsx emit-railway-envs-json.ts
- name: bin/railway promote
id: promote
env:
RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}
GHCR_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Tier-ordered closure plan (U4): promote-fleet.sh prefers CLOSURE_PLAN
# over SERVICES_CSV, promoting BY TIER (0->1->2) with dependent-tier
# gating AND within-tier parallel fan-out (PROMOTE_FANOUT). A serial
# `service=all` fleet overran the job timeout; tiered fan-out cuts the
# wall-clock so the whole fleet completes inside timeout-minutes.
CLOSURE_PLAN: ${{ needs.resolve-targets.outputs.closure_plan }}
# Retained as the backward-compat fallback: promote-fleet.sh uses this
# ONLY when CLOSURE_PLAN is empty (e.g. an older resolve step).
SERVICES_CSV: ${{ needs.resolve-targets.outputs.services_csv }}
DIGEST: ${{ inputs.digest }}
run: |
set -euo pipefail
if [ -z "$RAILWAY_TOKEN" ]; then
echo "::error::RAILWAY_TOKEN is not set"
exit 1
fi
# promote-fleet.sh runs each service in turn BEST-EFFORT: a single
# red service (e.g. a chronically-broken integration in the `all`
# set) must not abort promotion of the rest of the fleet. The script
# attempts every service, aggregates the succeeded/failed sets, emits
# a step summary, and exits non-zero iff ANY service failed (so the
# notify job still fires) — but only AFTER attempting all of them.
# bin/railway itself handles spec §7 preconditions (P1..P6);
# --require-staging-green is default-on and the prior job already
# established staging is green (defense in depth). We deliberately do
# NOT pass --confirm-divergence: WARN-divergence refusals are a real
# signal that must fail the run.
RAILWAY_BIN="showcase/bin/railway" \
showcase/scripts/promote-fleet.sh
verify-prod:
needs: [resolve-targets, promote]
# Run whenever promote actually RAN — success OR partial failure — so the
# services that DID promote still get prod verification. `!cancelled()`
# excludes a human-cancelled run; `needs.promote.result != 'skipped'`
# excludes the case where promote never ran (e.g. an upstream abort/skip).
# Under the default `if: success()` this job was SKIPPED on any partial
# promote failure, leaving the promoted services with zero prod
# verification — that is the bug this gate fixes.
if: ${{ !cancelled() && needs.promote.result != 'skipped' }}
runs-on: ubuntu-latest
timeout-minutes: 20
environment: railway
permissions:
contents: read
outputs:
# Distinguishes a job that actually PROBED prod (`success`) from one that
# SKIPPED probing because nothing promoted (`skipped`). The GitHub job
# `result` is `success` in BOTH cases (the skip path exits 0), so the
# notify job must read THIS output — not `needs.verify-prod.result` — to
# avoid reporting a misleading `verify-prod=success` when prod was never
# touched. A real probe failure / contract violation exits non-zero, so
# the job `result` becomes `failure` and this output is never written
# (notify falls back to the job result for that case).
status: ${{ steps.verify.outputs.status }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
- working-directory: showcase/scripts
run: npm ci
- name: Run verify-deploy --env prod
id: verify
working-directory: showcase/scripts
env:
RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}
# Scope verification to the services that ACTUALLY promoted (from the
# promote job's best-effort succeeded set), NOT the full requested set
# (resolve-targets) — verifying a service that failed to promote would
# guarantee a red verify and mask the health of the ones that did
# promote.
SERVICES_CSV: ${{ needs.promote.outputs.succeeded_csv }}
# The promote job's result, so the empty-CSV branch can tell a genuine
# all-failed run (empty succeeded set is expected) apart from a
# contract violation (promote reported success yet emitted no CSV).
PROMOTE_RESULT: ${{ needs.promote.result }}
run: |
set -euo pipefail
if [ -z "$RAILWAY_TOKEN" ]; then
echo "::error::RAILWAY_TOKEN is not set"
exit 1
fi
# If NOTHING promoted (every requested service failed, so the
# succeeded set is empty), there is nothing to verify — skip with a
# clear log line rather than calling verify-deploy.ts with an empty
# --services (which would either error or vacuously pass). The promote
# job already exited non-zero in that case, so the overall run is red
# via the notify state machine regardless.
#
# BUT: an empty succeeded set is only legitimate when promote did NOT
# succeed. If promote reported success and STILL emitted no CSV, the
# "promote already failed" assumption that justifies the vacuous skip
# is violated — fail loud instead of silently exiting 0.
if [ -z "$SERVICES_CSV" ]; then
if [ "$PROMOTE_RESULT" = "success" ]; then
echo "::error::promote reported success but succeeded_csv is empty — contract violation"
exit 1
fi
echo "::notice::succeeded_csv is empty (no services promoted, or promote did not run); skipping prod verification. The run is red via the promote job result if anything failed."
# Record that prod was SKIPPED, not verified. The job still exits 0
# (its `result` is `success`), so the notify job reads this `status`
# output to render `verify-prod=skipped` instead of a misleading
# `verify-prod=success`.
echo "status=skipped" >> "$GITHUB_OUTPUT"
exit 0
fi
# Run from showcase/scripts (where `npm ci` installed tsx) so npx uses
# the local install instead of network-fetching it. A non-zero exit
# here fails the step (job `result` = failure) and `status` is never
# written, so notify falls back to the job result.
npx tsx verify-deploy.ts --env prod --services "$SERVICES_CSV"
# Prod was actually probed and passed.
echo "status=success" >> "$GITHUB_OUTPUT"
# ── Prod equivalence gate (UNIT U10, spec §6.2) ──────────────────────
# After the closure pins (promote job) AND the per-service prod probe
# (above), re-sweep the promoted integration closure on the PROD control
# plane (prod harness scheduler + prod harness-workers, or scheduler
# INLINE fallback when workers are unprovisioned — §4.4) and run U9's
# equivalence gate over the FRESH prod rows vs current staging. Promote
# success flips to the equivalence definition (fail only on
# staging-green / prod-not-green, gray/stale excluded, one-directional).
#
# GATED ON CONFIGURATION: the equivalence gate needs prod + staging
# PocketBase creds and the prod Railway environment id. Those are an
# out-of-PR operational prerequisite (§8.1) — until the secrets are wired
# the step ANNOTATES "not configured" and is a no-op, so this job's
# existing per-service prod verification (above) remains the gate. Once
# configured, a gate FAILURE fails the step (and the run).
# Install the pnpm workspace ONLY when the equivalence gate is configured
# (the gate step's enqueue dynamically imports the harness producer graph,
# which lives in the pnpm workspace — `npm ci` in showcase/scripts alone
# does not resolve it). `--ignore-scripts` skips postinstall/Playwright
# browser downloads: the host only ENQUEUES + POLLS prod PocketBase; the
# prod harness-workers own the browser. Gated on the same config check as
# the gate step so the no-op (unconfigured) path stays cheap.
- name: Set up pnpm (equivalence gate only)
if: ${{ steps.verify.outputs.status == 'success' && vars.SHOWCASE_PROD_POCKETBASE_URL != '' && vars.SHOWCASE_STAGING_POCKETBASE_URL != '' && vars.SHOWCASE_RAILWAY_ENV_ID_PROD != '' }}
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: pnpm install (equivalence gate only)
if: ${{ steps.verify.outputs.status == 'success' && vars.SHOWCASE_PROD_POCKETBASE_URL != '' && vars.SHOWCASE_STAGING_POCKETBASE_URL != '' && vars.SHOWCASE_RAILWAY_ENV_ID_PROD != '' }}
run: pnpm install --ignore-scripts
- name: Prod equivalence re-sweep gate
if: ${{ steps.verify.outputs.status == 'success' }}
working-directory: showcase/scripts
env:
# The promoted closure subset that ACTUALLY pinned (best-effort) —
# the set to re-sweep + compare. Same scope as the prod probe above.
PROMOTED_CLOSURE_CSV: ${{ needs.promote.outputs.succeeded_csv }}
RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}
RAILWAY_PROJECT_ID: ${{ vars.SHOWCASE_RAILWAY_PROJECT_ID }}
RAILWAY_ENVIRONMENT_ID_PROD: ${{ vars.SHOWCASE_RAILWAY_ENV_ID_PROD }}
PROD_POCKETBASE_URL: ${{ vars.SHOWCASE_PROD_POCKETBASE_URL }}
PROD_POCKETBASE_SUPERUSER_EMAIL: ${{ secrets.SHOWCASE_PROD_POCKETBASE_SUPERUSER_EMAIL }}
PROD_POCKETBASE_SUPERUSER_PASSWORD: ${{ secrets.SHOWCASE_PROD_POCKETBASE_SUPERUSER_PASSWORD }}
STAGING_POCKETBASE_URL: ${{ vars.SHOWCASE_STAGING_POCKETBASE_URL }}
STAGING_POCKETBASE_SUPERUSER_EMAIL: ${{ secrets.SHOWCASE_STAGING_POCKETBASE_SUPERUSER_EMAIL }}
STAGING_POCKETBASE_SUPERUSER_PASSWORD: ${{ secrets.SHOWCASE_STAGING_POCKETBASE_SUPERUSER_PASSWORD }}
# §4.4 annotation: flip to "false" once prod harness-workers are
# provisioned-but-degraded is the default-true full-throughput path.
PROD_HARNESS_WORKERS_PROVISIONED: ${{ vars.SHOWCASE_PROD_HARNESS_WORKERS_PROVISIONED }}
run: |
set -euo pipefail
# OUT-OF-PR PREREQ GATE: the equivalence gate is inert until the prod
# + staging PocketBase + prod Railway env-id config is wired (§8.1).
# Absence is NOT a failure — the per-service prod probe above already
# gated the run; this step just annotates that the equivalence gate
# was skipped for lack of configuration.
if [ -z "${PROD_POCKETBASE_URL:-}" ] || [ -z "${STAGING_POCKETBASE_URL:-}" ] || [ -z "${RAILWAY_ENVIRONMENT_ID_PROD:-}" ]; then
echo "::notice::Prod equivalence gate not configured (prod/staging PocketBase + prod Railway env-id absent) — skipping the re-sweep gate. This is the §8.1 out-of-PR prerequisite; the per-service prod probe remains the gate."
exit 0
fi
if [ -z "${PROMOTED_CLOSURE_CSV:-}" ]; then
echo "::notice::no promoted closure to re-sweep — equivalence gate vacuously passes."
exit 0
fi
# A gate FAILURE (a genuine staging-green / prod-not-green regression)
# or a re-sweep timeout (REFUSE) exits non-zero → fails this step and
# the run, exactly like the per-service probe above. `pnpm exec` (not
# `npx`) so the dynamically-imported harness producer graph resolves
# through the workspace node_modules the install step above created.
pnpm exec tsx verify-prod-resweep.ts
# ── Prod pinned-ness gate (UNIT U12, spec §8.2) ──────────────────────
# COMPLEMENTS the equivalence gate above — it asserts PINNED-NESS, not
# equivalence. After the promote pins the closure and prod is verified
# healthy + equivalent, assert that EVERY prod service is on an immutable
# `@sha256:` digest, not a mutable `:latest` tag. A born-on-:latest prod
# service (deploy-to-railway.ts provisions an unpinned source.image, spec
# R-E) can be healthy AND equivalent while still floating on a mutable tag
# — a latent rollback/repro hazard the other gates do not catch. The gate
# logic lives in the bats-tested showcase/scripts/lint-prod-gate.sh (a thin
# `bin/railway lint-prod` wrapper) so it can't drift from its test (see
# __tests__/lint-prod-gate.bats). Runs whenever prod was actually probed
# (same `status == 'success'` guard as the equivalence gate); a non-zero
# exit (an unpinned prod service, or a hard lint-prod error) fails the step
# and the run. No `--exit-zero`: an unpinned prod service must red the run.
- name: Prod pinned-ness gate (lint-prod)
if: ${{ steps.verify.outputs.status == 'success' }}
env:
RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}
run: |
set -euo pipefail
if [ -z "$RAILWAY_TOKEN" ]; then
echo "::error::RAILWAY_TOKEN is not set"
exit 1
fi
showcase/scripts/lint-prod-gate.sh
notify:
# Single Slack message per run via the three-variant aggregated renderer
# (.github/workflows/showcase_promote_notify.yml): success ✅ / partial ⚠️
# (per-service Failed bullets, cross-posts #oss-alerts) / total ❌. This job
# REPLACES the old inline two-state notify (success/failure-only, which
# dumped the full requested CSV and mislabeled a partial promote as a blanket
# "Failed [all 39]"). It builds the results JSON (from the promote job's
# results_b64, enriched with this run's context) and DISPATCHES the renderer.
#
# Best-effort promote invariant (do NOT reintroduce a PRE gate):
# resolve-targets = HARD precondition (must succeed).
# verify-staging-precondition = ADVISORY only — surfaced in the Slack
# payload (`pre_staging`), never gates
# promote or run success.
# promote = best-effort (exits non-zero iff a
# service failed).
# verify-prod = verifies the succeeded subset.
needs: [resolve-targets, verify-staging-precondition, promote, verify-prod]
if: always()
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
# The job's own GITHUB_TOKEN cannot start NEW workflow runs (Actions'
# recursion-prevention drops workflow_dispatch events authenticated with
# GITHUB_TOKEN), so the renderer dispatch goes through the devops-bot App
# token minted below — mirroring canary.yml's cross-workflow dispatch.
contents: read
# actions:read lets the payload step query this run's own metadata
# (created_at) to compute real wall-clock elapsed for the Slack message.
actions: read
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Build notify payload
id: payload
env:
INPUT: ${{ inputs.service }}
# promote-fleet's results JSON (schema_version=1, succeeded[]+failed[]).
# Empty when the promote step did not run (upstream abort/skip/cancel).
RESULTS_B64: ${{ needs.promote.outputs.results_b64 }}
RESOLVE: ${{ needs.resolve-targets.result }}
PRE: ${{ needs.verify-staging-precondition.result }}
PROMOTE: ${{ needs.promote.result }}
PROD: ${{ needs.verify-prod.result }}
# Operator identity for the renderer's `operator_mention`. The renderer
# prefers a Slack users.lookupByEmail on operator_email; we have no
# reliable email for github.actor, so pass the actor as the git-name
# fallback (the renderer degrades to it when the email lookup is empty).
ACTOR: ${{ github.actor }}
# gh api (run metadata lookup for real wall-clock elapsed) needs a token.
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
# SKIP cases that must NOT post a Slack message (parity with the old
# neutral-state arms): a deliberate no-pick abort, or a human cancel.
if [ "$INPUT" = "__select_a_service__" ]; then
echo "::notice::no service selected (deliberate no-op abort); skipping Slack notify."
echo "dispatch=0" >> "$GITHUB_OUTPUT"
exit 0
fi
if [ "$RESOLVE" = "cancelled" ] || [ "$PRE" = "cancelled" ] || [ "$PROMOTE" = "cancelled" ] || [ "$PROD" = "cancelled" ]; then
echo "::notice::run cancelled by a human; skipping Slack notify (no red page)."
echo "dispatch=0" >> "$GITHUB_OUTPUT"
exit 0
fi
# Map the ADVISORY verify-staging-precondition result onto the
# renderer's pre_staging glyph vocabulary (green/amber/red/skipped).
case "$PRE" in
success) PRE_STAGING="green" ;;
failure) PRE_STAGING="amber" ;; # advisory red → amber (not a gate)
*) PRE_STAGING="skipped" ;;
esac
# 6-char lowercase hex run_id — the renderer's HARD CONTRACT
# (^[0-9a-f]{6}$; see its run-name directive). Derive it from this
# run's own id so it is stable + greppable, never random.
RUN_ID=$(printf '%06x' "$(( GITHUB_RUN_ID % 16777216 ))")
# Base results JSON: prefer promote-fleet's emitted blob (the SSOT for
# succeeded[]/failed[]). When promote never ran (no results_b64), the
# run aborted upstream — synthesize a total-failure blob so the
# renderer posts the ❌ variant instead of nothing.
if [ -n "${RESULTS_B64:-}" ]; then
if ! printf '%s' "$RESULTS_B64" | base64 -d > /tmp/results-base.json 2>/dev/null; then
echo "::error::failed to decode promote results_b64"
exit 1
fi
else
jq -nc '{schema_version:1, abort_reason:"fleet-preflight", succeeded:[], failed:[]}' > /tmp/results-base.json
fi
# Real wall-clock elapsed: this notify job runs last (needs: [...],
# if: always()), so (now - run.created_at) is the run's end-to-end
# duration. Query this run's own metadata via gh api (actions:read).
# Fall back to 0 only if the lookup fails or yields a non-sane value,
# in which case the renderer omits the "in {elapsed}" phrasing.
ELAPSED=0
created_at=$(gh api "repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" \
--jq '.created_at' 2>/dev/null || echo "")
if [ -n "$created_at" ]; then
start_epoch=$(date -u -d "$created_at" +%s 2>/dev/null || echo "")
if [ -n "$start_epoch" ]; then
now_epoch=$(date -u +%s)
delta=$(( now_epoch - start_epoch ))
# Guard against clock skew / parse glitches producing a negative.
if [ "$delta" -ge 0 ]; then
ELAPSED="$delta"
fi
fi
fi
# Enrich the base blob with this run's context (run_id, trigger,
# operator, elapsed, pre_staging). trigger=workflow (the renderer maps
# it to the `showcase_promote.yml` label). elapsed_seconds is the real
# wall-clock computed above; the renderer formats it as "Nm SSs".
ENRICHED_B64=$(jq -c \
--arg run_id "$RUN_ID" \
--arg trigger "workflow" \
--arg actor "$ACTOR" \
--arg pre "$PRE_STAGING" \
--argjson elapsed "$ELAPSED" \
'. + {run_id:$run_id, trigger:$trigger, operator_git_name:$actor, elapsed_seconds:$elapsed, pre_staging:$pre}' \
/tmp/results-base.json | base64 | tr -d '\n')
{
echo "dispatch=1"
echo "run_id=$RUN_ID"
echo "results_b64=$ENRICHED_B64"
} >> "$GITHUB_OUTPUT"
- name: Mint devops-bot token
id: app-token
if: steps.payload.outputs.dispatch == '1'
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: 1108748
private-key: ${{ secrets.DEVOPS_BOT_PRIVATE_KEY }}
# actions=write is the ONLY scope needed: dispatch the sibling renderer
# workflow. The default GITHUB_TOKEN cannot start new workflow runs.
permission-actions: write
- name: Dispatch showcase_promote_notify.yml
if: steps.payload.outputs.dispatch == '1'
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
RESULTS_B64: ${{ steps.payload.outputs.results_b64 }}
RUN_ID: ${{ steps.payload.outputs.run_id }}
run: |
set -euo pipefail
# Dispatch the three-variant renderer. It owns ALL Slack posting
# (#team-showcase init + thread, #oss-alerts cross-post on
# partial/total). Pass trigger=workflow + the 6-hex run_id (which is
# also the renderer's run-name for the CLI's gh-run-list polling
# contract). Dispatched on the default branch (no --ref) so the
# renderer's reviewed main-branch definition runs.
gh workflow run showcase_promote_notify.yml \
--repo "$GITHUB_REPOSITORY" \
-f results="$RESULTS_B64" \
-f trigger=workflow \
-f run_id="$RUN_ID"
echo "dispatched showcase_promote_notify.yml (run_id=$RUN_ID)"
+310
View File
@@ -0,0 +1,310 @@
#!/usr/bin/env bash
# showcase_promote_notify.dry-run.sh
#
# Mirrors the decode + render logic in `showcase_promote_notify.yml` so
# the workflow can be exercised without invoking Slack.
#
# Usage:
# showcase_promote_notify.dry-run.sh <base64-results-json>
# showcase_promote_notify.dry-run.sh --file <path-to-json>
#
# Output: prints, for each Slack call the workflow WOULD make, a block of
# the form:
# --- chat.postMessage ---
# channel: <channel>
# text: |
# <text>
#
# Exits 0 on success, 1 on decode/argv errors, 2 on schema_version mismatch
# (notify workflow aborts gracefully — we treat that as a non-fatal but
# distinct exit so callers can assert on it).
# ---------- alert post-and-verify predicate (shared with the workflow) ----------
# Slack returns HTTP 200 with `{"ok":false,"error":"..."}` on LOGICAL failures
# (channel_not_found, not_in_channel, ...). A failure-ALERT that is silently
# dropped pages nobody, so the live workflow MUST surface it. This predicate is
# the testable core of that surfacing logic: it inspects a captured Slack
# response and, when the post did NOT succeed, emits a GitHub `::warning::`
# (matching the workflow's existing `::warning::`/`>&2` idiom) and returns 1.
#
# Sourcing this script (e.g. from bats) defines this function without running
# the dry-run body — see the EXECUTION GUARD just below the function.
# $1 = label for the warning (e.g. "thread reply", "#oss-alerts cross-post")
# $2 = captured Slack API response body (JSON, or "{}" on transport failure)
slack_alert_posted_ok() {
local label="$1"
local resp="$2"
local ok
ok=$(printf '%s' "$resp" | jq -r '.ok // false' 2>/dev/null || echo false)
if [ "$ok" != "true" ]; then
local err
err=$(printf '%s' "$resp" | jq -r '.error // "unknown"' 2>/dev/null || echo unknown)
echo "::warning::Slack ${label} did NOT post (ok=${ok} error=${err}); failure alert may have been dropped" >&2
return 1
fi
return 0
}
# EXECUTION GUARD: define functions only when sourced. `return` outside a
# function is legal only in a sourced script (it errors when executed), so the
# subshell `(return 0 2>/dev/null)` succeeds iff we are being sourced — in which
# case we `return 0` here and skip the dry-run body below. When executed
# directly the subshell fails and execution falls through to `set -euo`.
(return 0 2>/dev/null) && return 0
set -euo pipefail
if [ "${1:-}" = "" ]; then
echo "usage: $0 <base64-results-json> | --file <path>" >&2
exit 1
fi
if [ "$1" = "--file" ]; then
if [ -z "${2:-}" ]; then
echo "usage: $0 --file <path>" >&2
exit 1
fi
if [ ! -f "$2" ]; then
echo "error: file not found: $2" >&2
exit 1
fi
cp "$2" /tmp/dry-run-results.json
else
if ! printf '%s' "$1" | base64 -d > /tmp/dry-run-results.json 2>/tmp/dry-run-results.err; then
echo "error: base64 decode failed: $(cat /tmp/dry-run-results.err)" >&2
exit 1
fi
fi
if ! jq -e . /tmp/dry-run-results.json >/dev/null 2>&1; then
echo "error: decoded payload is not valid JSON" >&2
exit 1
fi
R=/tmp/dry-run-results.json
schema_version=$(jq -r '.schema_version // empty' "$R")
if [ "$schema_version" != "1" ]; then
echo "warn: schema_version mismatch — expected 1, got '${schema_version}'; would abort Slack post"
exit 2
fi
run_id=$(jq -r '.run_id' "$R")
# Enforce the run_id contract — required for the CLI's gh run list polling.
# See HARD CONTRACT comment + run-name directive in showcase_promote_notify.yml.
# A malformed run_id is a dispatcher-contract violation, not a recoverable
# runtime condition, so hard-fail (exit 1) rather than warn-and-abort.
if ! printf '%s' "$run_id" | grep -Eq '^[0-9a-f]{6}$'; then
echo "error: run_id '$run_id' does not match ^[0-9a-f]{6}$ (breaks CLI polling contract; see run-name)" >&2
exit 1
fi
trigger=$(jq -r '.trigger' "$R")
operator_email=$(jq -r '.operator_email // ""' "$R")
operator_git_name=$(jq -r '.operator_git_name // ""' "$R")
# Coerce to an integer up front: elapsed_seconds may arrive as a float (e.g.
# 5.2) OR as a JSON STRING (e.g. "5.2"). `floor` on a string raises jq error 5
# ("number required") which, under `set -euo pipefail`, aborts the whole render
# step so NO Slack message posts. `tonumber?` parses numeric strings and
# swallows non-numeric input (-> 0); `floor` then yields the integer Bash
# `[ -gt ]`/`$(( ))` need.
elapsed=$(jq -r '(.elapsed_seconds // 0) | tonumber? // 0 | floor' "$R")
pre_staging=$(jq -r '.pre_staging // "skipped"' "$R")
abort_reason=$(jq -r '.abort_reason // ""' "$R")
succeeded_count=$(jq -r '.succeeded | length' "$R")
jq '.failed | sort_by(.service)' "$R" > /tmp/dry-run-failed-sorted.json
jq '[.[] | select(.category != "truncation-suffix")]' /tmp/dry-run-failed-sorted.json > /tmp/dry-run-failed-render.json
truncation_more=$(jq -r '[.[] | select(.category == "truncation-suffix") | .service] | .[0] // ""' /tmp/dry-run-failed-sorted.json)
# Counts:
# total_count = succeeded_count + failed_real_count
# succeeded_count = raw .succeeded length
# failed_real_count = .failed length minus truncation-suffix sentinels
# (rendered to operators on all display lines)
failed_real_count=$(jq 'length' /tmp/dry-run-failed-render.json)
total_count=$((succeeded_count + failed_real_count))
# Comma-separated list of the SUCCEEDED service names, for the ✅ success
# thread reply AND the ⚠️ partial reply's `Promoted:` line. The runtime blob
# emits .succeeded[] as {service} objects (see promote-fleet.sh); tolerate bare
# strings too (hand-written fixtures use them).
succeeded_csv=$(jq -r '[.succeeded[] | if type == "object" then .service else . end] | join(", ")' "$R")
# Names of every ATTEMPTED service (succeeded + real failures), for the init
# post. For `service=all` this is the drifted subset resolve-targets selected;
# for a scoped/single-service dispatch it is exactly what was requested. Either
# way it is the set we ATTEMPTED — we do not claim the rest was already current.
# Sorted for a stable, legible list; the truncation-suffix sentinel is excluded
# via /tmp/dry-run-failed-render.json.
attempted_csv=$(jq -rs '
(.[0] | [.succeeded[] | if type == "object" then .service else . end])
+ (.[1] | [.[].service])
| sort | join(", ")
' "$R" /tmp/dry-run-failed-render.json)
# GitHub Actions run URL — used by the success message's inline "View run" link.
# In CI GITHUB_REPOSITORY/GITHUB_RUN_ID are set; in a bare dry-run they may not
# be, so fall back to a stable placeholder so the rendered shape still matches.
gha_url="https://github.com/${GITHUB_REPOSITORY:-CopilotKit/CopilotKit}/actions/runs/${GITHUB_RUN_ID:-<run_id>}"
# elapsed is the real wall-clock seconds the dispatcher measured
# (showcase_promote.yml computes now - run.created_at). When it is a positive
# value we render " in Nm SSs"; when it is 0 (dispatcher could not measure it,
# or a hand-dispatch passed nothing) we OMIT the phrase entirely rather than
# print a meaningless "in 0m 00s".
fmt_elapsed() {
local total="$1"
local m=$((total / 60))
local s=$((total % 60))
printf '%dm %02ds' "$m" "$s"
}
if [ "$elapsed" -gt 0 ] 2>/dev/null; then
elapsed_phrase=" in $(fmt_elapsed "$elapsed")"
else
elapsed_phrase=""
fi
# operator mention: dry-run simulates a successful Slack lookup; falls back to git name then "unknown" if no email present.
if [ -n "$operator_email" ]; then
operator_mention="<lookupByEmail:${operator_email}>"
elif [ -n "$operator_git_name" ]; then
operator_mention="$operator_git_name"
else
operator_mention="unknown"
fi
case "$pre_staging" in
green) pre_staging_line="pre_staging: ✓ green" ;;
amber) pre_staging_line="pre_staging: ⚠ amber" ;;
red) pre_staging_line="pre_staging: ✗ red" ;;
skipped) pre_staging_line="pre_staging: — skipped" ;;
*) pre_staging_line="pre_staging: ${pre_staging}" ;;
esac
if [ "$trigger" = "cli" ]; then
# shellcheck disable=SC2016
trigger_label='`bin/railway --notify`'
else
# shellcheck disable=SC2016
trigger_label='`showcase_promote.yml`'
fi
# Name the services being promoted this run. We name only what was ATTEMPTED
# — accurate whether the dispatch was `service=all` (the drifted subset) or a
# single service. We do NOT claim the rest of the fleet was "already current":
# for a scoped/single-service dispatch that is false (it conflates "attempted"
# with "drifted"). Fall back to a bare count when the attempted set is empty
# (e.g. a fleet-preflight abort that touched zero services).
if [ -n "$attempted_csv" ]; then
init_headline="🚂 *Promoting showcase → prod* (${total_count}): ${attempted_csv}"
else
init_headline="🚂 *Promoting showcase → prod* (${total_count})"
fi
init_text="${init_headline}
operator ${operator_mention} · trigger ${trigger_label} · run \`${run_id}\`
${pre_staging_line}"
fail_bullets=$(jq -r '.[] | "• `\(.service)` — exit \(.exit) (\(.category))"' /tmp/dry-run-failed-render.json)
if [ -n "$truncation_more" ]; then
if [ -n "$fail_bullets" ]; then
fail_bullets="${fail_bullets}
${truncation_more}"
else
fail_bullets="${truncation_more}"
fi
fi
# Branching uses failed_real_count (excludes truncation sentinel) so a
# sentinel-only failed[] does not get mis-classified as partial and
# spuriously cross-posted to #oss-alerts.
#
# An abort_reason combined with zero successes is ALWAYS a total abort,
# regardless of failed_real_count — fleet-preflight refusals abort the
# whole run BEFORE any service is attempted (succeeded=[], failed=[] or
# sentinel-only). Without this guard, the failed_real_count==0 branch
# would fire first and mis-announce the run as a clean success.
if [ -n "$abort_reason" ] && [ "$succeeded_count" -eq 0 ]; then
outcome="total"
case "$abort_reason" in
fleet-preflight) reason_line="*Reason:* fleet-wide preflight refused" ;;
per-service) reason_line="*Reason:* all services individually refused" ;;
*) reason_line="*Reason:* aborted" ;;
esac
if [ "$failed_real_count" -eq 0 ]; then
# Fleet-preflight abort with zero services touched: no bullets to
# render, so omit the *Failed:* heading entirely.
thread_text="❌ *Aborted${elapsed_phrase}* — 0 ✓ · 0 ✗
${pre_staging_line}
${reason_line}"
else
thread_text="❌ *Aborted${elapsed_phrase}* — 0 ✓ · ${failed_real_count}
${pre_staging_line}
${reason_line}
*Failed:*
${fail_bullets}"
fi
elif [ "$failed_real_count" -eq 0 ]; then
outcome="success"
thread_text="✅ *Showcase Promoted to Prod* — ${succeeded_count} ✓ · <${gha_url}|View run>
Services: ${succeeded_csv}"
elif [ "$succeeded_count" -gt 0 ] && [ "$failed_real_count" -gt 0 ]; then
outcome="partial"
thread_text="⚠️ *Done${elapsed_phrase}* — ${succeeded_count} ✓ · ${failed_real_count}
*Promoted:* ${succeeded_csv}
*Failed:*
${fail_bullets}"
else
# succeeded_count == 0 && failed_real_count > 0 — per-service refusals
# without an abort_reason set (defensive fallback).
outcome="total"
case "$abort_reason" in
fleet-preflight) reason_line="*Reason:* fleet-wide preflight refused" ;;
per-service) reason_line="*Reason:* all services individually refused" ;;
*) reason_line="*Reason:* aborted" ;;
esac
thread_text="❌ *Aborted${elapsed_phrase}* — 0 ✓ · ${failed_real_count}
${pre_staging_line}
${reason_line}
*Failed:*
${fail_bullets}"
fi
emit() {
echo "--- chat.postMessage ---"
echo "channel: $1"
echo "text: |"
printf '%s\n' "$2" | sed 's/^/ /'
echo
}
emit "#team-showcase" "$init_text"
emit "#team-showcase (thread_ts=<init_ts>)" "$thread_text"
# Mirror the workflow's post-and-verify exit semantics so the dry-run exercises
# the SAME fail-loud/warn-only distinction the live .yml does (see the matching
# slack_alert_posted_ok calls there). No real Slack call happens here, so we
# feed each predicate a simulated response: a successful post by default (the
# dry-run convention — see the operator-mention block above), overridable via
# DRY_RUN_THREAD_RESP / DRY_RUN_OSS_RESP so a test can inject a 200/ok:false
# drop and assert on the exit code.
sim_ok='{"ok":true,"ts":"<sim>"}'
# Thread reply: informational, in the promote channel — warn-only (|| true),
# mirroring the .yml. A dropped summary post must not red the job.
slack_alert_posted_ok "thread reply" "${DRY_RUN_THREAD_RESP:-$sim_ok}" || true
if [ "$outcome" != "success" ]; then
case "$outcome" in
partial) oss_text="⚠️ showcase promote: ${succeeded_count} ✓ · ${failed_real_count} ✗ — thread: <permalink>" ;;
total) oss_text="❌ showcase promote aborted: 0 ✓ · ${failed_real_count} ✗ — thread: <permalink>" ;;
esac
emit "#oss-alerts" "$oss_text"
# Page-the-humans alert: FAIL LOUD, mirroring the .yml. No `|| true` — a
# 200/ok:false drop here means nobody is told the promote failed, so the
# predicate's non-zero return must abort (set -e) and red the run.
slack_alert_posted_ok "#oss-alerts cross-post" "${DRY_RUN_OSS_RESP:-$sim_ok}"
fi
echo "outcome=${outcome} run_id=${run_id}"
@@ -0,0 +1,433 @@
name: "Showcase: Promote Notify"
# HARD CONTRACT (spec N2): the CLI polls `gh run list` for a run whose
# display_title matches `promote-<run_id>`. Without this `run-name`
# directive, that polling lookup will always time out.
run-name: promote-${{ inputs.run_id }}
on:
workflow_dispatch:
inputs:
results:
description: "Base64-encoded results JSON (schema_version=1)"
required: true
type: string
trigger:
description: "Dispatch source"
required: true
type: choice
options:
- cli
- workflow
run_id:
description: "6-char lowercase hex run id"
required: true
type: string
permissions:
contents: read
jobs:
notify:
name: Post aggregated Slack notification
runs-on: ubuntu-latest
timeout-minutes: 5
env:
RESULTS_B64: ${{ inputs.results }}
TRIGGER: ${{ inputs.trigger }}
RUN_ID: ${{ inputs.run_id }}
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
TEAM_SHOWCASE_CHANNEL: "#team-showcase"
OSS_ALERTS_CHANNEL: "#oss-alerts"
steps:
- name: Decode and validate results
id: decode
run: |
set -euo pipefail
if ! command -v jq >/dev/null 2>&1; then
echo "::error::jq is required"
exit 1
fi
# Decode the base64 results blob into a JSON file. The blob may
# contain newlines from `base64` line-wrapping; -d handles that.
if ! printf '%s' "$RESULTS_B64" | base64 -d > /tmp/results.json 2>/tmp/results.err; then
echo "::error::base64 decode failed: $(cat /tmp/results.err)"
exit 1
fi
if ! jq -e . /tmp/results.json >/dev/null 2>&1; then
echo "::error::decoded payload is not valid JSON"
exit 1
fi
schema_version=$(jq -r '.schema_version // empty' /tmp/results.json)
if [ "$schema_version" != "1" ]; then
echo "::warning::schema_version mismatch — expected 1, got '${schema_version}'; aborting Slack post gracefully"
echo "abort=1" >> "$GITHUB_OUTPUT"
exit 0
fi
# Enforce the run_id contract — required for the CLI's gh run list polling.
# See HARD CONTRACT comment + run-name directive at top of file. A malformed
# run_id is a dispatcher-contract violation, not a recoverable runtime
# condition, so hard-fail (exit 1) rather than warn-and-abort.
if ! printf '%s' "$RUN_ID" | grep -Eq '^[0-9a-f]{6}$'; then
echo "::error::run_id '$RUN_ID' does not match ^[0-9a-f]{6}$ (breaks CLI polling contract; see run-name)"
exit 1
fi
echo "abort=0" >> "$GITHUB_OUTPUT"
- name: Render Slack messages and post
if: steps.decode.outputs.abort == '0'
env:
# Re-export for the script step
RESULTS_PATH: /tmp/results.json
run: |
set -euo pipefail
if [ -z "${SLACK_BOT_TOKEN:-}" ]; then
echo "::error::SLACK_BOT_TOKEN is not set"
exit 1
fi
# ---------- helpers ----------
slack_api() {
# $1 = method, $2 = JSON body
local method="$1"
local body="$2"
local resp http
resp=$(mktemp)
http=$(curl -sS -o "$resp" -w '%{http_code}' \
-X POST "https://slack.com/api/${method}" \
-H "Authorization: Bearer ${SLACK_BOT_TOKEN}" \
-H "Content-type: application/json; charset=utf-8" \
--data "$body" || echo "000")
if [ "$http" != "200" ]; then
echo "::warning::Slack ${method} non-2xx http=${http} body=$(head -c 500 "$resp")" >&2
echo "{}"
rm -f "$resp"
return 0
fi
cat "$resp"
rm -f "$resp"
}
# Slack returns HTTP 200 with `{"ok":false,"error":"..."}` on LOGICAL
# failures (channel_not_found, not_in_channel, ...). slack_api only
# surfaces non-2xx HTTP, so a failure-ALERT that posts 200/ok:false
# would otherwise be silently dropped — pages nobody. This predicate
# checks the captured response and emits a `::warning::` (mirroring the
# slack_api non-2xx idiom above) when the post did NOT succeed.
# $1 = label, $2 = captured Slack response body
slack_alert_posted_ok() {
local label="$1"
local resp="$2"
local ok
ok=$(printf '%s' "$resp" | jq -r '.ok // false' 2>/dev/null || echo false)
if [ "$ok" != "true" ]; then
local err
err=$(printf '%s' "$resp" | jq -r '.error // "unknown"' 2>/dev/null || echo unknown)
echo "::warning::Slack ${label} did NOT post (ok=${ok} error=${err}); failure alert may have been dropped" >&2
return 1
fi
return 0
}
# ---------- read payload ----------
R="$RESULTS_PATH"
run_id=$(jq -r '.run_id' "$R")
# Validate the BLOB's run_id, not just the RUN_ID input. The decode
# step (separate step, no shared shell vars) checks the input; the
# CLI/hand-dispatch path renders THIS value into the run-name and the
# Slack messages, so it must satisfy the same ^[0-9a-f]{6}$ contract.
# Mirrors showcase_promote_notify.dry-run.sh. Hard-fail (exit 1) — a
# malformed run_id is a dispatcher-contract violation, not recoverable.
if ! printf '%s' "$run_id" | grep -Eq '^[0-9a-f]{6}$'; then
echo "::error::run_id '$run_id' does not match ^[0-9a-f]{6}$ (breaks CLI polling contract; see run-name)"
exit 1
fi
trigger=$(jq -r '.trigger' "$R")
operator_email=$(jq -r '.operator_email // ""' "$R")
operator_git_name=$(jq -r '.operator_git_name // ""' "$R")
# Coerce to an integer up front: elapsed_seconds may arrive as a float
# (e.g. 5.2) OR as a JSON STRING (e.g. "5.2"). `floor` on a string
# raises jq error 5 ("number required") which, under `set -euo
# pipefail`, aborts the whole render step so NO Slack message posts.
# `tonumber?` parses numeric strings and swallows non-numeric input
# (-> 0); `floor` then yields the integer Bash `[ -gt ]`/`$(( ))` need.
elapsed=$(jq -r '(.elapsed_seconds // 0) | tonumber? // 0 | floor' "$R")
pre_staging=$(jq -r '.pre_staging // "skipped"' "$R")
abort_reason=$(jq -r '.abort_reason // ""' "$R")
succeeded_count=$(jq -r '.succeeded | length' "$R")
# Comma-separated list of the SUCCEEDED service names, for the ✅
# success thread reply AND the ⚠️ partial reply's `Promoted:` line.
# The runtime blob emits .succeeded[] as {service} objects (see
# promote-fleet.sh); tolerate bare strings too.
succeeded_csv=$(jq -r '[.succeeded[] | if type == "object" then .service else . end] | join(", ")' "$R")
# GitHub Actions run URL — used by the success message's inline
# "View run" link AND by the cross-post branch's permalink fallback.
# Computed once here so both render the SAME url.
gha_url="https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
# Sort failed alphabetically by service and split off the
# truncation-suffix sentinel (if any) so it renders as a
# trailing "+ K more" line instead of a bullet.
jq '.failed | sort_by(.service)' "$R" > /tmp/failed-sorted.json
jq '[.[] | select(.category != "truncation-suffix")]' /tmp/failed-sorted.json > /tmp/failed-render.json
truncation_more=$(jq -r '[.[] | select(.category == "truncation-suffix") | .service] | .[0] // ""' /tmp/failed-sorted.json)
# Counts:
# total_count = succeeded_count + failed_real_count
# succeeded_count = raw .succeeded length
# failed_real_count = .failed length minus truncation-suffix sentinels
# (rendered to operators on all display lines)
failed_real_count=$(jq 'length' /tmp/failed-render.json)
# Service total = succeeded + real failures (truncation entries
# are not real services).
total_count=$((succeeded_count + failed_real_count))
# Names of every ATTEMPTED service (succeeded + real failures), for
# the init post. For `service=all` this is the drifted subset
# resolve-targets selected; for a scoped/single-service dispatch it is
# exactly what was requested. Either way it is the set we ATTEMPTED —
# we do not claim the rest was already current. Sorted for a stable,
# legible list; the truncation-suffix sentinel (not a real service) is
# excluded via /tmp/failed-render.json.
attempted_csv=$(jq -rs '
(.[0] | [.succeeded[] | if type == "object" then .service else . end])
+ (.[1] | [.[].service])
| sort | join(", ")
' "$R" /tmp/failed-render.json)
# ---------- format elapsed seconds ----------
# elapsed is the real wall-clock seconds the dispatcher measured
# (showcase_promote.yml computes now - run.created_at). When it is a
# positive value we render " in Nm SSs"; when it is 0 (dispatcher
# could not measure it, or a hand-dispatch passed nothing) we OMIT the
# phrase entirely rather than print a meaningless "in 0m 00s".
fmt_elapsed() {
local total="$1"
local m=$((total / 60))
local s=$((total % 60))
printf '%dm %02ds' "$m" "$s"
}
if [ "$elapsed" -gt 0 ] 2>/dev/null; then
elapsed_phrase=" in $(fmt_elapsed "$elapsed")"
else
elapsed_phrase=""
fi
# ---------- resolve operator mention ----------
operator_mention=""
if [ -n "$operator_email" ]; then
# users.lookupByEmail requires GET (Slack Web API).
lookup_resp=$(curl -sS \
-G "https://slack.com/api/users.lookupByEmail" \
-H "Authorization: Bearer ${SLACK_BOT_TOKEN}" \
--data-urlencode "email=${operator_email}" || echo '{}')
ok=$(echo "$lookup_resp" | jq -r '.ok // false')
if [ "$ok" = "true" ]; then
uid=$(echo "$lookup_resp" | jq -r '.user.id // ""')
if [ -n "$uid" ]; then
operator_mention="<@${uid}>"
fi
fi
fi
if [ -z "$operator_mention" ]; then
if [ -n "$operator_git_name" ]; then
operator_mention="$operator_git_name"
elif [ -n "$operator_email" ]; then
operator_mention="$operator_email"
else
operator_mention="unknown"
fi
fi
# ---------- pre_staging glyph ----------
case "$pre_staging" in
green) pre_staging_line="pre_staging: ✓ green" ;;
amber) pre_staging_line="pre_staging: ⚠ amber" ;;
red) pre_staging_line="pre_staging: ✗ red" ;;
skipped) pre_staging_line="pre_staging: — skipped" ;;
*) pre_staging_line="pre_staging: ${pre_staging}" ;;
esac
# ---------- trigger label ----------
if [ "$trigger" = "cli" ]; then
trigger_label='`bin/railway --notify`'
else
trigger_label='`showcase_promote.yml`'
fi
# ---------- initiation post ----------
# Name the services being promoted this run. We name only what was
# ATTEMPTED — accurate whether the dispatch was `service=all` (the
# drifted subset) or a single service. We do NOT claim the rest of
# the fleet was "already current": for a scoped/single-service
# dispatch that is false (it conflates "attempted" with "drifted").
# Fall back to a bare count when the attempted set is empty (e.g. a
# fleet-preflight abort that touched zero services).
if [ -n "$attempted_csv" ]; then
init_headline="🚂 *Promoting showcase → prod* (${total_count}): ${attempted_csv}"
else
init_headline="🚂 *Promoting showcase → prod* (${total_count})"
fi
init_text="${init_headline}
operator ${operator_mention} · trigger ${trigger_label} · run \`${run_id}\`
${pre_staging_line}"
# Strip leading whitespace introduced by the heredoc-style indent above.
init_text=$(printf '%s\n' "$init_text" | sed 's/^[[:space:]]\{10\}//')
init_body=$(jq -nc \
--arg channel "$TEAM_SHOWCASE_CHANNEL" \
--arg text "$init_text" \
'{channel:$channel, text:$text}')
init_resp=$(slack_api chat.postMessage "$init_body")
init_ok=$(echo "$init_resp" | jq -r '.ok // false')
init_ts=$(echo "$init_resp" | jq -r '.ts // ""')
init_channel_id=$(echo "$init_resp" | jq -r '.channel // ""')
if [ "$init_ok" != "true" ] || [ -z "$init_ts" ]; then
echo "::warning::initiation post failed; continuing without threading"
fi
# ---------- permalink (for #oss-alerts cross-post) ----------
permalink=""
if [ -n "$init_ts" ] && [ -n "$init_channel_id" ]; then
perm_resp=$(curl -sS \
-G "https://slack.com/api/chat.getPermalink" \
-H "Authorization: Bearer ${SLACK_BOT_TOKEN}" \
--data-urlencode "channel=${init_channel_id}" \
--data-urlencode "message_ts=${init_ts}" || echo '{}')
if [ "$(echo "$perm_resp" | jq -r '.ok // false')" = "true" ]; then
permalink=$(echo "$perm_resp" | jq -r '.permalink // ""')
fi
fi
# ---------- build failure bullets ----------
fail_bullets=$(jq -r '.[] | "• `\(.service)` — exit \(.exit) (\(.category))"' /tmp/failed-render.json)
if [ -n "$truncation_more" ]; then
if [ -n "$fail_bullets" ]; then
fail_bullets="${fail_bullets}
${truncation_more}"
else
fail_bullets="${truncation_more}"
fi
fi
# ---------- determine outcome & build thread reply ----------
# Branching uses failed_real_count (excludes truncation sentinel)
# so a sentinel-only failed[] does not get mis-classified as
# partial and spuriously cross-posted to #oss-alerts.
#
# An abort_reason combined with zero successes is ALWAYS a total
# abort, regardless of failed_real_count — fleet-preflight
# refusals abort the whole run BEFORE any service is attempted
# (succeeded=[], failed=[] or sentinel-only). Without this guard,
# the failed_real_count==0 branch would fire first and
# mis-announce the run as a clean success.
if [ -n "$abort_reason" ] && [ "$succeeded_count" -eq 0 ]; then
outcome="total"
case "$abort_reason" in
fleet-preflight) reason_line="*Reason:* fleet-wide preflight refused" ;;
per-service) reason_line="*Reason:* all services individually refused" ;;
*) reason_line="*Reason:* aborted" ;;
esac
if [ "$failed_real_count" -eq 0 ]; then
# Fleet-preflight abort with zero services touched: no
# bullets to render, so omit the *Failed:* heading entirely.
thread_text="❌ *Aborted${elapsed_phrase}* — 0 ✓ · 0 ✗
${pre_staging_line}
${reason_line}"
else
thread_text="❌ *Aborted${elapsed_phrase}* — 0 ✓ · ${failed_real_count} ✗
${pre_staging_line}
${reason_line}
*Failed:*
${fail_bullets}"
fi
elif [ "$failed_real_count" -eq 0 ]; then
outcome="success"
thread_text="✅ *Showcase Promoted to Prod* — ${succeeded_count} ✓ · <${gha_url}|View run>
Services: ${succeeded_csv}"
elif [ "$succeeded_count" -gt 0 ] && [ "$failed_real_count" -gt 0 ]; then
outcome="partial"
thread_text="⚠️ *Done${elapsed_phrase}* — ${succeeded_count} ✓ · ${failed_real_count} ✗
*Promoted:* ${succeeded_csv}
*Failed:*
${fail_bullets}"
else
# succeeded_count == 0 && failed_real_count > 0 — per-service
# refusals without an abort_reason set (defensive fallback).
outcome="total"
case "$abort_reason" in
fleet-preflight) reason_line="*Reason:* fleet-wide preflight refused" ;;
per-service) reason_line="*Reason:* all services individually refused" ;;
*) reason_line="*Reason:* aborted" ;;
esac
thread_text="❌ *Aborted${elapsed_phrase}* — 0 ✓ · ${failed_real_count} ✗
${pre_staging_line}
${reason_line}
*Failed:*
${fail_bullets}"
fi
# Strip the 10-space indent the heredoc-style strings carry.
thread_text=$(printf '%s\n' "$thread_text" | sed 's/^[[:space:]]\{10\}//')
# ---------- post thread reply ----------
if [ -n "$init_ts" ]; then
thread_body=$(jq -nc \
--arg channel "$TEAM_SHOWCASE_CHANNEL" \
--arg text "$thread_text" \
--arg ts "$init_ts" \
'{channel:$channel, text:$text, thread_ts:$ts}')
thread_resp=$(slack_api chat.postMessage "$thread_body")
else
# Initiation failed; post the thread text as a top-level
# message so the operator still gets the summary.
fallback_body=$(jq -nc \
--arg channel "$TEAM_SHOWCASE_CHANNEL" \
--arg text "$thread_text" \
'{channel:$channel, text:$text}')
thread_resp=$(slack_api chat.postMessage "$fallback_body")
fi
# Surface a dropped summary post (200/ok:false). `|| true` keeps the
# ::warning:: visible without aborting the cross-post below.
slack_alert_posted_ok "thread reply" "$thread_resp" || true
# ---------- cross-post to #oss-alerts on partial/total failure ----------
if [ "$outcome" != "success" ]; then
# Build the trailing link suffix once so both branches share it
# without relying on a trailing-space suffix-strip.
if [ -n "$permalink" ]; then
link_suffix="thread: ${permalink}"
else
# No Slack permalink available; link to the GitHub Actions run
# instead. gha_url is defined earlier in this render step (where
# the payload is read) so the success message and this fallback
# share the same URL.
link_suffix="thread permalink unavailable; see ${gha_url}"
fi
case "$outcome" in
partial) oss_text="⚠️ showcase promote: ${succeeded_count} ✓ · ${failed_real_count} ✗ — ${link_suffix}" ;;
total) oss_text="❌ showcase promote aborted: 0 ✓ · ${failed_real_count} ✗ — ${link_suffix}" ;;
esac
oss_body=$(jq -nc \
--arg channel "$OSS_ALERTS_CHANNEL" \
--arg text "$oss_text" \
'{channel:$channel, text:$text}')
oss_resp=$(slack_api chat.postMessage "$oss_body")
# This is the page-the-humans alert. A 200/ok:false drop here means
# nobody is told the promote failed, so a silently-green renderer job
# would hide the dropped page. FAIL LOUD: the ::warning:: alone is
# easy to miss, so let the predicate's non-zero return abort this
# step (set -e) → the job goes red and the drop is visible. Unlike
# the thread reply (informational, in the promote channel), this
# alert MUST not be swallowed — so there is no `|| true` here.
slack_alert_posted_ok "#oss-alerts cross-post" "$oss_resp"
fi
echo "notify completed: outcome=${outcome} run_id=${run_id}"
+90
View File
@@ -0,0 +1,90 @@
name: "Showcase: Reconcile prod vs staging (on-demand drift check)"
# On-demand prod-vs-staging reconciliation. The showcase deploy model: staging
# tracks a mutable `:latest` tag and is continuously rebuilt; prod is pinned to
# an immutable `@sha256:` digest and advances only on an explicit promote. So
# prod can sit BEHIND a green staging — which is OFTEN INTENTIONAL (changes are
# batched and promoted deliberately), so a recurring drift alert would be noise.
#
# This workflow is therefore MANUAL ONLY (workflow_dispatch). There is no
# schedule and no unsolicited Slack alert. A maintainer runs it from the Actions
# tab to answer "is prod caught up with staging right now?"; the per-service
# reconcile table is rendered into the GH step summary, and the run exits
# nonzero when a column is stale so the drift is visibly flagged on the run.
#
# Read-only: runs `bin/railway reconcile-prod`, which performs NO promotes /
# mutations.
on:
workflow_dispatch:
concurrency:
group: showcase-reconcile-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
reconcile:
name: Reconcile prod vs staging
runs-on: ubuntu-latest
timeout-minutes: 10
environment: railway
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Set up Ruby
uses: ruby/setup-ruby@d45b1a4e94b71acab930e56e79c6aa188764e7f9 # v1.316.0
with:
ruby-version: "3.2"
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
- name: Regenerate SSOT JSON
# bin/railway reads showcase/scripts/railway-envs.generated.json to
# derive the prod-eligible (probe.prod) set; it is never committed, so
# regenerate it here. Skip the oxfmt-canonical pass (repo-root oxfmt is
# not installed by this scripts-scoped `npm ci`), mirroring the promote
# workflow's resolve/promote jobs.
working-directory: showcase/scripts
env:
EMIT_SKIP_OXFMT: "1"
run: |
npm ci
npx tsx emit-railway-envs-json.ts
- name: Reconcile gate
id: gate
env:
RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}
GHCR_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Path the gate writes machine-readable JSON to (uploaded as an
# artifact below). Cheap to keep — handy for a maintainer inspecting a
# manual run, but no Slack/alert consumer.
RECONCILE_JSON: ${{ github.workspace }}/reconcile.json
run: |
set -uo pipefail
if [ -z "${RAILWAY_TOKEN:-}" ]; then
echo "::error::RAILWAY_TOKEN secret not configured; cannot run the reconcile gate."
exit 1
fi
# Run the gate. reconcile-prod-gate.sh surfaces the per-service table
# into the GH step summary and exits 1 on a stale service / 2 on a hard
# error — either reds the run so a manual run visibly flags drift.
RAILWAY_BIN="showcase/bin/railway" \
showcase/scripts/reconcile-prod-gate.sh
- name: Upload reconcile JSON
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: reconcile-json
path: reconcile.json
if-no-files-found: ignore
retention-days: 7
@@ -0,0 +1,57 @@
name: "Showcase: Runtime-Route Wiring (PR)"
# Pre-merge guard for the OSS-451 failure class: a demo page whose CopilotKit
# `runtimeUrl` points at an `/api/copilotkit-<demo>` route that does not exist,
# so the page 404s on load (runtime_info_fetch_failed) and never mounts.
#
# The existing Showcase Build Check (showcase_build_check.yml) is a Docker
# *build* — it compiles a page that references a non-existent route just fine,
# because `runtimeUrl` is an opaque string with no build-time link to the
# route's existence. That blind spot is exactly how OSS-451 shipped. This
# static check runs alongside the build check and asserts that every SHIPPED
# demo (a demo listed in its integration's manifest `features`) wires its
# `runtimeUrl` to a route that actually exists.
#
# Fast, no Docker, no secrets. Should be added to the branch-protection
# required checks so it blocks merge like the build check does.
on:
pull_request:
paths:
- "showcase/integrations/**"
- "showcase/scripts/validate-runtime-routes.ts"
- "showcase/scripts/validate-runtime-routes.baseline.json"
- "showcase/scripts/package.json"
- ".github/workflows/showcase_validate-wiring.yml"
concurrency:
group: showcase-validate-wiring-${{ github.event.pull_request.number }}
cancel-in-progress: true
permissions:
contents: read
jobs:
validate-runtime-routes:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Setup Node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22"
- name: Install validator deps
working-directory: showcase/scripts
run: npm ci --no-audit --no-fund --ignore-scripts
- name: Validate runtime-route wiring (OSS-451 guard)
working-directory: showcase/scripts
run: npm run validate-routes
+770
View File
@@ -0,0 +1,770 @@
name: "Showcase: Validate"
on:
pull_request:
paths:
- "showcase/**"
- "examples/integrations/**"
- "package.json"
- "pnpm-lock.yaml"
- "pnpm-workspace.yaml"
- ".github/workflows/showcase_validate.yml"
- ".github/workflows/showcase_deploy.yml"
push:
branches: [main]
paths:
- "showcase/**"
- "examples/integrations/**"
- "package.json"
- "pnpm-lock.yaml"
- "pnpm-workspace.yaml"
- ".github/workflows/showcase_validate.yml"
- ".github/workflows/showcase_deploy.yml"
# Least-privilege by default. Individual jobs/steps can widen when needed.
permissions:
contents: read
# Split concurrency per event so main-branch push runs are never canceled
# mid-execution (we need Slack failure alerts to fire reliably). PR runs
# still cancel in progress to keep PR CI responsive.
concurrency:
group: showcase-validate-${{ github.ref }}-${{ github.event_name }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
validate:
name: Validate Showcase
# Hoist the Slack webhook into an env var so step-level `if:`
# expressions can reference it — `secrets.*` is not a valid
# named-value inside `if:` and causes a workflow startup failure
# on push events.
env:
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_OSS_ALERTS }}
# Depot (Startup plan, unlimited minutes) for persistent pnpm/npm
# cache across runs — cold ubuntu-latest runs were ~18-20m; Depot
# typically reduces to ~5-8m. 25m timeout retained as headroom.
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 25
outputs:
inline_slack_notifier_reached: ${{ steps.inline_slack_marker.outputs.reached }}
permissions:
actions: read
contents: read
# id-token: write is required for Depot OIDC auth (runs-on: depot-ubuntu-*).
id-token: write
defaults:
run:
# Pin shell so `set -euo pipefail` + `mapfile` behave the same
# across any future runner image changes (default on ubuntu is
# already bash, but we lock it explicitly).
shell: bash
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22
# Cache npm for the showcase/shell `npm ci` step below (shell is
# NOT a pnpm workspace member; it ships its own package-lock.json).
cache: "npm"
cache-dependency-path: showcase/shell/package-lock.json
- name: Setup pnpm
# Pinned to a specific minor rather than floating @v4 so that a
# silent upstream major/minor change can't alter install semantics
# on a random CI run. Bump deliberately when refreshing the toolchain.
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Verify lockfile is up to date
run: pnpm install --frozen-lockfile --ignore-scripts
- name: Enforce e2e spec count (baseline per package)
run: |
set -euo pipefail
shopt -s nullglob
# Single source of truth: showcase/scripts/fail-baseline.json
# `baselineDemoCount` is read here AND by validate-parity.ts so the
# per-package e2e-spec-count floor cannot drift between CI and the
# validator. If parsing fails we distinguish JSON syntax errors
# from schema failures (missing/non-integer/negative field).
set +e
MIN=$(node -e "
let v;
try {
v = require('./showcase/scripts/fail-baseline.json');
} catch (e) {
console.error('fail-baseline.json: JSON syntax error: ' + e.message);
process.exit(2);
}
const n = v.baselineDemoCount;
if (typeof n !== 'number' || !Number.isInteger(n) || n < 0) {
console.error('fail-baseline.json: schema failure: baselineDemoCount must be a non-negative integer');
process.exit(3);
}
console.log(n);
")
rc=$?
set -e
if [ "$rc" -ne 0 ]; then
# Preserve node's distinct rc (2=JSON syntax, 3=schema) in the
# annotation so the CI log pinpoints the cause without re-running.
echo "::error::Failed to read baselineDemoCount from showcase/scripts/fail-baseline.json (node exit=$rc; 2=JSON syntax, 3=schema)"
exit "$rc"
fi
failed=0
found=0
for pkg_dir in showcase/integrations/*/; do
[ -d "$pkg_dir" ] || continue
pkg=$(basename "$pkg_dir")
# Skip manifest-only packages (no src/ directory) — these are
# virtual/meta integrations (e.g. built-in-agent) that carry no
# source code or demos and therefore have no e2e specs to enforce.
if [ ! -d "${pkg_dir}src" ]; then
echo "skip: $pkg (manifest-only, no src/)"
continue
fi
found=$((found + 1))
e2e_dir="${pkg_dir}tests/e2e/"
if [ ! -d "$e2e_dir" ]; then
echo "::error file=$pkg_dir::Package '$pkg' is missing tests/e2e/ directory (required for baseline e2e coverage)"
failed=1
continue
fi
# Capture `find` output into a variable first so we can check
# its exit status directly. Bash process substitution (used with
# `mapfile < <(cmd)`) does NOT propagate the producer's exit
# status to the parent shell — `mapfile` only reports its own
# usage errors — so a failing `find` (EACCES on a subdir, ELOOP,
# transient I/O) would have been silently treated as "zero
# specs" and surfaced as the misleading "minimum required"
# error instead of the real root cause. Command substitution
# propagates `find`'s status via `$?` on the assignment, which
# we check immediately. A zero-spec result is a legitimate
# success from `find` and is handled by the `$count -lt $MIN`
# check below, not treated as a find failure.
# Aggregate find failures with the rest of the per-package
# failure modes (missing tests/e2e/, below-MIN count) so one bad
# package doesn't short-circuit reporting for the others. A
# single CI run should surface every problematic package at
# once; `exit "$failed"` at the end of the loop reports the
# aggregate.
if ! find_out=$(find "$e2e_dir" -maxdepth 1 -type f -name '*.spec.ts'); then
echo "::error file=$e2e_dir::find failed while enumerating specs for '$pkg'"
failed=1
continue
fi
specs=()
# Only populate the array if `find` produced output; `mapfile
# <<< ""` would otherwise create a single empty element and
# inflate the count by one.
if [ -n "$find_out" ]; then
mapfile -t specs <<< "$find_out"
fi
count=${#specs[@]}
if [ "$count" -lt "$MIN" ]; then
echo "::error file=$e2e_dir::Package '$pkg' has $count e2e spec(s); minimum required is $MIN"
failed=1
else
echo "ok: $pkg has $count spec(s)"
fi
done
if [ "$found" -eq 0 ]; then
echo "::error::No showcase/integrations/*/ directories found — baseline check cannot run"
exit 1
fi
exit "$failed"
- name: Run validate-parity (MUST checks gating)
working-directory: showcase/scripts
# MUST failures (missing manifest, missing src/app/demos dir) exit 1 and
# fail the PR. SHOULD deviations print warnings and exit 0. See
# showcase/scripts/validate-parity.ts for the full policy.
#
# `pnpm exec` resolves tsx from the pnpm-lock.yaml-pinned workspace
# install; `npx tsx` could fetch a drifting version on a registry
# cache miss.
run: pnpm exec tsx validate-parity.ts
- name: Run validate-fixture-tool-surface (aimock drift)
working-directory: showcase/scripts
# Cross-references every aimock fixture's returned tool-call names
# against the tool surface of each demo whose suggestion prompt
# contains the fixture's match substring. Catches the class of
# drift that caused the 2026-04-22 regression where generic
# substring matches (e.g. "pie chart") cross-fired across demos
# with different tool surfaces, leaving the UI blank in prod.
# See showcase/scripts/validate-fixture-tool-surface.ts and the
# postmortem linked from there.
run: pnpm exec tsx validate-fixture-tool-surface.ts
- name: Run validate-pins (ratchet)
working-directory: showcase/scripts
# Ratchet gate on pin drift. Baseline (count + SHA-256 hash of sorted
# unique FAIL lines) lives in `showcase/scripts/fail-baseline.json`;
# see that file for the full ratchet semantics and adjustment
# procedure. Weekly backlog visibility is provided by
# `.github/workflows/showcase_drift-report.yml`. Driving the drift to
# zero (and flipping this advisory ratchet to fully enforcing) is
# future work.
run: |
set -euo pipefail
# --- Load + validate baseline -----------------------------------
# `node -e` prints either a validated value or an error marker
# we match below. We deliberately do NOT let require() throw
# out of the subshell; we format a clean CI error instead.
#
# We distinguish three failure modes with distinct exit codes so
# the CI log pinpoints the cause without requiring a re-run:
# exit 2 => JSON syntax error (require() threw)
# exit 3 => schema failure (missing/wrong-typed required field)
# exit 4 => unexpected/unknown top-level field (typo guard)
#
# The unexpected-field check rejects silent typos like
# `validatepinsfailcount` or an accidentally-added `comment`
# field (distinct from the allowed leading underscore
# `_comment`) that would otherwise leave required fields
# undefined and be caught only via the schema branch with a
# more confusing message.
set +e
baseline_json=$(node -e "
const ALLOWED = ['_comment', 'validatePinsFailCount', 'validatePinsFailHash', 'baselineDemoCount'];
let v;
try {
v = require('./fail-baseline.json');
} catch (e) {
console.error('fail-baseline.json: JSON syntax error: ' + e.message);
process.exit(2);
}
const unexpected = Object.keys(v).filter(k => !ALLOWED.includes(k));
if (unexpected.length > 0) {
console.error('fail-baseline.json: unexpected field(s): ' + unexpected.join(', ') + '. Allowed fields: ' + ALLOWED.join(', '));
process.exit(4);
}
const c = v.validatePinsFailCount;
const h = v.validatePinsFailHash;
if (typeof c !== 'number' || !Number.isInteger(c) || c < 0) {
console.error('fail-baseline.json: schema failure: validatePinsFailCount must be a non-negative integer');
process.exit(3);
}
if (typeof h !== 'string' || !/^[0-9a-f]{64}$/.test(h)) {
console.error('fail-baseline.json: schema failure: validatePinsFailHash must be a 64-char lowercase hex SHA-256');
process.exit(3);
}
console.log(JSON.stringify({ count: c, hash: h }));
")
rc=$?
set -e
if [ "$rc" -ne 0 ]; then
# Preserve node's distinct rc (2=JSON syntax, 3=schema, 4=unexpected field)
# in the annotation so the CI log pinpoints the cause.
echo "::error::fail-baseline.json failed validation (node exit=$rc; 2=JSON syntax, 3=schema, 4=unexpected field)"
exit "$rc"
fi
baseline=$(node -e "console.log(JSON.parse(process.argv[1]).count)" "$baseline_json")
baseline_hash=$(node -e "console.log(JSON.parse(process.argv[1]).hash)" "$baseline_json")
# --- Run validator; separate internal crash from pin-drift exit -
# validate-pins exits 0 when FAIL=0, 1 when FAIL>0. Anything else
# (2+, uncaught throw, node crash, SIGSEGV) is an internal failure
# we must surface distinctly from a legitimate drift report.
#
# We deliberately keep stdout and stderr in separate variables.
# validate-pins.ts emits progress/summary on stdout and `[FAIL]`
# lines on stderr; mingling them with `2>&1` allowed progress
# chatter (or future stdout additions) to corrupt the hash input.
# The hash is computed strictly from stderr.
set +e
stderr_file=$(mktemp)
stdout=$(pnpm exec tsx validate-pins.ts 2>"$stderr_file")
rc=$?
stderr=$(cat "$stderr_file")
rm -f "$stderr_file"
set -e
# Replay both streams to the job log so humans can debug.
printf '%s\n' "$stdout"
printf '%s\n' "$stderr" >&2
if [ "$rc" -ne 0 ] && [ "$rc" -ne 1 ]; then
# Preserve validate-pins.ts's distinct exit code (2=EXIT_INTERNAL,
# 3=EXIT_UNREADABLE, 4+=future) so downstream consumers can
# distinguish "validator crashed" from "pin drift found" (which
# would be rc=1). Collapsing to `exit 1` would make an internal
# crash indistinguishable from legitimate drift in the PR check
# signal.
echo "::error::validate-pins.ts exited with unexpected code $rc (expected 0 or 1). This indicates an internal failure, not pin drift."
exit "$rc"
fi
# --- Parse Summary line (actual FAIL count) ---------------------
# Summary line is on stdout. If the validator output format
# changed (missing Summary, non-numeric FAIL), fail loudly
# instead of silently treating it as zero.
#
# Scope grep's no-match tolerance to grep alone by wrapping just
# the grep stage in a `{ ... || true; }` group. A trailing
# `|| true` on the whole pipeline would defeat `pipefail` and
# swallow producer/head failures too; we only want to tolerate
# grep finding no match (which `[ -z "$summary_line" ]` below
# already reports with a precise error).
summary_line=$(printf '%s\n' "$stdout" | { grep -E '^[[:space:]]*Summary:' || true; } | head -n 1)
if [ -z "$summary_line" ]; then
echo "::error::Could not find validate-pins 'Summary:' line in output"
exit 1
fi
# Word-boundary anchored to avoid matching e.g. `NEWFAIL=` or
# `TOTALFAIL=` if such tokens are ever added to the Summary line.
actual=$(printf '%s\n' "$summary_line" | grep -oE '\bFAIL=[0-9]+\b' | head -n 1 | cut -d= -f2)
if [ -z "${actual:-}" ] || ! [[ "$actual" =~ ^[0-9]+$ ]]; then
echo "::error::Could not parse FAIL=<int> from Summary line: $summary_line"
exit 1
fi
# --- Compute tuple hash of current FAIL set ---------------------
# Hash the sorted, deduplicated `[FAIL] ...` lines (stderr only).
# This catches the "count equal but set drifted" case: one FAIL
# healed while another regressed.
#
# Scope grep's no-match tolerance to grep alone by wrapping just
# the grep stage in a `{ ... || true; }` group. A trailing
# `|| true` on the whole pipeline would defeat `pipefail` and
# swallow sort/shasum/cut failures too; clean runs with zero
# `[FAIL]` lines must not be an error, so we tolerate grep's
# no-match here and only here.
actual_hash=$(printf '%s\n' "$stderr" | { grep -E '^\[FAIL\]' || true; } | LC_ALL=C sort -u | shasum -a 256 | cut -d' ' -f1)
echo "validate-pins FAIL: actual=$actual baseline=$baseline"
echo "validate-pins HASH: actual=$actual_hash baseline=$baseline_hash"
if [ "$actual" -gt "$baseline" ]; then
echo "::error::Pin drift increased: $actual FAIL(s) vs baseline $baseline. Fix the new drift or, with explicit sign-off, update showcase/scripts/fail-baseline.json (bump validatePinsFailCount to $actual and validatePinsFailHash to $actual_hash)."
exit 1
fi
if [ "$actual" -lt "$baseline" ]; then
echo "::error::Pin drift decreased: $actual FAIL(s) vs baseline $baseline. Ratchet down the baseline in showcase/scripts/fail-baseline.json (set validatePinsFailCount=$actual, validatePinsFailHash=$actual_hash)."
exit 1
fi
if [ "$actual_hash" != "$baseline_hash" ]; then
echo "::error::Pin drift SET changed (count equal at $actual, hash differs). One FAIL healed while another regressed — net zero on the counter but the failing tuples are not the same set. Update showcase/scripts/fail-baseline.json (validatePinsFailHash=$actual_hash) if this is intentional, or fix the new drift."
echo "--- FAIL lines (current) ---"
printf '%s\n' "$stderr" | grep -E '^\[FAIL\]' | LC_ALL=C sort -u
exit 1
fi
echo "Pin drift unchanged at baseline ($baseline, hash $baseline_hash)."
- name: Run build pipeline tests
working-directory: showcase/scripts
# Use pnpm to resolve the workspace-installed vitest (pinned via
# pnpm-lock.yaml) rather than `npx`, which could fetch a different
# version on a registry cache miss.
run: pnpm exec vitest run
- name: CVDIAG emit perf-regression gate
working-directory: showcase/harness
# Perf gate for the CVDIAG `CvdiagEmitter` hot path (plan unit L2-D).
# Pure instrumentation must stay cheap on the boundary it observes:
# spec §7 sets a 500µs/event prod budget; this gate holds emit at 50%
# of that for headroom — per-event median ≤100µs, p99 ≤250µs — and the
# bench's teardown throws (failing this step, and the job) on a >20%
# regression past either threshold. vitest `bench` is experimental but
# stable for this single-task run; the throw-on-breach is what gates,
# not the (advisory) hz/p99 table. Resolve vitest via pnpm so the
# pnpm-lock.yaml-pinned version is used (no registry-cache-miss drift).
run: pnpm exec vitest bench src/cvdiag/emit-perf.bench.ts --run
- name: Validate manifests & generate registry
working-directory: showcase/scripts
run: pnpm exec tsx generate-registry.ts
# ADVISORY ONLY — never fail the build. The promote dropdown is
# self-healed by the lefthook pre-commit hook; this step only warns if a
# commit somehow lands with a drifted showcase_promote.yml `service`
# dropdown (e.g. hook skipped). The trailing `|| true` keeps a non-zero
# `--check` exit from reddening validate.
- name: Advisory — promote dropdown drift check
working-directory: showcase/scripts
run: |
# ADVISORY ONLY: capture the exit code without letting a non-zero
# `--check` redden the build. `|| true` alone would discard rc and
# collapse every failure mode into the misleading "stale, re-run"
# warning. sync-promote-service-options.ts exits:
# 1 => drift (dropdown out of date; re-running the generator fixes it)
# 2 => read error
# 3 => missing/duplicate/malformed marker block (corruption)
# Only rc=1 is actually self-heals-by-rerun; rc>=2 needs a human, and
# the suggested re-run would itself fail — so report it distinctly.
set +e
pnpm exec tsx sync-promote-service-options.ts --check
rc=$?
set -e
if [ "$rc" -eq 1 ]; then
echo "::warning::showcase_promote.yml service dropdown is stale. Run: npx tsx showcase/scripts/sync-promote-service-options.ts and commit the result."
elif [ "$rc" -ge 2 ]; then
echo "::warning::sync-promote-service-options.ts failed (exit $rc) — marker block missing/duplicated or read error; investigate before trusting the dropdown."
fi
# Always succeed: this step must never fail the build (the lefthook
# pre-commit hook self-heals; CI only warns).
exit 0
- name: Bundle demo content
working-directory: showcase/scripts
run: pnpm exec tsx bundle-demo-content.ts
- name: Install showcase shell dependencies
working-directory: showcase/shell
# `showcase/shell` is NOT a pnpm workspace member (see pnpm-workspace.yaml)
# and ships its own `package-lock.json`. Use `npm ci` to get a
# reproducible install; `npm install` would re-resolve ranges.
# npm cache is configured at the setup-node step above via
# `cache-dependency-path: showcase/shell/package-lock.json`.
run: npm ci --ignore-scripts
- name: Build showcase shell
working-directory: showcase/shell
run: npm run build
# NOTE: Slack failure alert only fires on `push` (i.e. main-branch
# merges) by design. PR failures already surface in the PR checks UI
# and the PR author's inbox, and we don't want PR-author noise
# pinging the OSS alerts channel. Tradeoff: a broken PR that sneaks
# past review won't alert Slack until after merge.
#
# Extract the failed step name and first meaningful error line so the
# Slack payload is actionable at a glance rather than forcing a
# click-through to the workflow run. Bare "X failed" alerts bury the
# signal; red alerts must carry triage-ready detail per the oss-alerts
# policy. Writes `failed_step` and `error_excerpt` to $GITHUB_ENV for
# consumption by the notify step below.
#
# This step must NEVER fail the job (it runs on failure() already; a
# crash here would compound the original failure with extraction
# noise and could block the notify step). All extraction uses `|| true`
# fallbacks so a malformed jobs response or truncated log still yields
# sane defaults ("unknown" / "see workflow run for details").
- name: Extract failure details for Slack
id: extract
if: failure() && github.event_name == 'push' && env.SLACK_WEBHOOK != ''
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_REPO: ${{ github.repository }}
RUN_ID: ${{ github.run_id }}
run: |
set +e # best-effort: never block the notify step below
# --- Find the currently-running job and its first failed step ---
# The jobs API returns every job in the run. We identify *this*
# job by name (matches `jobs.validate.name`) rather than
# job.status=='in_progress', because at this point the step we're
# running hasn't flipped the job state yet in the API. Fall back
# to the first job with a failed step if the name match misses
# (e.g. future rename drift).
jobs_json=$(gh api "/repos/${GH_REPO}/actions/runs/${RUN_ID}/jobs" --paginate 2>/dev/null)
job_id=$(printf '%s' "$jobs_json" | jq -r '
.jobs // []
| map(select(.name == "Validate Showcase"))
| (.[0].id // empty)
' 2>/dev/null)
if [ -z "$job_id" ]; then
job_id=$(printf '%s' "$jobs_json" | jq -r '
.jobs // []
| map(select(.steps // [] | map(.conclusion) | index("failure")))
| (.[0].id // empty)
' 2>/dev/null)
fi
failed_step=$(printf '%s' "$jobs_json" | jq -r --arg id "$job_id" '
.jobs // []
| map(select((.id|tostring) == $id))
| (.[0].steps // [])
| map(select(.conclusion == "failure"))
| (.[0].name // "unknown step")
' 2>/dev/null)
[ -z "$failed_step" ] && failed_step="unknown step"
# --- Pull log and extract first meaningful error line ------------
# `gh run view --log-failed` output is TSV: job\tstep\ttimestamp + content.
# Strip the three leading columns to get the raw step output, strip
# ANSI escape codes, strip any stray BOM, skip runner/group/env
# header noise, then grab the first line matching a recognised
# error marker. Truncate to ~300 chars so the Slack payload stays
# well under the 800-char budget even with escaping overhead.
error_excerpt="see workflow run for details"
if [ -n "$job_id" ]; then
log_excerpt=$(gh run view "$RUN_ID" --repo "$GH_REPO" --log-failed --job="$job_id" 2>/dev/null \
| awk -F'\t' 'NF>=3 { sub(/^[\xEF\xBB\xBF]?[0-9T:.\-Z ]+/, "", $3); print $3 }' \
| sed 's/\x1b\[[0-9;]*[a-zA-Z]//g' \
| grep -vE '^(##\[|shell: |env: |Run |[[:space:]]*$)' \
| grep -m1 -E '^\[(FAIL|ERROR)\]|^Error:|^error:|^::error' \
| head -c 300)
if [ -n "$log_excerpt" ]; then
error_excerpt="$log_excerpt"
fi
fi
# --- Emit to $GITHUB_ENV using heredoc delimiter -----------------
# Heredoc delimiter protects against values that contain `=` or
# newlines breaking the KEY=VALUE format. The delimiter is a
# long random-ish string unlikely to appear in any log line.
{
echo "failed_step<<EOF_FAILED_STEP_b3f2"
printf '%s\n' "$failed_step"
echo "EOF_FAILED_STEP_b3f2"
echo "error_excerpt<<EOF_ERROR_EXCERPT_b3f2"
printf '%s\n' "$error_excerpt"
echo "EOF_ERROR_EXCERPT_b3f2"
} >> "$GITHUB_ENV"
exit 0 # belt-and-suspenders: never propagate a failure
- name: Mark inline Slack notifier reached
id: inline_slack_marker
if: failure() && github.event_name == 'push' && env.SLACK_WEBHOOK != ''
run: echo "reached=true" >> "$GITHUB_OUTPUT"
- name: Notify Slack (failure)
if: failure() && github.event_name == 'push' && env.SLACK_WEBHOOK != ''
uses: slackapi/slack-github-action@0d95c9a7becc1e6e297d76df9bc735c44f4cbcbc # v3.0.5
with:
webhook: ${{ secrets.SLACK_WEBHOOK_OSS_ALERTS }}
webhook-type: incoming-webhook
# Defensive: wrap dynamic values via toJSON(format(...)) so that
# if github.repository or the extracted failed_step / error_excerpt
# contain characters that would break the JSON payload (quotes,
# backslashes, newlines), the value is safely JSON-encoded instead
# of injected as raw text. Matches the pattern used in
# showcase_drift-report.yml. github.run_id is numeric so safe on
# its own, but we wrap it for consistency and defense-in-depth.
# env.failed_step and env.error_excerpt are populated by the
# preceding "Extract failure details" step (with safe fallbacks if
# extraction fails).
payload: |
{ "text": ${{ toJSON(format(':x: *Showcase validate*: failed — {0}: {1} | <https://github.com/{2}/actions/runs/{3}|View run>', env.failed_step, env.error_excerpt, github.repository, github.run_id)) }} }
- name: Log (no Slack — webhook unset)
if: failure() && github.event_name == 'push' && env.SLACK_WEBHOOK == ''
run: |
echo "::warning::showcase_validate failed on push but SLACK_WEBHOOK_OSS_ALERTS is not set; no Slack notification sent."
shell-script-tests:
name: Shell script tests (bats + shellcheck)
# Separate job (mirrors python-unit-tests) so the showcase shell-script
# regression suite runs independently of the JS/TS validate job. Runs on
# ubuntu-latest where shellcheck is preinstalled; bats is apt-installed.
# These tests gate the promote-fleet.sh best-effort loop + succeeded_csv
# export that the promote → verify-prod handoff depends on.
runs-on: ubuntu-latest
# The bats suite runs ~4.5min and keeps growing; at the old 5min job cap it
# raced the deadline and intermittently got cancelled mid-suite (all steps
# passing) rather than reported. Give headroom so a green suite reports green.
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Install bats
run: |
# GitHub's ubuntu-latest runner image preconfigures third-party apt
# repos (Microsoft / azure-cli) for preinstalled tooling this job does
# not use. When one of those repos serves invalid release metadata,
# `apt-get update` exits non-zero and `bash -e` aborts the step —
# even though bats comes from Ubuntu's own `universe` repo, which is
# unaffected. This job only needs Ubuntu packages, so drop those unused
# third-party repos before updating.
sudo rm -f /etc/apt/sources.list.d/*microsoft* /etc/apt/sources.list.d/*azure-cli*
sudo apt-get update
sudo apt-get install -y bats
- name: Shellcheck promote workflow scripts
# shellcheck is preinstalled on ubuntu-latest.
run: shellcheck showcase/scripts/promote-fleet.sh showcase/scripts/verify-prod-display.sh showcase/scripts/reconcile-prod-gate.sh
- name: Run bats suite
run: bats showcase/scripts/__tests__/
python-unit-tests:
name: Python unit tests (${{ matrix.python-version }})
# Separate job so pre-existing `validate-parity` failures don't mask new
# Python unit-test regressions. pytest runs independently of JS/TS checks.
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
strategy:
# Fail-fast disabled so a 3.10-only regression (e.g. typing_extensions
# fallback path breaking) doesn't cancel the 3.12 run and leave us
# guessing which version is the actual problem.
fail-fast: false
matrix:
# 3.10 covers the typing_extensions `NotRequired` fallback path used
# by aimock_toggle.py (stdlib `NotRequired` only landed in 3.11).
# 3.12 is the production/runner default. Pinning both guarantees we
# catch a regression in either branch the first time it lands.
python-version: ["3.10", "3.12"]
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Setup Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with:
python-version: ${{ matrix.python-version }}
cache: "pip"
cache-dependency-path: |
showcase/integrations/*/requirements.txt
- name: Install minimal test deps
# Always need pytest + typing_extensions. pytest-asyncio is required
# by langroid's test_agui_adapter.py (16 tests use
# `@pytest.mark.asyncio`); without it, pytest reports
# "async def functions are not natively supported" and skips them.
# pytest-mock is installed pre-emptively as it's commonly used by
# showcase package tests and is cheap to install.
# Per-package `requirements.txt` is installed inside the run loop
# below so tests that import runtime deps (openai, google.genai,
# httpx, opentelemetry, etc.) don't fail at collection time with
# ModuleNotFoundError. Conftest-based stub finders can't help
# because test_*.py imports the target deps BEFORE conftest runs.
run: python -m pip install --quiet pytest pytest-asyncio pytest-mock typing_extensions
- name: Run showcase package Python unit tests
# Keep scope narrow: only showcase/integrations/*/tests/python/ directories
# (not e2e, not langgraph which has its own runtime). Each package has
# its own conftest.py that wires up import paths; we cd into the pkg
# dir so those apply.
#
# Before running pytest in a package we install that package's own
# `requirements.txt` (if present) so runtime-dep imports in test modules
# resolve. Keeps CI parity with real runtime and avoids the fragile
# stub-finder dance conftest.py would need to do otherwise.
run: |
set -euo pipefail
failed=0
found=0
# Current interpreter major.minor (e.g. "3.10", "3.12"). Used
# below to skip packages whose runtime deps are incompatible
# with the matrix Python on this job.
py_mm=$(python -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")')
for pkg_dir in showcase/integrations/*/; do
tests_dir="${pkg_dir}tests/python"
[ -d "$tests_dir" ] || continue
found=$((found + 1))
pkg=$(basename "$pkg_dir")
# --- Per-package Python-version gates -------------------------
# Skip packages whose `requirements.txt` pins a dep whose
# `requires-python` excludes this interpreter. Surgical skip
# (not matrix exclusion) so the rest of the packages continue
# to exercise the 3.10 typing_extensions fallback path.
#
# claude-sdk-python: ag-ui-claude-sdk declares `requires-python >=3.11`;
# the package Dockerfile and production runner use Python 3.12.
# strands: ag_ui_strands==0.1.0 declares `requires-python >=3.12,<3.14`,
# so `pip install` fails on 3.10 before pytest even runs.
# langroid: tests import `typing.Self` (3.11+); on 3.10 the import fails
# at collection time. typing_extensions.Self would fix it but the tests
# are tightly coupled to the modern typing module.
# Revisit when ag_ui_strands relaxes its floor or when 3.10 is dropped.
if [ "$py_mm" = "3.10" ] && { [ "$pkg" = "claude-sdk-python" ] || [ "$pkg" = "strands" ] || [ "$pkg" = "langroid" ]; }; then
echo "--- pytest: $pkg --- SKIPPED on Python $py_mm (requires >=3.11/3.12)"
continue
fi
echo "--- pytest: $pkg ---"
if [ -f "${pkg_dir}requirements.txt" ]; then
echo "Installing ${pkg_dir}requirements.txt"
python -m pip install --quiet -r "${pkg_dir}requirements.txt" || {
echo "::error::pip install failed for $pkg"
failed=1
continue
}
fi
# Export PYTHONPATH so `from tools import ...` in agent modules
# resolves via the `tools` symlink at the integration root.
# Also include src/ so `from agents.X import ...` works even if
# a conftest.py omits the sys.path setup. Mirrors the local dev
# convention (`PYTHONPATH=. python ...` in package.json scripts).
(cd "$pkg_dir" && PYTHONPATH=".:src:${PYTHONPATH:-}" python -m pytest tests/python/ -v) || failed=1
done
if [ "$found" -eq 0 ]; then
echo "::warning::No showcase/integrations/*/tests/python/ directories found"
fi
exit "$failed"
notify:
# Slack #oss-alerts on any red. Never #engr (engr is sacred — release alerts only).
# Mirrors the workflow-level notify pattern in showcase_promote.yml so
# red runs surface uniformly across the showcase pipeline. The validate
# job has its own inline (and richer) push-only notifier that extracts
# the failing step + error excerpt; this job is the workflow-level
# safety net that also covers the python-unit-tests matrix job and the
# shell-script-tests job — which otherwise had no Slack signal at all.
# Webhook empty-guard mirrors promote.yml so an unset
# SLACK_WEBHOOK_OSS_ALERTS secret does not break the shell or red the
# workflow on this step.
needs: [validate, python-unit-tests, shell-script-tests]
if: always() && github.event_name == 'push' && !(needs.validate.result == 'failure' && needs.validate.outputs.inline_slack_notifier_reached == 'true' && needs.python-unit-tests.result == 'success' && needs.shell-script-tests.result == 'success')
runs-on: ubuntu-latest
timeout-minutes: 3
permissions:
contents: read
actions: read
env:
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_OSS_ALERTS }}
steps:
- name: Compute state
id: state
env:
VALIDATE: ${{ needs.validate.result }}
PYTEST: ${{ needs.python-unit-tests.result }}
SHELL: ${{ needs.shell-script-tests.result }}
run: |
set -euo pipefail
if [ "$VALIDATE" = "success" ] && [ "$PYTEST" = "success" ] && [ "$SHELL" = "success" ]; then
STATE="success"; ICON=":white_check_mark:"
else
STATE="failure"; ICON=":x:"
fi
{
echo "state=$STATE"
echo "icon=$ICON"
} >> "$GITHUB_OUTPUT"
- name: Post to #oss-alerts
if: steps.state.outputs.state == 'failure' && env.SLACK_WEBHOOK != ''
uses: slackapi/slack-github-action@0d95c9a7becc1e6e297d76df9bc735c44f4cbcbc # v3.0.5
with:
webhook: ${{ secrets.SLACK_WEBHOOK_OSS_ALERTS }}
webhook-type: incoming-webhook
# Newlines are injected via fromJSON('"\n"') (a real LF char) as {8},
# NOT a literal '\n' in the template: GitHub Actions expression string
# literals do not interpret backslash escapes, so a literal '\n' would
# survive toJSON as the two chars \\n and Slack would render it
# verbatim as "\n" instead of a line break.
payload: |
{
"text": ${{ toJSON(format(
'{0} *showcase_validate failed on {1}*{8}validate={2} python-unit-tests={3} shell-script-tests={4}{8}<{5}/{6}/actions/runs/{7}|View run>',
steps.state.outputs.icon,
github.ref,
needs.validate.result,
needs.python-unit-tests.result,
needs.shell-script-tests.result,
github.server_url,
github.repository,
github.run_id,
fromJSON('"\n"')
)) }}
}
- name: Log (no Slack — webhook unset)
if: steps.state.outputs.state == 'failure' && env.SLACK_WEBHOOK == ''
env:
REF: ${{ github.ref }}
run: |
echo "::warning::showcase_validate failed on $REF but SLACK_WEBHOOK_OSS_ALERTS is not set; no Slack notification sent."
+403
View File
@@ -0,0 +1,403 @@
name: social / copy-generator
on:
pull_request:
types: [opened]
issue_comment:
types: [edited]
workflow_dispatch:
inputs:
pr_number:
description: "PR number to generate social copies for"
required: true
type: number
permissions:
contents: read
jobs:
post-comment:
if: >-
(github.event_name == 'workflow_dispatch') ||
(github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
issues: write
steps:
- name: Post initial comment
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
env:
PR_NUMBER: ${{ github.event.inputs.pr_number || github.event.pull_request.number }}
with:
script: |
const prNumber = Number(process.env.PR_NUMBER);
// For workflow_dispatch, verify the PR isn't from a fork
if (context.eventName === 'workflow_dispatch') {
const pr = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber,
});
if (pr.data.head.repo?.full_name !== `${context.repo.owner}/${context.repo.repo}`) {
console.log('PR is from a fork, skipping');
return;
}
}
// Check if there's already a social-copy-generator comment on this PR
const comments = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
per_page: 100,
});
const existing = comments.data.find(c => c.body.includes('<!-- social-copy-generator -->'));
if (existing) {
console.log(`Comment already exists (id: ${existing.id}), skipping`);
return;
}
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
body: [
'<!-- social-copy-generator -->',
'### 📣 Social Copy Generator',
'',
'Generate social media copies (Twitter/X, LinkedIn, Blog Post) for this PR using Claude.',
'',
'- [ ] **Generate social media copies**',
].join('\n'),
});
generate:
if: >-
github.event_name == 'issue_comment' &&
github.event.issue.pull_request &&
contains(github.event.comment.body, '<!-- social-copy-generator -->') &&
contains(github.event.comment.body, '- [x]')
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
issues: write
concurrency:
group: social-copy-${{ github.event.issue.number }}
cancel-in-progress: true
steps:
- name: Verify checkbox transition and permissions
id: verify
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
// Check that the sender has write access (not an external collaborator or random user)
const sender = context.payload.sender.login;
try {
const { data: permissionLevel } = await github.rest.repos.getCollaboratorPermissionLevel({
owner: context.repo.owner,
repo: context.repo.repo,
username: sender,
});
const allowed = ['admin', 'write'];
if (!allowed.includes(permissionLevel.permission)) {
core.setOutput('should_run', 'false');
console.log(`User ${sender} has '${permissionLevel.permission}' permission, skipping`);
return;
}
} catch (e) {
core.setOutput('should_run', 'false');
console.log(`Could not verify permissions for ${sender}, skipping`);
return;
}
const oldBody = context.payload.changes?.body?.from || '';
const newBody = context.payload.comment.body;
const wasUnchecked = oldBody.includes('- [ ]');
const isNowChecked = newBody.includes('- [x]');
if (!wasUnchecked || !isNowChecked) {
core.setOutput('should_run', 'false');
console.log('Not a checkbox transition, skipping');
return;
}
core.setOutput('should_run', 'true');
core.setOutput('comment_id', context.payload.comment.id);
- name: Update comment to generating state
if: steps.verify.outputs.should_run == 'true'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
env:
COMMENT_ID: ${{ steps.verify.outputs.comment_id }}
with:
script: |
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: Number(process.env.COMMENT_ID),
body: [
'<!-- social-copy-generator -->',
'### 📣 Social Copy Generator',
'',
`⏳ **Analyzing PR changes with Claude...** This may take a minute. [View progress](${runUrl})`,
].join('\n'),
});
- name: Get PR details
if: steps.verify.outputs.should_run == 'true'
id: pr
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
const pr = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.issue.number,
});
core.setOutput('head_ref', pr.data.head.ref);
core.setOutput('head_sha', pr.data.head.sha);
core.setOutput('base_ref', pr.data.base.ref);
core.setOutput('title', pr.data.title);
core.setOutput('body', pr.data.body || '(no description)');
core.setOutput('number', String(context.issue.number));
- name: Checkout repo
if: steps.verify.outputs.should_run == 'true'
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
ref: ${{ steps.pr.outputs.head_sha }}
fetch-depth: 0
persist-credentials: false
- name: Remove repository Claude settings
if: steps.verify.outputs.should_run == 'true'
run: rm -f .claude/settings.json .claude/settings.local.json .mcp.json
- name: Get PR diff
if: steps.verify.outputs.should_run == 'true'
id: diff
env:
BASE_REF: ${{ steps.pr.outputs.base_ref }}
PR_NUMBER: ${{ steps.pr.outputs.number }}
GH_TOKEN: ${{ github.token }}
run: |
# Write diff files inside the repo so Claude's sandbox can access them
mkdir -p .claude-tmp
# Try git merge-base first (works for open PRs and merged PRs whose base is fetchable)
MERGE_BASE=$(git merge-base "origin/$BASE_REF" HEAD 2>/dev/null) || true
if [ -n "$MERGE_BASE" ]; then
git diff --stat "$MERGE_BASE"..HEAD > .claude-tmp/diff-stat.txt
git diff "$MERGE_BASE"..HEAD > .claude-tmp/diff-full.txt
else
# Fallback: use GitHub API diff (works for merged PRs where branch was deleted)
echo "Using GitHub API for diff (merge-base not found)"
gh pr diff "$PR_NUMBER" --patch > .claude-tmp/diff-full.txt
gh pr diff "$PR_NUMBER" | head -200 > .claude-tmp/diff-stat.txt
fi
- name: Setup pnpm
if: steps.verify.outputs.should_run == 'true'
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Setup Node.js
if: steps.verify.outputs.should_run == 'true'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22
cache: pnpm
- name: Install Claude Code
if: steps.verify.outputs.should_run == 'true'
# @anthropic-ai/claude-code is pinned as a root devDependency and installed
# from the frozen lockfile — no ad-hoc `npm install -g`. `--ignore-scripts`
# keeps install-time scripts from running against the PR-head checkout; the
# `cli-wrapper.cjs` entrypoint (used below) resolves the native binary from
# the installed optionalDependency without needing the postinstall step.
run: pnpm install --frozen-lockfile --ignore-scripts
- name: Write prompt file
if: steps.verify.outputs.should_run == 'true'
env:
PR_TITLE: ${{ steps.pr.outputs.title }}
PR_BRANCH: ${{ steps.pr.outputs.head_ref }}
PR_BODY: ${{ steps.pr.outputs.body }}
run: |
cat << 'PROMPT_EOF' > .claude-tmp/prompt.txt
You are a developer advocate and social media copywriter for CopilotKit — an open-source AI agent framework that lets developers add AI copilots to their products with React hooks, UI components, and agent infrastructure. CopilotKit connects frontend (React/Angular), runtime (Express/Hono), and AI agents (LangGraph, CrewAI, custom) via the AG-UI protocol.
A PR has been opened. Your job is to analyze the changes and generate social media copy that the team can use to announce this work.
## PR Information
PROMPT_EOF
echo "- **Title:** $PR_TITLE" >> .claude-tmp/prompt.txt
echo "- **Branch:** $PR_BRANCH" >> .claude-tmp/prompt.txt
echo "- **Description:**" >> .claude-tmp/prompt.txt
echo "$PR_BODY" >> .claude-tmp/prompt.txt
cat << 'PROMPT_EOF' >> .claude-tmp/prompt.txt
## Instructions
1. Read the diff stat file at `.claude-tmp/diff-stat.txt` to understand the scope of changes.
2. Read the full diff at `.claude-tmp/diff-full.txt` to understand the details.
3. If needed, read relevant source files to understand context.
4. Generate the following three pieces of content:
### Output Format
Produce your output in EXACTLY this format (including the markdown headers).
Do NOT include any preamble, analysis summary, or commentary before or after the three sections.
Start directly with the first header:
### 🐦 Twitter/X Post
(A concise, engaging tweet under 280 characters. No hashtags. Focus on the user benefit.)
### 💼 LinkedIn Post
(A professional post of 3-5 short paragraphs. Lead with the value proposition. Include a call to action. Add relevant hashtags. No length limit.)
### 📝 Blog Post Draft
(A blog-style announcement. Include a title, explain what changed, why it matters, and how to get started. No length limit — be as thorough as needed.)
## Guidelines
- Focus on user-facing impact, not implementation details
- Be enthusiastic but authentic — avoid hype
- Mention CopilotKit by name and link to https://github.com/CopilotKit/CopilotKit
- If the changes are internal/minor, still generate copy but note it's a maintenance/improvement update
PROMPT_EOF
- name: Run Claude Code
if: steps.verify.outputs.should_run == 'true'
id: claude
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
PROMPT=$(cat .claude-tmp/prompt.txt)
# Invoke the workspace-installed Claude Code via its cli-wrapper entrypoint.
# We installed with --ignore-scripts (PR-head safety), so the postinstall
# that copies the native binary over the `claude` bin stub did not run;
# cli-wrapper.cjs is the package's documented fallback that resolves and
# spawns the native binary from the installed optionalDependency.
node node_modules/@anthropic-ai/claude-code/cli-wrapper.cjs -p "$PROMPT" \
--output-format json \
--max-turns 10 \
--allowedTools "Read,Bash(git diff:*),Bash(git log:*),Bash(git show:*),Bash(cat:*)" \
--disallowedTools "TodoWrite,Edit,MultiEdit,Write,NotebookEdit,WebFetch,WebSearch,Task" \
> .claude-tmp/claude-output.json
- name: Extract result to markdown
if: steps.verify.outputs.should_run == 'true' && always() && steps.claude.outcome != 'skipped'
id: extract
run: |
node << 'EXTRACT_SCRIPT' > .claude-tmp/pr-social-copy.md
const fs = require('fs');
const file = '.claude-tmp/claude-output.json';
if (!fs.existsSync(file)) {
process.exit(0);
}
const raw = fs.readFileSync(file, 'utf8').trim();
let result = '';
// With --output-format json, output is a single JSON object with a "result" field
try {
const obj = JSON.parse(raw);
result = obj.result || '';
} catch {
// Fallback: try JSONL format (one JSON object per line)
for (const line of raw.split('\n')) {
try {
const obj = JSON.parse(line);
if (obj.type === 'result' && obj.result) {
result = obj.result;
}
if (!result && obj.type === 'assistant' && obj.message?.content) {
for (const block of obj.message.content) {
if (block.type === 'text' && block.text && block.text.includes('### ')) {
result = block.text;
}
}
}
} catch {}
}
}
// Strip any preamble before the first ### header
const idx = result.indexOf('### ');
const clean = idx >= 0 ? result.slice(idx) : result;
process.stdout.write(clean);
EXTRACT_SCRIPT
- name: Upload social copy artifact
if: steps.verify.outputs.should_run == 'true' && always() && steps.extract.outcome == 'success'
id: artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: social-copy-pr${{ github.event.issue.number }}
path: .claude-tmp/pr-social-copy.md
retention-days: 90
- name: Update comment with results
if: steps.verify.outputs.should_run == 'true' && always() && steps.extract.outcome == 'success'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
env:
HEAD_SHA: ${{ steps.pr.outputs.head_sha }}
COMMENT_ID: ${{ steps.verify.outputs.comment_id }}
ARTIFACT_ID: ${{ steps.artifact.outputs.artifact-id }}
with:
script: |
const fs = require('fs');
const output = fs.readFileSync('.claude-tmp/pr-social-copy.md', 'utf8').trim() || '_Claude did not produce output. Please try again._';
const timestamp = new Date().toISOString().replace('T', ' ').slice(0, 19) + ' UTC';
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
const artifactUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}/artifacts/${process.env.ARTIFACT_ID}`;
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: Number(process.env.COMMENT_ID),
body: [
'<!-- social-copy-generator -->',
'### 📣 Social Copy Generator',
'',
output,
'',
'---',
`_Generated at ${timestamp} from commit \`${process.env.HEAD_SHA}\` | [Workflow run](${runUrl}) | [Download markdown](${artifactUrl})_`,
'',
'- [ ] **Regenerate social media copies**',
].join('\n'),
});
- name: Update comment on failure
if: steps.verify.outputs.should_run == 'true' && always() && steps.extract.outcome != 'success'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
env:
COMMENT_ID: ${{ steps.verify.outputs.comment_id }}
with:
script: |
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: Number(process.env.COMMENT_ID),
body: [
'<!-- social-copy-generator -->',
'### 📣 Social Copy Generator',
'',
`❌ **Generation failed.** [View workflow logs](${runUrl}) for details.`,
'',
'- [ ] **Regenerate social media copies**',
].join('\n'),
});
+200
View File
@@ -0,0 +1,200 @@
name: release / create-pr
on:
workflow_dispatch:
inputs:
scope:
description: "What to release"
required: true
type: choice
options:
- monorepo
- angular
- channels
- channels-discord
- channels-intelligence
- channels-slack
- channels-teams
- channels-telegram
- channels-whatsapp
bump:
description: "Version bump level"
required: true
type: choice
options:
- patch
- minor
- major
dry_run:
description: "Dry run (preview without creating PR)"
required: false
default: false
type: boolean
concurrency:
# Scope the lock to the package being released so that, e.g., a `monorepo`
# create-pr run and an `angular` create-pr run proceed in independent lanes
# instead of queuing behind each other. Same-scope runs still serialize
# (cancel-in-progress: false), which is what protects the version bump.
group: release-pr-${{ inputs.scope }}
cancel-in-progress: false
permissions:
contents: write
pull-requests: write
env:
NX_VERBOSE_LOGGING: true
jobs:
create-release-pr:
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
timeout-minutes: 15
environment: npm
steps:
- name: Check for existing release PR
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { owner, repo } = context.repo;
const { data: prs } = await github.rest.pulls.list({
owner,
repo,
state: "open",
head_prefix: `${owner}:release/publish/`,
});
const releasePRs = prs.filter(pr => pr.head.ref.startsWith("release/publish/"));
if (releasePRs.length > 0) {
const existing = releasePRs.map(pr => ` - #${pr.number}: ${pr.title} (${pr.html_url})`).join("\n");
core.setFailed(
`An open release PR already exists. Close or merge it before creating a new one:\n${existing}`
);
}
- name: Checkout Repo
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
persist-credentials: false
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Setup Node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.x
- name: Install Dependencies
run: pnpm install --frozen-lockfile
- name: Prepare release
id: prepare
run: |
if [ "${{ inputs.dry_run }}" == "true" ]; then
pnpm tsx scripts/release/prepare-release.ts --bump ${{ inputs.bump }} --scope ${{ inputs.scope }} --dry-run
else
pnpm tsx scripts/release/prepare-release.ts --bump ${{ inputs.bump }} --scope ${{ inputs.scope }}
fi
- name: Sync plugin skills
if: inputs.dry_run != true
run: pnpm sync:plugin-skills
- name: Generate AI release notes
if: inputs.dry_run != true
id: ai_notes
run: pnpm tsx scripts/release/generate-ai-release-notes.ts "${{ steps.prepare.outputs.version }}"
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
NOTION_API_KEY: ${{ secrets.NOTION_API_KEY }}
NOTION_RELEASE_NOTES_PAGE: ${{ secrets.NOTION_RELEASE_NOTES_PAGE }}
- name: Mint devops-bot token
if: inputs.dry_run != true
id: app-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: 1108748
private-key: ${{ secrets.DEVOPS_BOT_PRIVATE_KEY }}
permission-contents: write
permission-pull-requests: write
permission-issues: write
- name: Create release PR
if: inputs.dry_run != true
id: create_pr
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8
env:
# The PR branch is validated by CI after creation; do not run local
# developer pre-commit hooks inside the automation commit.
LEFTHOOK: "0"
with:
token: ${{ steps.app-token.outputs.token }}
branch: release/publish/${{ inputs.scope }}/v${{ steps.prepare.outputs.version }}
delete-branch: true
commit-message: "chore: release ${{ inputs.scope }} v${{ steps.prepare.outputs.version }}"
title: "chore: release ${{ inputs.scope }} v${{ steps.prepare.outputs.version }}"
body: |
## Release ${{ inputs.scope }} v${{ steps.prepare.outputs.version }}
**Scope:** `${{ inputs.scope }}` | **Bump:** `${{ inputs.bump }}`
---
### How this release process works
1. **This PR was created automatically** by the "release / create-pr" workflow.
It bumped the `${{ inputs.scope }}` packages to `${{ steps.prepare.outputs.version }}`
and generated AI-enhanced release notes.
2. **CI runs on this PR** — the full test suite (unit tests, lint, type checks, build)
must pass before merging. This is the review gate.
3. **Review the release notes** in `release-notes.md` in this PR.
If a Notion draft was created, you can edit the release notes there before merging.
4. **When this PR is merged**, the `release / publish` workflow automatically:
- Builds all packages
- Publishes the `${{ inputs.scope }}` packages to npm at version `${{ steps.prepare.outputs.version }}`
- Creates git tag `${{ inputs.scope }}/v${{ steps.prepare.outputs.version }}`
- Creates a GitHub Release with the final release notes
### Before merging
- [ ] CI is green (tests, lint, types, build)
- [ ] Version bumps look correct
- [ ] Release notes are accurate (edit in Notion if a draft was created)
---
> **Do not merge until CI is fully green.** The full test suite runs automatically on this PR.
labels: release
- name: Comment Notion link on PR
if: inputs.dry_run != true && steps.ai_notes.outputs.notion_url
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
env:
NOTION_URL: ${{ steps.ai_notes.outputs.notion_url }}
PR_NUMBER: ${{ steps.create_pr.outputs.pull-request-number }}
with:
github-token: ${{ steps.app-token.outputs.token }}
script: |
const { owner, repo } = context.repo;
const prNumber = parseInt(process.env.PR_NUMBER, 10);
const notionUrl = process.env.NOTION_URL;
if (prNumber && notionUrl) {
await github.rest.issues.createComment({
owner,
repo,
issue_number: prNumber,
body: `📝 **Release notes draft:** ${notionUrl}\n\nYou can edit the release notes in Notion before merging. The final content will be used for the GitHub Release.`,
});
}
+104
View File
@@ -0,0 +1,104 @@
name: static / bundle-size
on:
pull_request:
paths-ignore:
- "README.md"
- "examples/**"
- "showcase/**"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
NODE_OPTIONS: "--max-old-space-size=4096"
NX_VERBOSE_LOGGING: true
jobs:
# Posts a per-PR comment with per-file gzip diffs for the 9 in-scope packages.
# Runs preactjs/compressed-size-action, which builds the PR head and the base
# branch, scans the `pattern` glob in each, and diffs the gzip sizes; the
# comment updates in place on subsequent pushes. No hard-fail (Phase 1) — see
# dev-docs/bundle-size.md for the rollout plan.
#
# Fork limitation: pull_request runs from a fork get a read-only GITHUB_TOKEN,
# so the action cannot post the comment and prints the report to the logs
# instead. The measurement still runs. Accepted for Phase 1 (informational, no
# hard-fail); see dev-docs/bundle-size.md for the relay pattern if the comment
# ever becomes required.
bundle-size:
runs-on: ubuntu-latest
timeout-minutes: 30
# pull-requests: write is scoped to this job only — it's the one that posts
# the size-diff comment. The import-size job below stays read-only.
permissions:
contents: read
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
fetch-depth: 0
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Use Node.js 20
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.x
# setup-node built-in cache is fork-safe (fork PRs can't write to base repo cache)
cache: "pnpm"
cache-dependency-path: "**/pnpm-lock.yaml"
- name: Measure compressed bundle sizes
uses: preactjs/compressed-size-action@f322c295dde06a1cb7ccaef105732dd8a726d1d9 # 2.10.0
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
# Use the root `build` script (present on both this branch and the base
# branch) so compressed-size-action can build both sides for comparison.
# The `pattern` below restricts measurement to the 9 in-scope packages.
build-script: build
pattern: "packages/{core,shared,react-core,react-ui,react-textarea,runtime-client-gql,web-inspector,voice,a2ui-renderer}/dist/**/*.{mjs,js,cjs}"
# Measures what an app importing { CopilotChat } from
# @copilotkit/react-core/v2 bundles, by driving esbuild over a synthetic entry
# and summing the gzipped output. Writes the total to the GitHub job summary
# for per-PR visibility. It is a relative regression signal across PRs, not a
# production Vite/Next figure — see dev-docs/bundle-size.md.
copilotchat-import-size:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Setup pnpm
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Use Node.js 20
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.x
cache: "pnpm"
cache-dependency-path: "**/pnpm-lock.yaml"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build react-core
run: npx nx run @copilotkit/react-core:build
- name: Measure CopilotChat bundle regression signal
run: pnpm --filter @copilotkit/react-core size:headline
+104
View File
@@ -0,0 +1,104 @@
name: static / check binaries
on:
pull_request:
branches: [main]
permissions:
contents: read
jobs:
check-config-files:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Check build config allowlist
run: bash .github/scripts/check-config-allowlist.sh
check-binaries:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
fetch-depth: 0
persist-credentials: false
- name: Check for binary and build artifacts
env:
BASE_REF: ${{ github.event.pull_request.base.ref }}
run: |
VIOLATIONS=0
# Get list of added/modified files in the PR
CHANGED_FILES=$(git diff --name-only "origin/${BASE_REF}...HEAD")
if [ -z "$CHANGED_FILES" ]; then
echo "No changed files detected."
exit 0
fi
# Check for binary file extensions
BINARY_FILES=$(echo "$CHANGED_FILES" | grep -iE '\.(exe|dll|so|dylib|o|obj|a|lib|wasm)$' || true)
if [ -n "$BINARY_FILES" ]; then
echo "::error::Binary files detected in PR:"
echo "$BINARY_FILES"
VIOLATIONS=1
fi
# Check for build directories
BUILD_FILES=$(echo "$CHANGED_FILES" | grep -E '/build/' || true)
if [ -n "$BUILD_FILES" ]; then
echo "::error::Files in build directories detected in PR:"
echo "$BUILD_FILES"
VIOLATIONS=1
fi
# Check for dSYM directories
DSYM_FILES=$(echo "$CHANGED_FILES" | grep -E '\.dSYM/' || true)
if [ -n "$DSYM_FILES" ]; then
echo "::error::dSYM debug symbol directories detected in PR:"
echo "$DSYM_FILES"
VIOLATIONS=1
fi
# Check for large files (>1MB) among changed files
# Exclude known large files: lockfiles, assets, bundled actions
LARGE_FILES=""
while IFS= read -r file; do
if [ -f "$file" ]; then
case "$file" in
pnpm-lock.yaml|*/pnpm-lock.yaml|*/package-lock.json|*/poetry.lock) continue ;;
assets/*|examples/*/preview.gif|examples/*/assets/*) continue ;;
.github/actions/*/dist/*) continue ;;
showcase/shell/src/data/*|showcase/shell-docs/src/data/*|showcase/shell-dojo/src/data/demo-content.json) continue ;;
esac
SIZE=$(wc -c < "$file" | tr -d ' ')
if [ "$SIZE" -gt 1048576 ]; then
LARGE_FILES="${LARGE_FILES}${file} ($(( SIZE / 1024 )) KB)\n"
fi
fi
done <<< "$CHANGED_FILES"
if [ -n "$LARGE_FILES" ]; then
echo "::error::Files over 1 MB detected in PR:"
echo -e "$LARGE_FILES"
VIOLATIONS=1
fi
if [ "$VIOLATIONS" -eq 1 ]; then
echo ""
echo "This PR contains binary artifacts, build outputs, or oversized files."
echo "Please remove them and update your .gitignore if needed."
exit 1
fi
echo "No binary artifacts or oversized files detected."
+57
View File
@@ -0,0 +1,57 @@
name: static / compat
on:
push:
branches: [main]
paths-ignore:
- "README.md"
- "examples/**"
- "showcase/**"
pull_request:
paths-ignore:
- "README.md"
- "examples/**"
- "showcase/**"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
compat-check:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Use Node.js 20
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.x
# setup-node built-in cache is fork-safe (fork PRs can't write to base repo cache)
cache: "pnpm"
cache-dependency-path: "**/pnpm-lock.yaml"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build packages
run: >
npx nx run-many -t build
--projects=@copilotkit/core,@copilotkit/shared,@copilotkit/react-core,@copilotkit/react-ui,@copilotkit/react-textarea,@copilotkit/runtime-client-gql,@copilotkit/web-inspector,@copilotkit/voice,@copilotkit/a2ui-renderer
- name: Run compat-check
run: >
npx nx run-many -t compat-check
--projects=@copilotkit/core,@copilotkit/shared,@copilotkit/react-core,@copilotkit/react-ui,@copilotkit/react-textarea,@copilotkit/runtime-client-gql,@copilotkit/web-inspector,@copilotkit/voice,@copilotkit/a2ui-renderer
+55
View File
@@ -0,0 +1,55 @@
name: static / danger
on:
pull_request:
paths:
- "sdk-python/copilotkit/langgraph_agent.py"
- "packages/sdk-js/src/langgraph.ts"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
NX_VERBOSE_LOGGING: true
NX_CI_EXECUTION_ID: ${{ github.head_ref }}-${{ github.sha }}-${{ github.run_attempt }}
NX_CI_EXECUTION_ENV: "Danger"
permissions:
contents: read
jobs:
danger:
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
# Danger posts review comments on the PR via GITHUB_TOKEN
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Use Node.js 20
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.x
# setup-node built-in cache is fork-safe (fork PRs can't write to base repo cache)
cache: "pnpm"
cache-dependency-path: "**/pnpm-lock.yaml"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Run Danger
run: pnpm exec danger ci
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+419
View File
@@ -0,0 +1,419 @@
name: static / quality
on:
push:
branches: [main]
paths-ignore:
- "README.md"
- "examples/**"
pull_request:
branches: [main]
paths-ignore:
- "README.md"
- "examples/**"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
NODE_OPTIONS: "--max-old-space-size=4096"
NX_VERBOSE_LOGGING: true
NX_CI_EXECUTION_ID: ${{ github.head_ref }}-${{ github.sha }}-${{ github.run_attempt }}
NX_CI_EXECUTION_ENV: "Static Quality"
permissions:
contents: read
jobs:
format:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: write
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
ref: ${{ github.event_name == 'pull_request' && github.head_ref || github.ref }}
# Check the head branch out from the head repo, not the base repo.
# For fork PRs the head branch only exists on the fork, so defaulting
# to the base repo makes checkout fail with "a branch or tag with the
# name '<branch>' could not be found". Same-repo PRs resolve to the
# base repo unchanged, so the auto-format push-back below still works.
repository: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name || github.repository }}
token: ${{ secrets.GITHUB_TOKEN }}
# Full history so we can diff HEAD against the current base branch
# tip to scope the formatter to PR-changed files.
fetch-depth: 0
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.x
cache: pnpm
cache-dependency-path: "**/pnpm-lock.yaml"
- name: Install oxfmt
# oxfmt is pinned as a root devDependency and installed from the frozen
# lockfile — no ad-hoc `npm install -g`. `--ignore-scripts` keeps
# install-time scripts from running against the PR-head checkout this job
# uses. Put node_modules/.bin on PATH so the bare `oxfmt` calls below
# (invoked via xargs) resolve the pinned binary.
run: |
pnpm install --frozen-lockfile --ignore-scripts
echo "$(pwd)/node_modules/.bin" >> "$GITHUB_PATH"
- name: Install ruff
# Pin ruff so a compromised or breaking release can't land on the next
# PR run with the persisted-credentials write token in this job. The
# official ruff-action installs the pinned version (via uv) and puts
# `ruff` on PATH for the format steps below; `args: --version` makes the
# action install-only (it defaults to running `ruff check` otherwise).
# Bump the version manually when needed (ruff isn't tracked by Dependabot).
uses: astral-sh/ruff-action@278981a28ce3188b1e39527901f38254bf3aac89 # v4.1.0
with:
version: "0.15.13"
args: "--version"
- name: Collect PR-changed files for formatting
if: github.event_name == 'pull_request'
id: changed
env:
PR_BASE_REF: ${{ github.event.pull_request.base.ref }}
run: |
base_ref="${PR_BASE_REF}"
# Fetch the current tip of the base branch so the merge-base tracks
# main as it advances (using the PR's stored base.sha would pull in
# every file main has touched since the PR opened).
git fetch --no-tags origin "$base_ref"
# Scope to files changed between the current base-branch merge-base
# and HEAD so advances on main don't drag unrelated files into the
# PR. Restrict to oxfmt-supported extensions so oxfmt never errors
# on an unknown target. Canonical list lives upstream in oxfmt
# (https://github.com/oxc-project/oxc-formatter) — update here when
# oxfmt adds a new format.
#
# Exclude lockfiles: they match *.json / *.yaml but oxfmt rejects
# them internally (size threshold or filename heuristic), which
# caused lockfile-only PRs to fail with "Expected at least one
# target file". Lockfiles are auto-generated by npm/pnpm and should
# never be hand-formatted regardless.
git diff --name-only --diff-filter=ACMR "origin/$base_ref"...HEAD -- \
'*.js' '*.jsx' '*.ts' '*.tsx' '*.mjs' '*.cjs' \
'*.json' '*.jsonc' '*.json5' \
'*.md' \
'*.css' '*.yml' '*.yaml' '*.html' '*.vue' '*.py' \
':!**/package-lock.json' ':!**/pnpm-lock.yaml' ':!**/yarn.lock' \
> .pr-format-files.txt
: > .pr-format-files.existing.txt
while IFS= read -r f; do
[ -n "$f" ] && [ -f "$f" ] && printf '%s\n' "$f" >> .pr-format-files.existing.txt
done < .pr-format-files.txt
# Drop tracked-but-gitignored paths (e.g. fixtures under a
# `recorded/` rule). Without this, oxfmt would rewrite them and
# the auto-commit step's `git add` would refuse the ignored
# path, killing the whole step and leaving the PR unfixed.
if [ -s .pr-format-files.existing.txt ]; then
git ls-files -i -c --exclude-standard > .pr-format-files.ignored.txt
grep -vxFf .pr-format-files.ignored.txt .pr-format-files.existing.txt > .pr-format-files.scoped.txt || true
mv .pr-format-files.scoped.txt .pr-format-files.existing.txt
fi
count=$(wc -l < .pr-format-files.existing.txt | tr -d ' ')
echo "count=$count" >> "$GITHUB_OUTPUT"
echo "PR-changed format candidates: $count"
cat .pr-format-files.existing.txt
- name: Run formatter (fix on PR)
run: |
if [ "${{ steps.changed.outputs.count }}" = "0" ]; then
echo "No formattable files changed in this PR — skipping."
exit 0
fi
# oxfmt: auto-fix JS/TS/JSON/MD/CSS/YAML/HTML/Vue
if ! xargs -a .pr-format-files.existing.txt oxfmt --no-error-on-unmatched-pattern --write; then
echo "::warning::oxfmt exited with error — auto-fix may be incomplete"
fi
# ruff: auto-fix Python
py_files=$(grep -E '\.py$' .pr-format-files.existing.txt || true)
if [ -n "$py_files" ]; then
echo "$py_files" | xargs ruff format || echo "::warning::ruff format exited with error"
fi
# Trigger the auto-commit only when one of the SCOPED PR files
# actually changed. A whole-tree `git diff` here also trips on
# unrelated working-tree drift (e.g. an LFS smudge on a tracked
# `*.png filter=lfs` file), which would set format_fixed=true while
# the scoped `git add` below stages nothing — making `git commit`
# fail with "nothing to commit". Diffing only the scoped files keeps
# the trigger aligned with what the commit step can actually stage.
# shellcheck disable=SC2046 # intentional split: each path is a
# separate `git diff` pathspec arg; the `-s` guard rules out the
# empty-arg (whole-tree) case, and PR paths never contain spaces.
if [ -s .pr-format-files.existing.txt ] && \
! git diff --quiet -- $(cat .pr-format-files.existing.txt); then
echo "format_fixed=true" >> "$GITHUB_ENV"
fi
# Check mode: verify everything is formatted
xargs -a .pr-format-files.existing.txt oxfmt --no-error-on-unmatched-pattern --check
if [ -n "$py_files" ]; then
echo "$py_files" | xargs ruff format --check
fi
- name: Configure git for push
if: >-
env.format_fixed == 'true' &&
github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.event.pull_request.base.repo.full_name
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git config --local url."https://x-access-token:${TOKEN}@github.com/".insteadOf "https://github.com/"
env:
TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Commit formatting fixes
if: >-
env.format_fixed == 'true' &&
github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.event.pull_request.base.repo.full_name
run: |
if [ -z "$(git diff --name-only)" ]; then
echo "No formatting changes to commit"
exit 0
fi
# Stage only the files the formatter was scoped to operate on.
# Piping `git diff --name-only` into `git add` is unsafe: if a
# tracked-but-gitignored path shows up in the diff, `git add`
# aborts the whole step and the auto-fix push never lands —
# leaving formatting violations on the PR branch and (post-
# merge) on main.
xargs -a .pr-format-files.existing.txt git add --
# Guard against an empty staged set: if the scoped `git add` staged
# nothing (e.g. the whole-tree drift that set format_fixed=true lives
# entirely outside the scoped files), `git commit` would exit 1 and
# fail the job. Treat an empty index as a no-op instead.
if git diff --cached --quiet; then
echo "No scoped formatting changes to commit"
exit 0
fi
git commit -m "style: auto-fix formatting"
git push
oxlint:
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Use Node.js 20
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.x
# setup-node built-in cache is fork-safe (fork PRs can't write to base repo cache)
cache: "pnpm"
cache-dependency-path: "**/pnpm-lock.yaml"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Run oxlint check
run: pnpm run lint
package-quality:
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Use Node.js 20
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.x
# setup-node built-in cache is fork-safe (fork PRs can't write to base repo cache)
cache: "pnpm"
cache-dependency-path: "**/pnpm-lock.yaml"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Configure Nx Cloud environment
run: |
{
echo "NX_CI_EXECUTION_ID=${{ github.run_id }}-${{ github.run_attempt }}-quality-packages"
echo "NX_CLOUD_NO_TIMEOUTS=true"
echo "NX_CLOUD_DISTRIBUTED_EXECUTION=false"
echo "NX_NO_CLOUD=true"
} >> "$GITHUB_ENV"
- name: Run publint and attw
run: pnpm run check:packages
check-types:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
env:
# tsc on @copilotkit/runtime needs ~10 GB: the AI SDK v6 tool()
# generics explode against zod 3 schemas (~40M type instantiations,
# ~5 min check time). Bounding the worst inline schemas helps but the
# cost is systemic to the ai x zod type interaction, so this job gets
# a 12 GB heap instead of the workflow-level 4 GB default.
NODE_OPTIONS: "--max-old-space-size=12288"
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Use Node.js 20
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.x
# setup-node built-in cache is fork-safe (fork PRs can't write to base repo cache)
cache: "pnpm"
cache-dependency-path: "**/pnpm-lock.yaml"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Configure Nx Cloud environment
run: |
{
echo "NX_CI_EXECUTION_ID=${{ github.run_id }}-${{ github.run_attempt }}-quality-check-types"
echo "NX_CLOUD_NO_TIMEOUTS=true"
echo "NX_CLOUD_DISTRIBUTED_EXECUTION=false"
echo "NX_NO_CLOUD=true"
} >> "$GITHUB_ENV"
- name: Generate GraphQL codegen files
run: npx nx run @copilotkit/runtime-client-gql:graphql-codegen
- name: Run check-types
# Invoke nx directly: `pnpm run check-types -- --parallel=1` makes
# nx forward --parallel=1 to each package's tsc command instead of
# consuming it. --parallel=1 keeps tsc within the runner's 16 GB
# RAM: the @copilotkit/runtime check alone peaks near 10 GB.
run: npx nx run-many -t check-types --parallel=1
commitlint:
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
fetch-depth: 0
persist-credentials: false
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Use Node.js 20
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 20.x
# setup-node built-in cache is fork-safe (fork PRs can't write to base repo cache)
cache: "pnpm"
cache-dependency-path: "**/pnpm-lock.yaml"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Validate current commit (last commit) with commitlint
if: github.event_name == 'push'
run: |
# Skip merge commits. GitHub's "Create a merge commit" option takes
# the message from the PR body, which can contain markdown lists
# that parse as additional (empty) commit subjects and fail
# subject-empty / type-empty — see commit 5ed233f01.
parents=$(git rev-list --parents -n 1 HEAD | awk '{print NF - 1}')
if [ "$parents" -gt 1 ]; then
echo "HEAD is a merge commit ($parents parents) — skipping commitlint."
exit 0
fi
npx commitlint --last --verbose
- name: Validate PR commits with commitlint
id: commitlint
if: github.event_name == 'pull_request'
continue-on-error: true
env:
PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: npx commitlint --from "${PR_BASE_SHA}" --to "${PR_HEAD_SHA}" --verbose 2>&1 | tee /tmp/commitlint-output.txt
- name: Post fix suggestion on failure
if: github.event_name == 'pull_request' && steps.commitlint.outcome == 'failure'
continue-on-error: true
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
const fs = require('fs');
const output = fs.readFileSync('/tmp/commitlint-output.txt', 'utf8');
const body = `### ❌ Commitlint failed\n\nCommit messages must follow [Conventional Commits](https://www.conventionalcommits.org/).\n\n**Valid prefixes:** \`feat:\`, \`fix:\`, \`docs:\`, \`style:\`, \`refactor:\`, \`test:\`, \`chore:\`, \`ci:\`, \`perf:\`, \`build:\`\n\n**Example:** \`feat: add user authentication\`\n\n<details><summary>Full output</summary>\n\n\`\`\`\n${output}\n\`\`\`\n</details>\n\nTo fix, amend your commit messages:\n\`\`\`bash\ngit rebase -i HEAD~N # N = number of commits to fix\n# Change 'pick' to 'reword' for bad commits\n\`\`\``;
// Find existing comment to update
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes('Commitlint failed'));
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body,
});
}
- name: Fail if commitlint failed
if: github.event_name == 'pull_request' && steps.commitlint.outcome == 'failure'
run: exit 1
+340
View File
@@ -0,0 +1,340 @@
name: test / e2e / dojo
on:
push:
branches: [main]
paths:
- "packages/**"
- "sdk-python/**"
- ".github/workflows/test_e2e-dojo.yml"
pull_request:
branches: [main]
paths:
- "packages/**"
- "sdk-python/**"
- ".github/workflows/test_e2e-dojo.yml"
workflow_dispatch:
inputs:
branch:
description: "Branch to run the workflow on"
required: true
default: "main"
type: string
env:
NX_VERBOSE_LOGGING: true
NX_CI_EXECUTION_ID: ${{ github.head_ref }}-${{ github.sha }}-${{ github.run_attempt }}
NX_CI_EXECUTION_ENV: "E2E Dojo"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
detect-changes:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
ts-changed: ${{ steps.changes.outputs.ts }}
python-changed: ${{ steps.changes.outputs.python }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
id: changes
with:
filters: |
ts:
- 'packages/**'
- '.github/workflows/test_e2e-dojo.yml'
python:
- 'sdk-python/**'
dojo:
needs: detect-changes
name: ${{ matrix.suite }}
# 4-vCPU runner (was depot-ubuntu-24.04 = 2 vCPU) to speed the build, the
# dojo prep, and Playwright worker parallelism on the long-pole suites.
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 30
permissions:
contents: read
strategy:
fail-fast: false
matrix:
include:
- suite: a2a-middleware
test_path: tests/a2aMiddlewareTests
services: ["dojo", "a2a-middleware"]
wait_on: http://localhost:9999,tcp:localhost:8011,tcp:localhost:8012,tcp:localhost:8013,tcp:localhost:8014
needs_python: true
- suite: adk-middleware
test_path: tests/adkMiddlewareTests
services: ["dojo", "adk-middleware"]
wait_on: http://localhost:9999,tcp:localhost:8010
needs_python: true
- suite: agno
test_path: tests/agnoTests
services: ["dojo", "agno"]
wait_on: http://localhost:9999,tcp:localhost:8002
needs_python: true
- suite: crew-ai
test_path: tests/crewAITests
services: ["dojo", "crew-ai"]
wait_on: http://localhost:9999,tcp:localhost:8003
needs_python: true
- suite: langgraph-python
test_path: tests/langgraphPythonTests
services: ["dojo", "langgraph-platform-python"]
wait_on: http://localhost:9999,tcp:localhost:8005
needs_python: true
- suite: langgraph-typescript
test_path: tests/langgraphTypescriptTests
services: ["dojo", "langgraph-platform-typescript"]
wait_on: http://localhost:9999,tcp:localhost:8006
needs_python: false
- suite: langgraph-fastapi
test_path: tests/langgraphFastAPITests
services: ["dojo", "langgraph-fastapi"]
wait_on: http://localhost:9999,tcp:localhost:8004
needs_python: true
- suite: llama-index
test_path: tests/llamaIndexTests
services: ["dojo", "llama-index"]
wait_on: http://localhost:9999,tcp:localhost:8007
needs_python: true
- suite: mastra
test_path: tests/mastraTests
services: ["dojo", "mastra"]
wait_on: http://localhost:9999,tcp:localhost:8008
needs_python: false
- suite: mastra-agent-local
test_path: tests/mastraAgentLocalTests
services: ["dojo"]
wait_on: http://localhost:9999
needs_python: false
- suite: middleware-starter
test_path: tests/middlewareStarterTests
services: ["dojo"]
wait_on: http://localhost:9999
needs_python: false
- suite: pydantic-ai
test_path: tests/pydanticAITests
services: ["dojo", "pydantic-ai"]
wait_on: http://localhost:9999,tcp:localhost:8009
needs_python: true
- suite: server-starter
test_path: tests/serverStarterTests
services: ["dojo", "server-starter"]
wait_on: http://localhost:9999,tcp:localhost:8000
needs_python: true
- suite: server-starter-all
test_path: tests/serverStarterAllFeaturesTests
services: ["dojo", "server-starter-all"]
wait_on: http://localhost:9999,tcp:localhost:8001
needs_python: true
- suite: aws-strands
test_path: tests/awsStrandsTests
services: ["dojo", "aws-strands"]
wait_on: http://localhost:9999,tcp:localhost:8017
needs_python: true
steps:
- name: Check relevance
id: should-run
run: |
if [[ "${{ needs.detect-changes.outputs.ts-changed }}" == "true" ]] || \
[[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
echo "skip=false" >> $GITHUB_OUTPUT
elif [[ "${{ needs.detect-changes.outputs.python-changed }}" == "true" ]] && \
[[ "${{ matrix.needs_python }}" == "true" ]]; then
echo "skip=false" >> $GITHUB_OUTPUT
else
echo "skip=true" >> $GITHUB_OUTPUT
echo "⏭️ Skipping ${{ matrix.suite }} — no relevant changes"
fi
- name: Detect fork PR
id: fork-check
env:
EVENT_NAME: ${{ github.event_name }}
PR_HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name }}
REPO_FULL: ${{ github.repository }}
run: |
if [[ "$EVENT_NAME" == "pull_request" && \
"$PR_HEAD_REPO" != "$REPO_FULL" ]]; then
echo "prefix=fork-" >> "$GITHUB_OUTPUT"
else
echo "prefix=" >> "$GITHUB_OUTPUT"
fi
- name: Checkout CPK
if: steps.should-run.outputs.skip != 'true'
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
lfs: true
path: CopilotKit
ref: ${{ github.event.inputs.branch || github.ref }}
persist-credentials: false
- name: Checkout AGUI
if: steps.should-run.outputs.skip != 'true'
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
repository: ag-ui-protocol/ag-ui
path: ag-ui
ref: main
persist-credentials: false
- name: Set up Node.js
if: steps.should-run.outputs.skip != 'true'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22"
- name: Install pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
if: steps.should-run.outputs.skip != 'true'
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
with:
package_json_file: CopilotKit/package.json
# Now that pnpm is available, cache its store to speed installs
- name: Resolve pnpm store path
if: steps.should-run.outputs.skip != 'true'
id: pnpm-store
run: echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_OUTPUT
- name: Cache pnpm store
if: steps.should-run.outputs.skip != 'true'
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ steps.pnpm-store.outputs.STORE_PATH }}
key: ${{ steps.fork-check.outputs.prefix }}${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ steps.fork-check.outputs.prefix }}${{ runner.os }}-pnpm-store-
# Cache Python tool caches and virtualenvs; restore only to avoid long saves
- name: Cache Python dependencies (restore-only)
if: steps.should-run.outputs.skip != 'true' && matrix.needs_python
id: cache-python
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
~/.cache/pip
~/.cache/pypoetry
~/.cache/uv
**/.venv
key: ${{ steps.fork-check.outputs.prefix }}${{ runner.os }}-pydeps-${{ hashFiles('**/poetry.lock', '**/pyproject.toml') }}
restore-keys: |
${{ steps.fork-check.outputs.prefix }}${{ runner.os }}-pydeps-
- name: Install Poetry
if: steps.should-run.outputs.skip != 'true' && matrix.needs_python
uses: snok/install-poetry@a783c322200f0519c7926aa6faa857c4e23e9263 # v1.4.2
with:
version: latest
virtualenvs-create: true
virtualenvs-in-project: true
- name: Install uv
if: steps.should-run.outputs.skip != 'true' && matrix.needs_python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
- name: Install cpk dependencies
if: steps.should-run.outputs.skip != 'true'
working-directory: CopilotKit
run: pnpm install --frozen-lockfile
- name: Configure Nx Cloud environment
if: steps.should-run.outputs.skip != 'true'
run: |
echo "NX_CI_EXECUTION_ID=${{ github.run_id }}-${{ github.run_attempt }}-e2e-dojo-${{ matrix.suite }}" >> $GITHUB_ENV
echo "NX_CLOUD_NO_TIMEOUTS=true" >> $GITHUB_ENV
echo "NX_CLOUD_DISTRIBUTED_EXECUTION=false" >> $GITHUB_ENV
echo "NX_NO_CLOUD=true" >> $GITHUB_ENV
- name: Build cpk packages
if: steps.should-run.outputs.skip != 'true'
working-directory: CopilotKit
env:
NODE_OPTIONS: --max-old-space-size=8192
# Match the 4-vCPU runner so the monorepo build uses the extra cores.
NX_PARALLEL: 4
run: pnpm build
- name: Install ag-ui dependencies
if: steps.should-run.outputs.skip != 'true'
working-directory: ag-ui
run: pnpm install --frozen-lockfile
- name: Prepare dojo for e2e
working-directory: ag-ui/apps/dojo
env:
NODE_OPTIONS: --max-old-space-size=8192
if: steps.should-run.outputs.skip != 'true' && join(matrix.services, ',') != ''
run: node ./scripts/prep-dojo-everything.js --only ${{ join(matrix.services, ',') }}
- name: Link cpk into ag-ui
if: steps.should-run.outputs.skip != 'true'
working-directory: CopilotKit
run: node ../ag-ui/apps/dojo/scripts/link-cpk.js ${{ github.workspace }}/CopilotKit/packages
- name: Install e2e dependencies
if: steps.should-run.outputs.skip != 'true'
working-directory: ag-ui/apps/dojo/e2e
run: |
pnpm install
- name: write langgraph env files
working-directory: ag-ui/integrations/langgraph
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
LANGSMITH_API_KEY: ${{ secrets.LANGSMITH_API_KEY }}
if: steps.should-run.outputs.skip != 'true' && (contains(join(matrix.services, ','), 'langgraph-fastapi') || contains(join(matrix.services, ','), 'langgraph-platform-python') || contains(join(matrix.services, ','), 'langgraph-platform-typescript'))
run: |
echo "OPENAI_API_KEY=${OPENAI_API_KEY}" > python/examples/.env
echo "LANGSMITH_API_KEY=${LANGSMITH_API_KEY}" >> python/examples/.env
echo "OPENAI_API_KEY=${OPENAI_API_KEY}" > typescript/examples/.env
echo "LANGSMITH_API_KEY=${LANGSMITH_API_KEY}" >> typescript/examples/.env
- name: Run dojo+agents
uses: JarvusInnovations/background-action@2428e7b970a846423095c79d43f759abf979a635 # v1
env:
NODE_OPTIONS: --max-old-space-size=8192
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
LANGSMITH_API_KEY: ${{ secrets.LANGSMITH_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_GEMINI_API_KEY }}
if: steps.should-run.outputs.skip != 'true' && join(matrix.services, ',') != '' && contains(join(matrix.services, ','), 'dojo')
with:
run: |
node ../scripts/run-dojo-everything.js --only ${{ join(matrix.services, ',') }}
working-directory: ag-ui/apps/dojo/e2e
wait-on: ${{ matrix.wait_on }}
wait-for: 300000
- name: Run tests ${{ matrix.suite }}
if: steps.should-run.outputs.skip != 'true'
working-directory: ag-ui/apps/dojo/e2e
env:
NODE_OPTIONS: --max-old-space-size=8192
BASE_URL: http://localhost:9999
PLAYWRIGHT_SUITE: ${{ matrix.suite }}
run: |
pnpm test -- ${{ matrix.test_path }}
- name: Upload traces ${{ matrix.suite }}
if: always() && steps.should-run.outputs.skip != 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ matrix.suite }}-playwright-traces
path: |
ag-ui/apps/dojo/e2e/test-results/${{ matrix.suite }}/**/*
ag-ui/apps/dojo/e2e/playwright-report/**/*
retention-days: 7
+139
View File
@@ -0,0 +1,139 @@
name: test / e2e / legacy-v1
on:
push:
branches: [main]
paths:
- "examples/**"
- ".github/workflows/test_e2e-legacy-v1.yml"
pull_request:
branches: [main]
paths:
- "examples/**"
- ".github/workflows/test_e2e-legacy-v1.yml"
workflow_dispatch:
inputs:
branch:
description: "Branch to run the workflow on"
required: true
default: "main"
type: string
env:
NX_VERBOSE_LOGGING: true
NX_CI_EXECUTION_ID: ${{ github.head_ref }}-${{ github.sha }}-${{ github.run_attempt }}
NX_CI_EXECUTION_ENV: "E2E Examples"
# Least-privilege by default. Individual jobs/steps can widen when needed.
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
examples:
name: ${{ matrix.example }}
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 20
permissions:
contents: read
# id-token: write is required for Depot OIDC auth (runs-on: depot-ubuntu-*).
id-token: write
strategy:
fail-fast: false
matrix:
example:
- form-filling
- travel
- research-canvas
- chat-with-your-data
- state-machine
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
lfs: true
ref: ${{ github.event.inputs.branch || github.ref }}
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22"
- name: Install pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Detect fork PR
id: fork-check
env:
EVENT_NAME: ${{ github.event_name }}
PR_HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name }}
REPO_FULL: ${{ github.repository }}
run: |
if [[ "$EVENT_NAME" == "pull_request" && \
"$PR_HEAD_REPO" != "$REPO_FULL" ]]; then
echo "prefix=fork-" >> "$GITHUB_OUTPUT"
else
echo "prefix=" >> "$GITHUB_OUTPUT"
fi
- name: Resolve pnpm store path
id: pnpm-store
run: echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_OUTPUT
- name: Cache pnpm store
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ steps.pnpm-store.outputs.STORE_PATH }}
key: ${{ steps.fork-check.outputs.prefix }}${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ steps.fork-check.outputs.prefix }}${{ runner.os }}-pnpm-store-
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Configure Nx Cloud environment
run: |
echo "NX_CI_EXECUTION_ID=${{ github.run_id }}-${{ github.run_attempt }}-e2e-examples-${{ matrix.example }}" >> $GITHUB_ENV
echo "NX_CLOUD_NO_TIMEOUTS=true" >> $GITHUB_ENV
echo "NX_CLOUD_DISTRIBUTED_EXECUTION=false" >> $GITHUB_ENV
echo "NX_NO_CLOUD=true" >> $GITHUB_ENV
- name: Build CopilotKit
id: build-cpk
run: pnpm build
- name: Install e2e dependencies
working-directory: examples/e2e
run: pnpm install --frozen-lockfile
- name: Install Playwright browsers
working-directory: examples/e2e
run: pnpm exec playwright install --with-deps chromium
- name: Run e2e tests
working-directory: examples/e2e
env:
EXAMPLE: ${{ matrix.example }}
CI: "true"
OPENAI_API_KEY: test
NEXT_PUBLIC_CPK_PUBLIC_API_KEY: ""
NEXT_PUBLIC_COPILOT_PUBLIC_API_KEY: ""
LANGSMITH_API_KEY: ""
run: pnpm test
- name: Upload Playwright artifacts
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: examples-e2e-${{ matrix.example }}
path: |
examples/e2e/test-results/**/*
examples/e2e/playwright-report/**/*
retention-days: 7
@@ -0,0 +1,555 @@
name: "test / e2e / showcase / on-demand"
# SECURITY — residual trust model (read before editing):
#
# This workflow EXISTS to execute PR-HEAD code (Playwright tests, Next.js dev
# server, Python agent, pip install of PR-controlled requirements.txt). Several
# hardening layers reduce blast radius:
# - `author_association` gate limits the `issue_comment` trigger to OWNER /
# MEMBER / COLLABORATOR (third-party commenters cannot spawn runs).
# - workflow-level `permissions: contents: read` means the heavy test job's
# GITHUB_TOKEN cannot mutate the repo; the `post-result` job gets write
# perms scoped to just the final PR comment.
# - `persist-credentials: false` on `actions/checkout` prevents the token
# from being left behind in `.git/config` where PR-HEAD build hooks might
# read it.
# - `pnpm install --ignore-scripts` / `npm install --ignore-scripts` block
# install-time hooks in PR-controlled JS manifests from executing on the
# runner. The Python install uses `pip install --prefer-binary` (prefers
# wheels, falls back to sdist on transitive deps that lack a wheel for
# linux-x86_64/py3.12). We used to use `--only-binary :all:` for a hard
# block against source-build hooks, but CrewAI's transitive graph
# (tiktoken / chromadb / litellm cadence releases) regularly ships a
# sdist-only revision that makes every CI run fail-loud with "Could not
# find a version that satisfies the requirement". `--prefer-binary` trades
# that hard guarantee for reliability — the `author_association` gate
# above still limits WHO can trigger this workflow, so the residual risk
# is bounded to a trusted commenter. See also the "Start Python agent"
# step for the in-context trade-off rationale.
# - A strict slug whitelist (`^[a-z0-9-]+$` + existing-dir check) and the
# `env:`-based pattern for UNTRUSTED values (comment body, dispatch slug)
# prevent shell injection / path traversal.
#
# What this is NOT: a security boundary against a malicious trusted commenter.
# The last line of defense is the SOCIAL CONTRACT that a trusted commenter
# reviews the PR diff BEFORE typing `/test-aimock` — if a compromised / rogue
# OWNER/MEMBER/COLLABORATOR comments on an attacker's PR, they get a full
# runner exec with the job's token. That is an accepted residual risk for the
# developer-velocity benefit of PR-triggered E2E runs. Do not loosen the
# `author_association` gate without revisiting the threat model above.
#
# Known TOCTOU — comment-trigger vs resolved HEAD SHA:
# "Resolve PR HEAD ref" below calls `pulls.get` at job start. There is a
# window between the trusted commenter typing `/test-aimock` (reviewed diff
# D1) and the workflow actually calling `pulls.get` (resolves whatever HEAD
# is current — possibly D2 after a force-push). A PR author who force-pushes
# malicious content AFTER the trusted comment but BEFORE the resolve call
# gets their code executed. GitHub Actions does NOT natively support pinning
# the SHA at comment time (no `comment.commit_sha` equivalent), so this gap
# is architectural. The `author_association` gate + code-review social
# contract are the mitigations; the residual TOCTOU risk is accepted. If
# GitHub ever ships a comment-time SHA field, pin to it and drop this note.
on:
issue_comment:
types: [created]
workflow_dispatch:
inputs:
slug:
description: "Package slug to test (Python integration with aimock support)"
required: true
# Only Python integrations that exercise the AIMOCK_URL path
# end-to-end belong here. Restricting the enum prevents accidental
# dispatch of a TS-only (mastra) or Java (spring-ai) slug that would
# skip the Python agent startup step and then fail with a misleading
# Playwright timeout. When a new Python slug is validated, append it.
#
# No `default:` is set — the operator must pick a slug explicitly. A
# hidden default would silently bind manual dispatches to whichever
# slug happens to be first in the enum, which contradicts the
# "no silent fallback" guarantee the comment-path extractor enforces.
type: choice
options:
- crewai-crews
- langgraph-python
# Default to read-only at the job level. The only step that needs write access
# is "Post result to PR" at the end — we grant it write perms inline there.
# Keeping the workflow-level perms read-only means every intermediate step
# (including `pip install` on attacker-controlled requirements.txt) runs with
# a token that cannot mutate the repo.
permissions:
contents: read
jobs:
aimock-e2e:
# Only run on PR comments matching `/test-aimock ` (trailing space REQUIRED)
# from trusted authors, or manual dispatch. The trailing space tightens
# the match so unrelated text like `/test-aimocker` or `don't /test-aimock-like-this`
# does NOT trigger the workflow. The author_association gate additionally
# prevents arbitrary third-party commenters from triggering runs with
# attacker-controlled comment bodies (which the 'Determine slug' step then
# parses — see env-based shell interpolation below). A bare `/test-aimock`
# alone (no trailing space) is rejected by design; commenters must pick a
# slug explicitly — no silent fallback to crewai-crews (see "Determine slug"
# step below).
# `startsWith` (not `contains`) is the Actions-level gate: it requires
# `/test-aimock ` to be the FIRST token of the comment, so embedded mentions
# (in code blocks, quoted replies, or mid-sentence prose) cannot spin up a
# runner. The shell extractor in the "Determine slug" step uses the same
# leading anchor (`^/test-aimock[[:space:]]+…`) as defense-in-depth; both
# layers agree on "first token only" so a future edit that loosens either
# layer alone cannot bypass validation. Commenters who
# want to add narration around the command should put the command on its
# own line at the top of the comment.
if: >
github.event_name == 'workflow_dispatch' ||
(github.event.issue.pull_request
&& startsWith(github.event.comment.body, '/test-aimock ')
&& contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association))
# Pinned to ubuntu-latest deliberately: the 'Determine slug' step uses
# POSIX-only `grep -oE` + `sed` (no `grep -oP` / PCRE) so a future BSD
# grep would still work, but ubuntu-latest keeps the install/setup matrix
# consistent with every other showcase workflow.
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
# For issue_comment events, we need to resolve the PR HEAD SHA ourselves
# because the event payload doesn't include pull_request.head.sha
- name: Resolve PR HEAD ref
id: pr-ref
if: github.event_name == 'issue_comment'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
const { data: pr } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.issue.number,
});
// Refuse to run against a closed / merged PR. A trusted commenter
// typing `/test-aimock` on a stale closed PR would otherwise
// re-exec the old HEAD — either wasting CI or (if the PR was
// closed BECAUSE it was bad) re-running known-bad code. Fail loud.
if (pr.state !== 'open') {
core.setFailed(`PR #${pr.number} is ${pr.state} (not open). Refusing to run E2E on a non-open PR.`);
return;
}
core.setOutput('ref', pr.head.sha);
core.setOutput('pr_number', pr.number);
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
ref: ${{ steps.pr-ref.outputs.ref || github.sha }}
# Do NOT leave the workflow's GITHUB_TOKEN in `.git/config` after
# checkout. PR-HEAD code (pip build hooks, Next.js dev scripts,
# Playwright fixtures) runs on this runner; a credential left in the
# working tree could be read by that code and exfiltrated. The job's
# `permissions: contents: read` limits blast radius, but defense-in-
# depth cheap — disable credential persistence.
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22.x
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Determine slug
id: slug
# SECURITY: comment body and dispatch slug are UNTRUSTED. Pass via env
# (NOT via `${{ ... }}` expression interpolation) so shell never parses
# attacker-controlled text. Then validate against a strict whitelist
# before anything downstream uses $SLUG as a path / package name — so
# `../../../etc/shadow` or similar cannot reach `cd`/`pip install`.
env:
EVENT_NAME: ${{ github.event_name }}
COMMENT_BODY: ${{ github.event.comment.body }}
DISPATCH_SLUG: ${{ github.event.inputs.slug }}
run: |
set -euo pipefail
if [ "$EVENT_NAME" = "workflow_dispatch" ]; then
SLUG="$DISPATCH_SLUG"
else
# POSIX-safe extraction (no `grep -oP` / PCRE `\K`): match
# `/test-aimock` ONLY at the start of the comment body, followed
# by whitespace + a slug. The leading anchor (^) matches exactly
# what the job-level `if:` gate enforces via
# `startsWith(github.event.comment.body, '/test-aimock ')` — both
# layers agree that the command must be the FIRST token of the
# body, so an edit that loosens either layer cannot accidentally
# desynchronize from the other. This blocks
# `/test-aimocker` or mid-line mentions from matching.
# Works on both GNU grep (ubuntu-latest) and BSD grep.
SLUG=$(printf '%s' "$COMMENT_BODY" \
| grep -oE '^/test-aimock[[:space:]]+[^[:space:]]+' \
| head -n1 \
| sed 's|^/test-aimock[[:space:]]*||' \
|| true)
# No default slug fallback. A bare `/test-aimock` (no slug) or a
# match that only skimmed our boundary (e.g. `/test-aimocker x`)
# FAILS the workflow rather than silently running against
# crewai-crews. A hidden default is a footgun: a trusted commenter
# typing `don't /test-aimock-like-this` would otherwise spawn a
# full CI run against the wrong package.
if [ -z "$SLUG" ]; then
echo "::error::No slug provided. Usage: '/test-aimock <slug>' (e.g. '/test-aimock crewai-crews')"
exit 1
fi
fi
# Strict slug whitelist: lowercase alphanumerics + hyphens only. This
# blocks path traversal (`../`), absolute paths, command substitution,
# and anything else that could escape `showcase/integrations/$SLUG`.
case "$SLUG" in
''|*[!a-z0-9-]*)
echo "::error::Invalid slug '$SLUG' — must match ^[a-z0-9-]+$"
exit 1
;;
esac
# Belt-and-suspenders: the slug must correspond to an existing package
# directory. Rejects typos and anything that bypasses the regex.
if [ ! -d "showcase/integrations/$SLUG" ]; then
echo "::error::Slug '$SLUG' does not map to showcase/integrations/$SLUG"
exit 1
fi
echo "slug=$SLUG" >> "$GITHUB_OUTPUT"
# NOTE on `${{ steps.slug.outputs.slug }}` vs `env:` pattern:
# Downstream steps interpolate `steps.slug.outputs.slug` directly into
# the shell script body. This is SAFE here because the "Determine slug"
# step above whitelists the value against `^[a-z0-9-]+$` AND rejects any
# slug that doesn't map to an existing package directory — so the value
# that reaches these interpolations is always a trusted, validated
# identifier. We still use the `env:`-based defensive default for
# downstream script bodies that handle anything else UNTRUSTED (see the
# `actions/github-script` step at the bottom of the workflow).
- name: Detect package type
id: pkg-type
run: |
SLUG="${{ steps.slug.outputs.slug }}"
PKG_DIR="showcase/integrations/$SLUG"
if [ -f "$PKG_DIR/requirements.txt" ] || [ -f "$PKG_DIR/pyproject.toml" ]; then
echo "has_python=true" >> "$GITHUB_OUTPUT"
else
echo "has_python=false" >> "$GITHUB_OUTPUT"
fi
# Detect agent server type: langgraph (langgraph_cli dev on :8123)
# vs uvicorn (agent_server:app on :8000). The two use different
# start commands, ports, and health endpoints.
if [ -f "$PKG_DIR/langgraph.json" ]; then
echo "agent_type=langgraph" >> "$GITHUB_OUTPUT"
echo "agent_port=8123" >> "$GITHUB_OUTPUT"
echo "agent_health_path=/ok" >> "$GITHUB_OUTPUT"
else
echo "agent_type=uvicorn" >> "$GITHUB_OUTPUT"
echo "agent_port=8000" >> "$GITHUB_OUTPUT"
echo "agent_health_path=/health" >> "$GITHUB_OUTPUT"
fi
# aimock_toggle.py ships in crewai-crews and wires AIMOCK_URL
# end-to-end via configure_aimock(). Packages without it (e.g.
# langgraph-python) can still use aimock — the workflow injects
# OPENAI_BASE_URL directly on the agent process. Log the status
# but do not block; the toggle is a nice-to-have, not a gate.
if [ -f "$PKG_DIR/src/aimock_toggle.py" ]; then
echo "ships_toggle=true" >> "$GITHUB_OUTPUT"
echo "::notice::Slug '$SLUG' ships aimock_toggle.py — aimock redirect handled by configure_aimock()"
else
echo "ships_toggle=false" >> "$GITHUB_OUTPUT"
echo "::notice::Slug '$SLUG' does not ship aimock_toggle.py — aimock redirect will be injected via OPENAI_BASE_URL env var"
fi
# Still require Python — this workflow cannot exercise TS-only or
# Java slugs (no Python agent to start).
if [ ! -f "$PKG_DIR/requirements.txt" ] && [ ! -f "$PKG_DIR/pyproject.toml" ]; then
echo "::error::Slug '$SLUG' has no requirements.txt or pyproject.toml — this workflow only exercises Python-backed packages."
exit 1
fi
- name: Install aimock
run: |
# aimock is pinned as a workspace dependency (@copilotkit/showcase-scripts)
# and installed from the frozen lockfile — no ad-hoc `npm install -g`.
# A frozen install guarantees the exact pinned version resolves (the old
# caret floor could drift to a bad publish); this keeps the CI signal
# reproducible AND satisfies zizmor's adhoc-packages audit.
#
# `--ignore-scripts`: a trusted commenter can run this workflow on a PR
# whose package.json is untrusted content, so we never execute install-time
# scripts. aimock's `llmock` bin runs fine without them.
#
# `--filter` scopes the install to just the aimock owner package so we
# don't pay for the full monorepo install here (the per-slug package deps
# are installed later in "Install package dependencies").
pnpm --filter @copilotkit/showcase-scripts install --frozen-lockfile --ignore-scripts
- name: Start aimock
run: |
# Invoke the workspace-installed `llmock` bin directly from the repo root.
# `llmock` is aimock's fixtures-based CLI (the package also ships an
# `aimock` bin, which is the newer config-only CLI that does NOT accept
# --fixtures). Running from the repo root keeps the root-relative
# --fixtures paths correct (a `pnpm --filter exec` would run inside
# showcase/scripts and break them).
AIMOCK_BIN="./showcase/scripts/node_modules/.bin/llmock"
if [ ! -x "$AIMOCK_BIN" ]; then
echo "::error::aimock binary not found at $AIMOCK_BIN after workspace install"
exit 1
fi
# Fixture layout matches docker-compose.local.yml: feature-parity.json
# was split into per-framework shared/d4/d5-recorded/d6 directories
# (directory-based loading, one --fixtures per directory).
"$AIMOCK_BIN" --port 4010 --host 127.0.0.1 \
--fixtures showcase/aimock/shared \
--fixtures showcase/aimock/d4 \
--fixtures showcase/aimock/d5-recorded \
--fixtures showcase/aimock/d6 \
--validate-on-load &
AIMOCK_PID=$!
echo "AIMOCK_PID=$AIMOCK_PID" >> "$GITHUB_ENV"
# Wait for aimock to be ready. Capture the PID + `kill -0` inside
# the loop so an aimock that crashes on startup (bad fixture path,
# port in use, binary import error) fails fast instead of burning
# the full 20s polling a dead process.
#
# Probe `/__aimock/health` — aimock's actual readiness endpoint.
# Root `/` returns HTTP 404 (aimock serves `/__aimock/*` and `/v1/*`
# only), and `curl -sf` treats 404 as failure, so probing `/` would
# loop until the budget expired and then hard-fail every run.
#
# `--max-time 2 --connect-timeout 1` caps each probe so a hung
# socket cannot blow the loop's 20-iteration budget.
for i in $(seq 1 20); do
if ! kill -0 "$AIMOCK_PID" 2>/dev/null; then
echo "::error::aimock process (PID $AIMOCK_PID) exited before becoming ready — check the preceding aimock stdout/stderr."
exit 1
fi
curl -sf --max-time 2 --connect-timeout 1 http://localhost:4010/__aimock/health > /dev/null 2>&1 && break
sleep 1
done
curl -sf --max-time 2 --connect-timeout 1 http://localhost:4010/__aimock/health || { echo "aimock failed to start"; exit 1; }
- name: Setup Python agent
if: steps.pkg-type.outputs.has_python == 'true'
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with:
python-version: "3.12"
# Cache pip to avoid reinstalling CrewAI's heavy transitive dep
# tree on every PR run. Key scopes to the selected slug so each
# package gets its own cache bucket keyed on its requirements.txt.
cache: "pip"
cache-dependency-path: showcase/integrations/${{ steps.slug.outputs.slug }}/requirements.txt
- name: Start Python agent
if: steps.pkg-type.outputs.has_python == 'true'
run: |
SLUG="${{ steps.slug.outputs.slug }}"
AGENT_TYPE="${{ steps.pkg-type.outputs.agent_type }}"
AGENT_PORT="${{ steps.pkg-type.outputs.agent_port }}"
AGENT_HEALTH="${{ steps.pkg-type.outputs.agent_health_path }}"
cd "showcase/integrations/$SLUG"
# SECURITY / RELIABILITY trade-off: `pip install` runs setup.py /
# PEP 517 build hooks from PR-controlled packages. Unlike npm / pnpm
# there is no `--ignore-scripts` flag for pip; the closest equivalent
# is `--only-binary :all:` (wheel-only, blocks source-build hooks).
#
# We previously used `--only-binary :all:` but CrewAI's dependency
# graph (tiktoken / chromadb / litellm etc.) regularly ships a
# sdist-only revision of a transitive dep. That made every CI run
# fail with "Could not find a version that satisfies the requirement"
# — not a security win but a CI outage. `--prefer-binary` keeps the
# wheel-first preference (most installs remain hook-free) and only
# falls back to sdist when a wheel isn't published for
# linux-x86_64/py3.12. The `author_association` gate at the job
# level still restricts WHO can trigger this workflow, so the
# residual source-build-hook risk is bounded to a trusted commenter.
pip install --prefer-binary -r requirements.txt
if [ "$AGENT_TYPE" = "langgraph" ]; then
# langgraph-python: start via langgraph_cli dev on port 8123.
# Uses langgraph.json for graph configuration. The /ok endpoint
# is the readiness probe. Inject OPENAI_BASE_URL + dummy key
# directly since langgraph-python does not ship aimock_toggle.py.
if [ ! -f "langgraph.json" ]; then
echo "::error::Slug '$SLUG' detected as langgraph but langgraph.json is missing."
exit 1
fi
OPENAI_BASE_URL=http://localhost:4010/v1 \
OPENAI_API_KEY=sk-aimock-dev-ci-only \
python -u -m langgraph_cli dev \
--config langgraph.json \
--host 127.0.0.1 \
--port "$AGENT_PORT" \
--no-browser &
else
# uvicorn-based agent (crewai-crews): start via agent_server:app
# on port 8000. Packages that ship aimock_toggle.py wire
# OPENAI_BASE_URL internally — set AIMOCK_URL only so the toggle
# itself is exercised end-to-end.
if [ ! -f "src/agent_server.py" ]; then
echo "::error::Slug '$SLUG' is missing src/agent_server.py — uvicorn agent type requires the FastAPI entrypoint."
exit 1
fi
export PYTHONPATH="$PWD/src:${PYTHONPATH:-}"
AIMOCK_URL=http://localhost:4010/v1 \
python -m uvicorn "agent_server:app" --host 127.0.0.1 --port "$AGENT_PORT" &
fi
# Wait for agent to be ready. Cold imports (litellm + crew graph
# or langgraph compile) can exceed 60s on a cold runner, so give
# it 90s (45 iterations x 2s). Mirrors the aimock start pattern:
# loop + hard-fail so a cryptic Playwright timeout doesn't mask a
# bind/startup failure.
for i in $(seq 1 45); do
curl -sf --max-time 2 --connect-timeout 1 "http://localhost:${AGENT_PORT}${AGENT_HEALTH}" > /dev/null 2>&1 && break
sleep 2
done
curl -sf --max-time 2 --connect-timeout 1 "http://localhost:${AGENT_PORT}${AGENT_HEALTH}" > /dev/null 2>&1 \
|| { echo "Python agent failed to start on :${AGENT_PORT}"; exit 1; }
- name: Install package dependencies
run: |
SLUG="${{ steps.slug.outputs.slug }}"
cd "showcase/integrations/$SLUG"
# `--ignore-scripts`: a trusted commenter can run `/test-aimock` on
# a PR whose package.json is untrusted content. Without this flag
# an attacker's postinstall script would execute on the runner with
# the workflow's token. The E2E path (Playwright + Next.js dev) does
# not require install-time scripts to succeed.
pnpm install --ignore-scripts
- name: Start dev server
run: |
SLUG="${{ steps.slug.outputs.slug }}"
AGENT_TYPE="${{ steps.pkg-type.outputs.agent_type }}"
AGENT_PORT="${{ steps.pkg-type.outputs.agent_port }}"
cd "showcase/integrations/$SLUG"
# Invoke `next dev` directly instead of `pnpm dev` — the package's
# `pnpm dev` script spawns a SECOND agent process via concurrently,
# but the previous "Start Python agent" step already bound the agent
# port. A second bind would fail with EADDRINUSE. Running Next
# directly also keeps the aimock env flow clean.
#
# `OPENAI_BASE_URL` + `OPENAI_API_KEY` on Next are DEFENSIVE ONLY.
# Next proxies chat traffic to the Python agent via the CopilotKit
# runtime — it does not call OpenAI directly. Setting these prevents
# accidental real-API fallback if a future route adds a direct call.
#
# Agent URL wiring differs by agent type:
# - uvicorn (crewai-crews): AGENT_URL=http://localhost:8000
# - langgraph: LANGGRAPH_DEPLOYMENT_URL=http://localhost:8123
# (langgraph-python's Next routes read this env var, defaulting
# to localhost:8123 if unset — but we set it explicitly for clarity)
# Export the correct agent URL env var for Next.js to read.
if [ "$AGENT_TYPE" = "langgraph" ]; then
export LANGGRAPH_DEPLOYMENT_URL="http://localhost:${AGENT_PORT}"
else
export AGENT_URL="http://localhost:${AGENT_PORT}"
fi
export OPENAI_BASE_URL=http://localhost:4010/v1
export OPENAI_API_KEY=sk-aimock-dev-ci-only
npx next dev --turbopack &
# Wait for dev server. `--max-time 2 --connect-timeout 1` caps each
# probe so a hung socket can't blow the loop budget.
for i in $(seq 1 30); do
curl -sf --max-time 2 --connect-timeout 1 http://localhost:3000 > /dev/null 2>&1 && break
sleep 2
done
curl -sf --max-time 2 --connect-timeout 1 http://localhost:3000 || { echo "Dev server failed to start"; exit 1; }
- name: Install Playwright
run: |
cd "showcase/integrations/${{ steps.slug.outputs.slug }}"
npx playwright install chromium --with-deps
- name: Re-probe aimock liveness
# aimock was readiness-checked once right after startup, but several
# steps (Python agent start, pnpm install, Next dev startup, Playwright
# install) may have run for multiple minutes since. If aimock died
# during any of that time, Playwright would silently run against real
# OpenAI because OPENAI_BASE_URL=http://localhost:4010/v1 still points
# at the (now dead) port — curl would refuse the connection, litellm
# would fall through to the default OpenAI endpoint, and the test
# would pass/fail on REAL traffic with REAL costs. Fail loud before
# Playwright runs.
run: |
if ! curl -sf --max-time 2 --connect-timeout 1 http://localhost:4010/__aimock/health > /dev/null 2>&1; then
echo "::error::aimock is no longer responding on :4010. Refusing to run Playwright against a dead aimock (would silently hit real OpenAI)."
exit 1
fi
- name: Run Playwright tests
run: |
SLUG="${{ steps.slug.outputs.slug }}"
cd "showcase/integrations/$SLUG"
BASE_URL=http://localhost:3000 npx playwright test --reporter=list
env:
CI: "true"
# Dead env — Next.js is already running from the "Start dev server"
# step above (which set these inline on that process). Env set here
# would only affect the `npx playwright test` process, which does not
# read OPENAI_BASE_URL / OPENAI_API_KEY. Leaving unset to avoid the
# false impression that these values flow to the running Next server.
- name: Re-check aimock liveness after Playwright
if: always()
# Defense-in-depth: aimock might have OOM'd DURING the Playwright run.
# If that happened, the test either silently used stale fixtures (no-op
# after aimock died if responses were cached) or fell through to real
# OpenAI. Fail the job loudly so a dead aimock cannot masquerade as a
# green run. Keeps the 4010-is-still-alive invariant symmetric with the
# pre-Playwright re-probe above.
run: |
if [ -n "${AIMOCK_PID:-}" ] && ! kill -0 "$AIMOCK_PID" 2>/dev/null; then
echo "::error::aimock process (PID $AIMOCK_PID) died during the Playwright run. Playwright results are untrusted — it may have hit real OpenAI or returned stale fixtures."
exit 1
fi
- name: Upload test artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-report-${{ steps.slug.outputs.slug }}
path: showcase/integrations/${{ steps.slug.outputs.slug }}/playwright-report/
retention-days: 7
if-no-files-found: ignore
outputs:
slug: ${{ steps.slug.outputs.slug }}
# Post the final status as a PR comment. Separated into its own job so
# the write perms (pull-requests + issues) are scoped to JUST this job —
# the heavy test job above runs with `contents: read` only, so a compromised
# transitive dep in `pip install` on a PR-controlled requirements.txt
# cannot mutate PRs / issues with the workflow's token.
post-result:
needs: aimock-e2e
if: github.event_name == 'issue_comment' && always() && needs.aimock-e2e.result != 'skipped'
runs-on: ubuntu-latest
timeout-minutes: 2
permissions:
pull-requests: write
issues: write
steps:
- name: Post result to PR
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
# Pass dynamic values through env (NOT `${{ ... }}` interpolation into
# the script body). Even though the slug is whitelisted upstream, the
# env-var pattern is the defensive default: any future additions that
# aren't pre-validated cannot accidentally reach script text.
env:
SLUG: ${{ needs.aimock-e2e.outputs.slug }}
JOB_STATUS: ${{ needs.aimock-e2e.result }}
with:
script: |
const slug = process.env.SLUG || '(unknown)';
const jobStatus = process.env.JOB_STATUS;
const status = jobStatus === 'success' ? '✅' : '❌';
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: `${status} **Aimock E2E Tests** (\`${slug}\`): ${jobStatus}\n\n[View run](${runUrl})`
});
@@ -0,0 +1,86 @@
name: test / integration / docs
on:
pull_request:
branches: [main]
paths:
- "showcase/shell-docs/src/content/**"
- "showcase/shell-docs/model-allowlist.json"
- "scripts/validate-doc-model-names.ts"
- "scripts/doc-tests/**"
- ".github/workflows/test_integration-docs.yml"
push:
branches: [main]
paths:
- "showcase/shell-docs/src/content/**"
- "showcase/shell-docs/model-allowlist.json"
- "scripts/validate-doc-model-names.ts"
- "scripts/doc-tests/**"
- ".github/workflows/test_integration-docs.yml"
# Least-privilege by default. Individual jobs/steps can widen when needed.
permissions:
contents: read
jobs:
validate-model-names:
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 15
permissions:
contents: read
# id-token: write is required for Depot OIDC auth (runs-on: depot-ubuntu-*).
id-token: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm tsx scripts/validate-doc-model-names.ts
doc-tests:
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 15
needs: validate-model-names
permissions:
contents: read
# id-token: write is required for Depot OIDC auth (runs-on: depot-ubuntu-*).
id-token: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22
cache: pnpm
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with:
python-version: "3.12"
- run: pnpm install --frozen-lockfile
- name: Start aimock
run: |
# aimock is pinned as a workspace dependency (@copilotkit/showcase-scripts)
# and installed from the frozen lockfile above — no ad-hoc `npm install -g`.
# The `llmock` bin is aimock's fixtures-based CLI (the package also ships an
# `aimock` bin, which is the newer config-only CLI that does NOT accept
# --fixtures). Invoke the workspace-installed bin directly from the repo
# root so the root-relative --fixtures path resolves correctly (a
# `pnpm --filter exec` would run inside showcase/scripts and break the path).
nohup ./showcase/scripts/node_modules/.bin/llmock --fixtures scripts/doc-tests/fixtures --validate-on-load > /tmp/aimock.log 2>&1 &
for i in $(seq 1 60); do
if curl -sf http://localhost:4010/health; then
echo "aimock ready"
exit 0
fi
sleep 1
done
echo "aimock failed to start. Logs:"
cat /tmp/aimock.log
exit 1
- run: pnpm tsx scripts/doc-tests/extract.ts
- run: pnpm tsx scripts/doc-tests/run.ts
@@ -0,0 +1,118 @@
name: test / integration / runtime
on:
push:
branches: [main]
paths:
- "packages/runtime/**"
- ".github/workflows/test_integration-runtime.yml"
pull_request:
branches: [main]
paths:
- "packages/runtime/**"
- ".github/workflows/test_integration-runtime.yml"
workflow_dispatch:
inputs:
branch:
description: "Branch to run the workflow on"
required: true
default: "main"
type: string
# Least-privilege by default. Individual jobs/steps can widen when needed.
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
node:
name: "runtime / node"
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 15
permissions:
contents: read
# id-token: write is required for Depot OIDC auth (runs-on: depot-ubuntu-*).
id-token: write
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
ref: ${{ github.event.inputs.branch || github.ref }}
persist-credentials: false
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Use Node.js 22.x
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22.x"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Configure Nx Cloud environment
run: |
echo "NX_CI_EXECUTION_ID=${{ github.run_id }}-${{ github.run_attempt }}-integration-node" >> $GITHUB_ENV
echo "NX_CLOUD_NO_TIMEOUTS=true" >> $GITHUB_ENV
echo "NX_CLOUD_DISTRIBUTED_EXECUTION=false" >> $GITHUB_ENV
echo "NX_NO_CLOUD=true" >> $GITHUB_ENV
- name: Build runtime
run: pnpm nx run @copilotkit/runtime:build
- name: Run Node.js server integration tests
working-directory: packages/runtime
run: pnpm exec vitest run src/v2/runtime/__tests__/integration/node-servers.integration.test.ts
bun:
name: "runtime / bun"
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 15
permissions:
contents: read
# id-token: write is required for Depot OIDC auth (runs-on: depot-ubuntu-*).
id-token: write
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
ref: ${{ github.event.inputs.branch || github.ref }}
persist-credentials: false
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Use Node.js 22.x
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22.x"
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: latest
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Configure Nx Cloud environment
run: |
echo "NX_CI_EXECUTION_ID=${{ github.run_id }}-${{ github.run_attempt }}-integration-bun" >> $GITHUB_ENV
echo "NX_CLOUD_NO_TIMEOUTS=true" >> $GITHUB_ENV
echo "NX_CLOUD_DISTRIBUTED_EXECUTION=false" >> $GITHUB_ENV
echo "NX_NO_CLOUD=true" >> $GITHUB_ENV
- name: Build runtime
run: pnpm nx run @copilotkit/runtime:build
- name: Run Bun server integration tests
working-directory: packages/runtime
run: bun test src/v2/runtime/__tests__/integration/bun/bun-servers.integration.test.ts
+195
View File
@@ -0,0 +1,195 @@
name: test / smoke / starter
# PR build-sanity gate (model B, §e / Phase 5) PLUS the post-merge 6h
# floating-dependency breakage detector. The eventual live dashboard signal
# for starter health comes from the harness HTTP-probing the deployed
# (sleepable) Railway starter services — but those S5 Railway services do
# NOT exist yet, so this `schedule` cron remains the ONLY post-merge
# detector that catches a starter broken by a floating transitive dependency
# or an upstream CopilotKit release. We KEEP the 6h cron (and the harness
# alert path, alerts/alert-engine.ts → #oss-alerts, carries the
# scheduled-failure Slack signal too) until S5 starter-service probing is
# confirmed live, at which point the cron can be retired in favour of the
# harness signal. The fast pre-merge build-sanity gate is the
# `pull_request` trigger on examples/integrations/**: it builds + smoke-
# tests each starter offline against aimock on every starter change. The
# post-merge GHCR `:latest` publish that feeds the Railway services lives in
# showcase_build.yml's build-starters job (push:main, §b stage 1), so it is
# intentionally NOT duplicated here.
on:
schedule:
# Every 6h — post-merge floating-dependency / upstream-release breakage
# detector. Stays until S5 harness probing of live Railway starter
# services is confirmed, then retire in favour of the harness signal.
- cron: "0 */6 * * *"
workflow_run:
workflows: ["publish / release"]
types: [completed]
pull_request:
paths:
- "examples/integrations/**"
- ".github/workflows/test_smoke-starter.yml"
workflow_dispatch: {}
permissions:
contents: read
packages: read
jobs:
smoke-starter:
runs-on: ubuntu-latest
timeout-minutes: 20
env:
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_OSS_ALERTS }}
strategy:
matrix:
starter:
- langgraph-python
- mastra
- langgraph-js
- crewai-crews
- pydantic-ai
- adk
- agno
- llamaindex
- langgraph-fastapi
- strands-python
- ms-agent-framework-python
- ms-agent-framework-dotnet
fail-fast: false
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
fetch-depth: 1
lfs: false
persist-credentials: false
- name: Run starter smoke tests
working-directory: examples/integrations/${{ matrix.starter }}
env:
STARTER: ${{ matrix.starter }}
# Starters whose SDK can't be intercepted by aimock (currently
# google-adk — google-genai ignores endpoint overrides) need a
# real provider key. docker-compose.test.yml for those starters
# reads this via `${GOOGLE_API_KEY:-test-key-for-aimock}` so
# unaffected starters still run offline against aimock.
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
run: |
docker compose -f docker-compose.test.yml up --abort-on-container-exit --exit-code-from tests
- name: Capture failure cause
if: failure()
id: failure-cause
working-directory: examples/integrations/${{ matrix.starter }}
env:
EVENT_NAME: ${{ github.event_name }}
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_USER_LOGIN: ${{ github.event.pull_request.user.login }}
PR_TITLE: ${{ github.event.pull_request.title }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
{
echo "summary<<CAUSE_EOF"
# Capture error from container logs (containers still running at this point)
build_err=$(docker compose -f docker-compose.test.yml logs app 2>&1 | grep -E "ERR_|Error:|error:|Build error|failed to" | head -3)
agent_err=$(docker compose -f docker-compose.test.yml logs agent 2>&1 | grep -E "Error:|Traceback|ModuleNotFoundError|ImportError" | head -3)
test_err=$(docker compose -f docker-compose.test.yml logs tests 2>&1 | grep -E "✘|FAIL|Error:|expect\(" | head -5)
if [ -n "$build_err" ]; then
echo "Build failure:"
echo "$build_err"
elif [ -n "$agent_err" ]; then
echo "Agent crash:"
echo "$agent_err"
elif [ -n "$test_err" ]; then
echo "Test failure:"
echo "$test_err"
else
echo "Unknown failure — check run logs"
fi
echo ""
# Identify recent changes that may have caused the failure
if [ "$EVENT_NAME" = "pull_request" ]; then
echo "Triggered by PR #${PR_NUMBER} (${PR_USER_LOGIN}): ${PR_TITLE}"
echo "Head: ${PR_HEAD_SHA}"
elif [ "$EVENT_NAME" = "schedule" ] || [ "$EVENT_NAME" = "workflow_run" ]; then
# Use GitHub API instead of git log (avoids needing deep clone)
echo "Recent commits touching this starter (last 12h):"
gh api "repos/${{ github.repository }}/commits?path=examples/integrations/${{ matrix.starter }}&since=$(date -u -d '12 hours ago' +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -v-12H +%Y-%m-%dT%H:%M:%SZ)&per_page=5" \
--jq '.[] | "\(.sha[0:7]) \(.commit.message | split("\n")[0])"' 2>/dev/null || true
starter_commits=$(gh api "repos/${{ github.repository }}/commits?path=examples/integrations/${{ matrix.starter }}&since=$(date -u -d '12 hours ago' +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -v-12H +%Y-%m-%dT%H:%M:%SZ)&per_page=1" --jq 'length' 2>/dev/null || echo "0")
if [ "$starter_commits" = "0" ]; then
echo "No starter code changes — likely a floating dependency update or upstream CopilotKit release"
echo "Recent releases:"
gh api "repos/${{ github.repository }}/releases?per_page=3" \
--jq '.[] | "\(.tag_name) (\(.published_at[0:10]))"' 2>/dev/null | head -3 || true
fi
fi
echo "CAUSE_EOF"
} >> "$GITHUB_OUTPUT"
- name: Tear down
if: always()
working-directory: examples/integrations/${{ matrix.starter }}
run: docker compose -f docker-compose.test.yml down -v
- name: Upload test artifacts on failure
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: smoke-starter-${{ matrix.starter }}
path: showcase/tests/test-results/
retention-days: 7
# Sanitize the failure summary and stash it in $GITHUB_ENV so the
# inline Slack payload can reference it via `env.summary`. We don't
# write to a payload file because slackapi/slack-github-action@v2.1.0
# rejects payload files that don't end in `.json`/`.yaml`/`.yml`
# (mktemp produces extensionless files), and `jq --rawfile` can also
# crash under `set -e` on edge-case inputs, leaving an empty/missing
# payload and silently suppressing the alert. Inline `payload:` with
# `toJSON(format(...))` sidesteps both failure modes — quotes,
# backslashes, newlines in the summary are safely JSON-encoded by
# toJSON, and there's no intermediate file to mishandle. This is the
# same pattern used in test_smoke-starter-deployed.yml (PR #4068) and
# showcase_validate.yml.
- name: Prepare Slack alert fields
if: failure() && env.SLACK_WEBHOOK != '' && (github.event_name == 'schedule' || github.event_name == 'workflow_run')
env:
SUMMARY_RAW: ${{ steps.failure-cause.outputs.summary }}
run: |
# Strip ANSI sequences (SGR, OSC, and G0/G1 charset designators), then truncate
# to 200 bytes and drop any trailing partial UTF-8 bytes so we don't emit mojibake.
SUMMARY=$(printf '%s' "$SUMMARY_RAW" | head -3 \
| sed -E 's/\x1b\[[0-9;?]*[A-Za-z]//g; s/\x1b\][^\x07]*\x07//g; s/\x1b[()][A-Za-z0-9]//g' \
| head -c 200 | iconv -f UTF-8 -t UTF-8//IGNORE)
if [ -z "$SUMMARY" ]; then
SUMMARY="(no failure detail captured — see job log)"
fi
# Emit via heredoc so embedded `=`, quotes, or newlines don't
# break KEY=VALUE parsing in $GITHUB_ENV.
{
echo "summary<<EOF_SUMMARY_7a4c"
printf '%s\n' "$SUMMARY"
echo "EOF_SUMMARY_7a4c"
} >> "$GITHUB_ENV"
- name: Alert Slack on failure
if: failure() && env.SLACK_WEBHOOK != '' && (github.event_name == 'schedule' || github.event_name == 'workflow_run')
uses: slackapi/slack-github-action@0d95c9a7becc1e6e297d76df9bc735c44f4cbcbc # v3.0.5
with:
webhook: ${{ secrets.SLACK_WEBHOOK_OSS_ALERTS }}
webhook-type: incoming-webhook
# NOTE: `\n` is NOT an escape sequence in GitHub Actions expression
# string literals — `format()` would emit the two literal characters
# backslash+n, which `toJSON` then encodes as `\\n`, so Slack renders
# a literal "\n" instead of a line break. Inject real newlines via
# `fromJSON('"\n"')` ({6}) so `toJSON` encodes them as a single `\n`
# that Slack honors. The code-fence delimiters sit on their own lines
# so the summary renders as a proper code block.
payload: |
{ "text": ${{ toJSON(format(':x: `[ci:{2}]` *Starter smoke test failing: {0}*{6}<{1}/{2}/actions/runs/{3}|View run> · <{1}/{2}/actions/runs/{3}/job/{4}|View job>{6}```{6}{5}{6}```', matrix.starter, github.server_url, github.repository, github.run_id, github.job, env.summary, fromJSON('"\n"'))) }} }
@@ -0,0 +1,58 @@
name: test / unit / python-sdk
on:
push:
branches: [main]
paths:
- "sdk-python/**"
- ".github/workflows/test_unit-python-sdk.yml"
pull_request:
branches: [main]
paths:
- "sdk-python/**"
- ".github/workflows/test_unit-python-sdk.yml"
# Least-privilege by default. Individual jobs/steps can widen when needed.
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 10
permissions:
contents: read
# id-token: write is required for Depot OIDC auth (runs-on: depot-ubuntu-*).
id-token: write
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with:
python-version: ${{ matrix.python-version }}
- name: Install Poetry
uses: snok/install-poetry@a783c322200f0519c7926aa6faa857c4e23e9263 # v1.4.2
with:
version: latest
virtualenvs-create: true
virtualenvs-in-project: true
- name: Install dependencies
working-directory: sdk-python
run: poetry lock && poetry install --with dev
- name: Run tests
working-directory: sdk-python
run: poetry run python -m pytest tests/ -v
+127
View File
@@ -0,0 +1,127 @@
name: test / unit / spring-ai
# Unit tests for the spring-ai showcase integration.
#
# The Dockerfile uses `-DskipTests` and intentionally omits `src/test/`
# (production image stays lean), so wire-shape contracts authored in
# `src/test/java` would otherwise never execute in CI. This workflow gives
# them a home: gated on the same `spring_ai` paths-filter as the rest of the
# showcase pipeline, runs `mvn -pl spring-ai test` after building the AG-UI
# community artifacts from source (same install dance the Dockerfile does).
#
# The critical contract under test is
# RunErrorEventWireShapeTest — a @SpringBootTest that asserts the application
# ObjectMapper actually applies JacksonConfig's customizer (catches the
# AG-UI AgUiAutoConfiguration bare-ObjectMapper bean-race that previously
# made the wire-shape rename inert in production).
on:
pull_request:
paths:
- "showcase/integrations/spring-ai/**"
- ".github/workflows/test_unit-spring-ai.yml"
push:
branches: [main]
paths:
- "showcase/integrations/spring-ai/**"
- ".github/workflows/test_unit-spring-ai.yml"
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
spring-ai-unit:
name: "spring-ai unit"
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false
- name: Set up JDK 21 (matches Dockerfile)
uses: actions/setup-java@0f481fcb613427c0f801b606911222b5b6f3083a # v5.5.0
with:
distribution: temurin
java-version: "21"
# NOTE: do NOT use `cache: maven` here — setup-java's implicit cache
# keys on hashFiles('**/pom.xml') only, which is BLIND to AG_UI_SHA
# bumps. The explicit actions/cache step below keys on AG_UI_SHA so
# the cache invalidates whenever the pinned SHA changes.
- name: Load AG_UI_SHA from .ag-ui-sha
# Single source of truth for the AG-UI commit shared with the
# Dockerfile (`showcase/integrations/spring-ai/.ag-ui-sha`). Keeps CI
# and the production image building against the same snapshot.
run: |
set -euo pipefail
AG_UI_SHA="$(tr -d '[:space:]' < showcase/integrations/spring-ai/.ag-ui-sha)"
if [ -z "${AG_UI_SHA}" ]; then
echo "::error::.ag-ui-sha is empty"; exit 1
fi
echo "AG_UI_SHA=${AG_UI_SHA}" >> "${GITHUB_ENV}"
- name: Cache Maven repository (keyed on AG_UI_SHA)
# Explicit cache keyed on AG_UI_SHA so SHA bumps invalidate the cache
# — otherwise CI would silently reuse stale AG-UI community artifacts
# installed under a prior SHA (R3-A1).
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-agui-${{ env.AG_UI_SHA }}-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-agui-${{ env.AG_UI_SHA }}-
${{ runner.os }}-maven-
- name: Install AG-UI Java SDK from source
# The spring-ai showcase depends on AG-UI community Maven artifacts
# (com.ag-ui.community:spring-ai / java-server / spring) that aren't
# on Maven Central yet. The Dockerfile installs them by cloning the
# upstream repo and running `mvn install`; do the same here so the
# showcase's own `mvn test` can resolve them from the local repo.
env:
GIT_LFS_SKIP_SMUDGE: "1"
run: |
set -euo pipefail
# Belt-and-suspenders: evict any stale AG-UI artifacts that a
# restore-keys cache hit might have repopulated. The cache key
# above pins to AG_UI_SHA, but a `restore-keys` fallback can
# still surface older AG-UI jars on a fresh SHA. Force a clean
# install from this SHA.
rm -rf \
~/.m2/repository/com/ag-ui \
~/.m2/repository/io/github/ag-ui-protocol
rm -rf /tmp/ag-ui
# `git clone` only fetches the default branch by default — if
# AG_UI_SHA points at a commit on a PR branch or older history
# not reachable from HEAD, the subsequent `checkout` fails with
# a cryptic "reference is not a tree". Explicitly fetch the SHA
# first so checkout is always against a known-local commit
# (R7-A2). `--depth 1` keeps the cache lean.
git clone --no-checkout https://github.com/ag-ui-protocol/ag-ui.git /tmp/ag-ui
git -C /tmp/ag-ui fetch --depth 1 origin "${AG_UI_SHA}"
git -C /tmp/ag-ui checkout "${AG_UI_SHA}"
cd /tmp/ag-ui/sdks/community/java
mvn install \
-pl servers/spring,integrations/spring-ai -am \
-DskipTests -Dgpg.skip=true \
-Dmaven.javadoc.skip=true -Djavadoc.skip=true \
-Dmaven.source.skip=true -Dcheckstyle.skip=true \
-Dmaven.site.skip=true -Dreporting.skip=true \
-Dassembly.skipAssembly=true \
-B
- name: Run unit tests
working-directory: showcase/integrations/spring-ai
# Belt-and-suspenders: @TestPropertySource in the suite supplies a
# placeholder key, but some Spring auto-configuration paths read the
# env var directly on context init. A literal non-secret string keeps
# CI deterministic without leaking real credentials.
env:
OPENAI_API_KEY: test-key-not-used
run: mvn -B test
+181
View File
@@ -0,0 +1,181 @@
name: test / unit
on:
push:
branches: [main]
paths-ignore:
- "README.md"
- "examples/**"
- "showcase/**"
- "sdk-python/**"
pull_request:
branches: [main]
paths-ignore:
- "README.md"
- "examples/**"
- "showcase/**"
- "sdk-python/**"
workflow_dispatch:
inputs:
branch:
description: "Branch to run the workflow on"
required: true
default: "main"
type: string
env:
NODE_OPTIONS: "--max-old-space-size=4096"
NX_VERBOSE_LOGGING: true
NX_CI_EXECUTION_ID: ${{ github.head_ref }}-${{ github.sha }}-${{ github.run_attempt }}
NX_CI_EXECUTION_ENV: "Unit Tests"
# Least-privilege by default. Individual jobs/steps can widen when needed.
# id-token: write is required for Depot OIDC auth (runs-on: depot-ubuntu-*).
permissions:
contents: read
id-token: write
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
unit:
name: "unit"
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
node-version: [20.x, 22.x, 24.x]
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
ref: ${{ github.event.inputs.branch || github.ref }}
persist-credentials: false
# Full history so `nx affected` can diff HEAD against the PR base /
# the previous push, instead of rebuilding+retesting every package
# on every run. A shallow clone has no merge-base to diff against.
fetch-depth: 0
- name: Setup pnpm
# Omit `version:` so pnpm/action-setup inherits from the repo's
# `packageManager` field in package.json (via corepack).
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: ${{ matrix.node-version }}
# Do NOT use cache: "pnpm" here — its key omits the Node.js version,
# so a better-sqlite3 binary compiled for one Node ABI (e.g. ABI 137
# from Node 24) would be served to a job running a different ABI
# (Node 20 = ABI 115, Node 22 = ABI 127), causing "Module did not
# self-register". We handle pnpm caching manually below with the
# node-version in the key.
# Fork-safety note: actions/cache is equally fork-safe — GitHub
# prevents fork PRs from writing to the base repo's cache at the platform level.
- name: Get pnpm store directory
id: pnpm-cache
run: echo "store-path=$(pnpm store path --silent)" >> $GITHUB_OUTPUT
- name: Cache pnpm store (scoped to Node.js version)
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ steps.pnpm-cache.outputs.store-path }}
key: ${{ runner.os }}-pnpm-store-${{ matrix.node-version }}-${{ hashFiles('pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-${{ matrix.node-version }}-
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Configure Nx Cloud environment
run: |
echo "NX_CI_EXECUTION_ID=${{ github.run_id }}-${{ github.run_attempt }}-unit-v1-${{ matrix.node-version }}" >> $GITHUB_ENV
echo "NX_CLOUD_NO_TIMEOUTS=true" >> $GITHUB_ENV
echo "NX_CLOUD_DISTRIBUTED_EXECUTION=false" >> $GITHUB_ENV
echo "NX_NO_CLOUD=true" >> $GITHUB_ENV
echo "NX_TUI=false" >> $GITHUB_ENV
- name: Determine affected range
# Pass GitHub context through env (not inline ${{ }} in the script) to
# avoid template-injection — base_ref is attacker-influenceable.
env:
EVENT_NAME: ${{ github.event_name }}
BASE_REF: ${{ github.base_ref }}
BEFORE_SHA: ${{ github.event.before }}
run: |
if [ "$EVENT_NAME" = "pull_request" ]; then
# Diff against the merge-base with the (current tip of the) base
# branch so advances on main don't drag unrelated packages in.
git fetch --no-tags origin "$BASE_REF"
BASE=$(git merge-base FETCH_HEAD HEAD)
elif [ "$EVENT_NAME" = "push" ]; then
# BEFORE_SHA (github.event.before) is the previous tip of this branch.
BASE="$BEFORE_SHA"
if [ -z "$BASE" ] \
|| [ "$BASE" = "0000000000000000000000000000000000000000" ] \
|| ! git cat-file -e "${BASE}^{commit}" 2>/dev/null; then
# First push / force-push / unknown parent → previous commit.
BASE=$(git rev-parse HEAD~1 2>/dev/null || git rev-parse HEAD)
fi
fi
echo "NX_BASE=${BASE}" >> "$GITHUB_ENV"
echo "NX_HEAD=$(git rev-parse HEAD)" >> "$GITHUB_ENV"
echo "Affected range: ${BASE:-<full>}...$(git rev-parse HEAD)"
- name: Generate GraphQL codegen files
run: npx nx run @copilotkit/runtime-client-gql:graphql-codegen
- name: Select test projects
id: select
# PR/push → only packages affected since the base. workflow_dispatch
# (manual / nightly-style full run) → every package with tests.
# `--projects` scopes to packages/** in `nx show projects` (it does NOT
# in the `nx affected` run form, which also pulls in downstream
# examples/storybook — hence the show-projects → run-many split).
env:
EVENT_NAME: ${{ github.event_name }}
# The workflow sets NX_VERBOSE_LOGGING=true, which makes `nx show
# projects` print "[isolated-plugin] spawned worker…" to stdout and
# corrupt the --json payload we parse below. Force it off here.
NX_VERBOSE_LOGGING: "false"
run: |
# Editing this workflow can't surface as an "affected" nx package, so
# `nx affected` would select nothing and the build/test path would go
# unexercised on the very PR that changes it. Force a full run when
# this file itself changed in the range, same as a manual dispatch.
FULL=false
if [ "$EVENT_NAME" = "workflow_dispatch" ]; then
FULL=true
elif git diff --name-only "$NX_BASE" "$NX_HEAD" \
| grep -qx '.github/workflows/test_unit.yml'; then
FULL=true
echo "test_unit.yml changed in range → running ALL packages."
fi
if [ "$FULL" = "true" ]; then
PROJECTS=$(npx nx show projects --projects='packages/**' --exclude=@copilotkit/demo-agents -t test --json)
else
PROJECTS=$(npx nx show projects --affected --base="$NX_BASE" --head="$NX_HEAD" --projects='packages/**' --exclude=@copilotkit/demo-agents -t test --json)
fi
LIST=$(printf '%s' "$PROJECTS" | node -e "let d='';process.stdin.on('data',c=>d+=c).on('end',()=>process.stdout.write(JSON.parse(d).join(',')))")
echo "projects=$LIST" >> "$GITHUB_OUTPUT"
if [ -n "$LIST" ]; then echo "has=true" >> "$GITHUB_OUTPUT"; else echo "has=false" >> "$GITHUB_OUTPUT"; fi
echo "Selected projects: ${LIST:-<none>}"
- name: Build and test affected packages
if: steps.select.outputs.has == 'true'
# run-many builds each selected package's upstream deps via `^build`,
# so unchanged dependencies are still compiled when something needs them.
env:
PROJECTS: ${{ steps.select.outputs.projects }}
run: npx nx run-many -t build,test --projects="$PROJECTS" --exclude=@copilotkit/demo-agents
- name: No affected packages
if: steps.select.outputs.has != 'true'
run: echo "No package code affected since the base — skipping build & test."
- name: Run release script tests
run: npx vitest run --config scripts/release/vitest.config.mts
+43
View File
@@ -0,0 +1,43 @@
name: Update PR branch
on:
pull_request_target:
types: [labeled]
permissions:
contents: write
pull-requests: write
jobs:
update-branch:
if: github.event.label.name == 'qa:update-branch'
runs-on: ubuntu-latest
steps:
- name: Update PR branch with base
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
with:
script: |
const { owner, repo } = context.repo;
const pull_number = context.payload.pull_request.number;
try {
await github.rest.pulls.updateBranch({ owner, repo, pull_number });
core.info(`Updated branch for PR #${pull_number}`);
} catch (error) {
if (error.status === 422) {
core.info(`Branch already up to date or cannot be updated: ${error.message}`);
} else {
core.setFailed(`Failed to update branch: ${error.message}`);
}
} finally {
try {
await github.rest.issues.removeLabel({
owner,
repo,
issue_number: pull_number,
name: 'qa:update-branch',
});
} catch (error) {
core.warning(`Could not remove label: ${error.message}`);
}
}
+79
View File
@@ -0,0 +1,79 @@
# zizmor configuration — static analysis for GitHub Actions workflows.
#
# Docs: https://docs.zizmor.sh/configuration/
#
# This file lives at `.github/zizmor.yml` because zizmor auto-discovers
# it there. The companion workflow at `.github/workflows/security_zizmor.yml`
# runs zizmor in CI and fails the build on findings at the configured level.
#
# When a finding genuinely cannot be remediated (e.g. the action behavior
# is correct but tripped a check), add it under `rules:` with a clear
# justification comment — never blanket-suppress.
rules:
# `template-injection` flags ${{ }} expansions that put untrusted input
# (issue titles, PR titles, branch names, comment bodies) directly into
# `run:` scripts. The fix is always to route through `env:` and reference
# the env var inside the shell. We treat any new occurrence as a blocker.
template-injection:
ignore: []
# `dangerous-triggers` flags `pull_request_target` and similar triggers
# that run with secrets against fork-controlled refs. Anything new must
# be reviewed by a maintainer.
#
# Suppressions below were each reviewed by a maintainer; new additions
# must come with a comment explaining why the workflow is safe.
dangerous-triggers:
ignore:
# update-branch.yml: pull_request_target gated on `types: [labeled]`
# with a maintainer-applied "update-branch" label. The workflow uses
# the base-repo checkout (not the PR head), so no fork-controlled
# code runs with the elevated token. This is the documented safe
# pattern for label-gated automation.
- update-branch.yml
# showcase_deploy.yml / showcase_capture-previews.yml: workflow_run
# triggered by the in-repo "Showcase: Build & Push" workflow. Only
# main-branch builds fire workflow_run, and the downstream workflows
# never check out PR-head code — they checkout the same trusted
# main-branch ref or use the head_branch of the triggering run
# (which is also main per the upstream filter).
- showcase_deploy.yml
- showcase_capture-previews.yml
# test_smoke-starter.yml: workflow_run triggered by the in-repo
# "publish / release" workflow. Listens to release completions; no
# fork-controlled refs reach this workflow.
- test_smoke-starter.yml
# `unpinned-uses` is satisfied because every `uses:` is SHA-pinned and
# Renovate keeps them current (see renovate.json → local>CopilotKit/renovate).
# `excessive-permissions` flags jobs that grant token scopes they don't
# need. We surface but don't auto-suppress — fixing requires per-job
# `permissions:` blocks that match the actual API calls in the job.
excessive-permissions:
ignore: []
# `artipacked` flags `actions/checkout` calls that leave the default
# GITHUB_TOKEN persisted in the local git config. Every other checkout
# in the repo sets `persist-credentials: false`. The suppressions below
# are the workflows that legitimately need the persisted token to push
# back to the repo (release tagging, docs sync, PR comment from
# pkg-pr-new). Each call site carries a `persist-credentials required:`
# comment immediately above it.
#
# static_quality.yml was removed: its format job now uses
# persist-credentials: false and injects credentials via insteadOf
# only before the push step.
artipacked:
ignore:
# Release workflows that push tags / branches via GITHUB_TOKEN.
# publish-release.yml is intentionally NOT suppressed: its build job
# now uses persist-credentials: false (the artifact it uploads would
# otherwise leak a workflow-scoped token through .git/config) and the
# publish job downloads the artifact and sets up its own git auth via
# `git config insteadOf`.
- prerelease.yml
# pkg-pr-new posts snapshot comments on the PR using the repo token
# — credentials must remain persisted for the action to authenticate.
- publish-commit.yml