Files
simstudioai--sim/helm/sim/tests/copilot-secret-modes_test.yaml
wehub-resource-sync d25d482dc2
Publish CLI Package / publish-npm (push) Waiting to run
Publish Python SDK / publish-pypi (push) Waiting to run
Publish TypeScript SDK / publish-npm (push) Waiting to run
CI / Migrate Dev DB (push) Has been skipped
CI / Detect Version (push) Has been cancelled
CI / Migrate DB (push) Has been cancelled
CI / Build Dev ECR (./docker/app.Dockerfile, ECR_APP) (push) Has been cancelled
CI / Build Dev ECR (./docker/db.Dockerfile, ECR_MIGRATIONS) (push) Has been cancelled
CI / Build Dev ECR (./docker/pii.Dockerfile, ECR_PII) (push) Has been cancelled
CI / Build Dev ECR (./docker/realtime.Dockerfile, ECR_REALTIME) (push) Has been cancelled
CI / Deploy Trigger.dev (Dev) (push) Has been cancelled
CI / Build AMD64 (./docker/app.Dockerfile, ECR_APP, ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Build AMD64 (./docker/db.Dockerfile, ECR_MIGRATIONS, ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Build AMD64 (./docker/pii.Dockerfile, ECR_PII, ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Build AMD64 (./docker/realtime.Dockerfile, ECR_REALTIME, ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/app.Dockerfile, ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/db.Dockerfile, ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/pii.Dockerfile, ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (./docker/realtime.Dockerfile, ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Check Docs Changes (push) Has been cancelled
CI / Process Docs (push) Has been cancelled
CI / Create GitHub Release (push) Has been cancelled
CI / Test and Build (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 13:20:55 +08:00

350 lines
16 KiB
YAML

suite: copilot secret modes — inline / existingSecret / ESO routing (best-practices fix regression net)
release:
name: t
namespace: sim
tests:
- it: checksum/secret changes when an ESO remoteRefs.copilot mapping changes, forcing a rollout (Cursor round 4 finding)
template: deployment-copilot.yaml
documentSelector:
path: kind
value: Deployment
set:
app.env.BETTER_AUTH_SECRET: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.ENCRYPTION_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.INTERNAL_API_SECRET: x
app.env.CRON_SECRET: x
postgresql.auth.password: xxxxxxxx
copilot.enabled: true
copilot.postgresql.enabled: false
copilot.database.url: "postgresql://x:x@x:5432/x"
externalSecrets.enabled: true
externalSecrets.secretStoreRef.name: sim-store
externalSecrets.remoteRefs.app.BETTER_AUTH_SECRET: path/to/auth
externalSecrets.remoteRefs.app.ENCRYPTION_KEY: path/to/enc
externalSecrets.remoteRefs.app.INTERNAL_API_SECRET: path/to/iapi
externalSecrets.remoteRefs.app.CRON_SECRET: path/to/cron
externalSecrets.remoteRefs.postgresql.password: path/to/pgpw
externalSecrets.remoteRefs.copilot.AGENT_API_DB_ENCRYPTION_KEY: path/to/agentdbkey
externalSecrets.remoteRefs.copilot.INTERNAL_API_SECRET: path/to/iapi
externalSecrets.remoteRefs.copilot.LICENSE_KEY: path/to/license-v1
externalSecrets.remoteRefs.copilot.SIM_BASE_URL: path/to/baseurl
externalSecrets.remoteRefs.copilot.SIM_AGENT_API_KEY: path/to/agentkey
externalSecrets.remoteRefs.copilot.REDIS_URL: path/to/redis
externalSecrets.remoteRefs.copilot.OPENAI_API_KEY_1: path/to/openai
asserts:
- isNotEmpty:
path: spec.template.metadata.annotations["checksum/secret"]
- matchRegex:
path: spec.template.metadata.annotations["checksum/secret"]
pattern: "^[a-f0-9]{64}$"
- it: ESO mode with only the required copilot secrets mapped renders cleanly (envDefaults regression net — Greptile round 1 finding)
template: deployment-copilot.yaml
set:
app.env.BETTER_AUTH_SECRET: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.ENCRYPTION_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.INTERNAL_API_SECRET: x
app.env.CRON_SECRET: x
postgresql.auth.password: xxxxxxxx
copilot.enabled: true
copilot.postgresql.enabled: false
copilot.database.url: "postgresql://x:x@x:5432/x"
externalSecrets.enabled: true
externalSecrets.secretStoreRef.name: sim-store
externalSecrets.remoteRefs.app.BETTER_AUTH_SECRET: path/to/auth
externalSecrets.remoteRefs.app.ENCRYPTION_KEY: path/to/enc
externalSecrets.remoteRefs.app.INTERNAL_API_SECRET: path/to/iapi
externalSecrets.remoteRefs.app.CRON_SECRET: path/to/cron
externalSecrets.remoteRefs.postgresql.password: path/to/pgpw
# Only the genuinely-secret keys are mapped — PORT/SERVICE_NAME/ENVIRONMENT/LOG_LEVEL
# live in copilot.server.envDefaults now, not copilot.server.env, so they must NOT
# need a remoteRefs.copilot entry.
externalSecrets.remoteRefs.copilot.AGENT_API_DB_ENCRYPTION_KEY: path/to/agentdbkey
externalSecrets.remoteRefs.copilot.INTERNAL_API_SECRET: path/to/iapi
externalSecrets.remoteRefs.copilot.LICENSE_KEY: path/to/license
externalSecrets.remoteRefs.copilot.SIM_BASE_URL: path/to/baseurl
externalSecrets.remoteRefs.copilot.SIM_AGENT_API_KEY: path/to/agentkey
externalSecrets.remoteRefs.copilot.REDIS_URL: path/to/redis
externalSecrets.remoteRefs.copilot.OPENAI_API_KEY_1: path/to/openai
documentSelector:
path: kind
value: Deployment
asserts:
- isKind: { of: Deployment }
- contains:
path: spec.template.spec.containers[0].env
content:
name: PORT
value: "8080"
- contains:
path: spec.template.spec.containers[0].env
content:
name: SERVICE_NAME
value: copilot
- it: inline mode renders the chart-managed copilot Secret
template: secrets-copilot.yaml
set:
app.env.BETTER_AUTH_SECRET: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.ENCRYPTION_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.INTERNAL_API_SECRET: x
app.env.CRON_SECRET: x
postgresql.auth.password: xxxxxxxx
copilot.enabled: true
copilot.postgresql.enabled: true
copilot.postgresql.auth.password: xxxxxxxx
copilot.server.env.AGENT_API_DB_ENCRYPTION_KEY: x
copilot.server.env.INTERNAL_API_SECRET: x
copilot.server.env.LICENSE_KEY: x
copilot.server.env.SIM_BASE_URL: x
copilot.server.env.SIM_AGENT_API_KEY: x
copilot.server.env.REDIS_URL: x
copilot.server.env.OPENAI_API_KEY_1: x
asserts:
- hasDocuments: { count: 1 }
- isKind: { of: Secret }
- equal: { path: metadata.name, value: t-sim-copilot-env }
- it: existingSecret mode does not shadow the pre-created Secret's values with envDefaults (Greptile + Cursor round 2 finding)
template: deployment-copilot.yaml
documentSelector:
path: kind
value: Deployment
set:
app.env.BETTER_AUTH_SECRET: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.ENCRYPTION_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.INTERNAL_API_SECRET: x
app.env.CRON_SECRET: x
postgresql.auth.password: xxxxxxxx
copilot.enabled: true
copilot.postgresql.enabled: false
copilot.database.url: "postgresql://x:x@x:5432/x"
copilot.server.secret.create: false
copilot.server.secret.name: my-existing-copilot-secret
asserts:
- notExists:
path: spec.template.spec.containers[0].env
- it: existingSecret mode still renders extraEnv even though envDefaults are skipped
template: deployment-copilot.yaml
documentSelector:
path: kind
value: Deployment
set:
app.env.BETTER_AUTH_SECRET: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.ENCRYPTION_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.INTERNAL_API_SECRET: x
app.env.CRON_SECRET: x
postgresql.auth.password: xxxxxxxx
copilot.enabled: true
copilot.postgresql.enabled: false
copilot.database.url: "postgresql://x:x@x:5432/x"
copilot.server.secret.create: false
copilot.server.secret.name: my-existing-copilot-secret
copilot.server.extraEnv:
- name: CUSTOM_VAR
value: custom-value
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: CUSTOM_VAR
value: custom-value
- notContains:
path: spec.template.spec.containers[0].env
content:
name: PORT
value: "8080"
- it: copilot's own existingSecret is still the source of truth even when externalSecrets.enabled=true globally for other components (Greptile round 3 finding)
template: deployment-copilot.yaml
documentSelector:
path: kind
value: Deployment
set:
app.env.BETTER_AUTH_SECRET: ""
app.env.ENCRYPTION_KEY: ""
app.env.INTERNAL_API_SECRET: ""
app.env.CRON_SECRET: ""
postgresql.auth.password: ""
copilot.enabled: true
copilot.postgresql.enabled: false
copilot.database.url: "postgresql://x:x@x:5432/x"
copilot.server.secret.create: false
copilot.server.secret.name: my-existing-copilot-secret
externalSecrets.enabled: true
externalSecrets.secretStoreRef.name: sim-store
externalSecrets.remoteRefs.app.BETTER_AUTH_SECRET: path/to/auth
externalSecrets.remoteRefs.app.ENCRYPTION_KEY: path/to/enc
externalSecrets.remoteRefs.app.INTERNAL_API_SECRET: path/to/iapi
externalSecrets.remoteRefs.app.CRON_SECRET: path/to/cron
externalSecrets.remoteRefs.postgresql.password: path/to/pgpw
asserts:
- notExists:
path: spec.template.spec.containers[0].env
- equal:
path: spec.template.spec.containers[0].envFrom[0].secretRef.name
value: my-existing-copilot-secret
- it: existingSecret mode skips the chart-managed copilot Secret and the ExternalSecret
templates:
- secrets-copilot.yaml
- external-secret-copilot.yaml
set:
app.env.BETTER_AUTH_SECRET: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.ENCRYPTION_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.INTERNAL_API_SECRET: x
app.env.CRON_SECRET: x
postgresql.auth.password: xxxxxxxx
copilot.enabled: true
copilot.postgresql.enabled: true
copilot.postgresql.auth.password: xxxxxxxx
copilot.server.secret.create: false
copilot.server.secret.name: my-existing-copilot-secret
asserts:
- hasDocuments: { count: 0 }
- it: ESO mode renders an ExternalSecret instead of the chart-managed copilot Secret
template: external-secret-copilot.yaml
set:
app.env.BETTER_AUTH_SECRET: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.ENCRYPTION_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.INTERNAL_API_SECRET: x
app.env.CRON_SECRET: x
postgresql.auth.password: xxxxxxxx
copilot.enabled: true
copilot.postgresql.enabled: false
copilot.database.url: "postgresql://x:x@x:5432/x"
externalSecrets.enabled: true
externalSecrets.secretStoreRef.name: sim-store
externalSecrets.remoteRefs.app.BETTER_AUTH_SECRET: path/to/auth
externalSecrets.remoteRefs.app.ENCRYPTION_KEY: path/to/enc
externalSecrets.remoteRefs.app.INTERNAL_API_SECRET: path/to/iapi
externalSecrets.remoteRefs.app.CRON_SECRET: path/to/cron
externalSecrets.remoteRefs.postgresql.password: path/to/pgpw
externalSecrets.remoteRefs.copilot.PORT: path/to/port
externalSecrets.remoteRefs.copilot.SERVICE_NAME: path/to/name
externalSecrets.remoteRefs.copilot.ENVIRONMENT: path/to/env
externalSecrets.remoteRefs.copilot.LOG_LEVEL: path/to/log
externalSecrets.remoteRefs.copilot.AGENT_API_DB_ENCRYPTION_KEY: path/to/agentdbkey
externalSecrets.remoteRefs.copilot.INTERNAL_API_SECRET: path/to/iapi
externalSecrets.remoteRefs.copilot.LICENSE_KEY: path/to/license
externalSecrets.remoteRefs.copilot.SIM_BASE_URL: path/to/baseurl
externalSecrets.remoteRefs.copilot.SIM_AGENT_API_KEY: path/to/agentkey
externalSecrets.remoteRefs.copilot.REDIS_URL: path/to/redis
externalSecrets.remoteRefs.copilot.OPENAI_API_KEY_1: path/to/openai
asserts:
- isKind: { of: ExternalSecret }
- equal: { path: metadata.name, value: t-sim-copilot-env }
- equal: { path: spec.secretStoreRef.name, value: sim-store }
- it: ESO mode skips the chart-managed copilot Secret
template: secrets-copilot.yaml
set:
app.env.BETTER_AUTH_SECRET: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.ENCRYPTION_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.INTERNAL_API_SECRET: x
app.env.CRON_SECRET: x
postgresql.auth.password: xxxxxxxx
copilot.enabled: true
copilot.postgresql.enabled: true
copilot.postgresql.auth.password: xxxxxxxx
externalSecrets.enabled: true
externalSecrets.secretStoreRef.name: sim-store
externalSecrets.remoteRefs.app.BETTER_AUTH_SECRET: path/to/auth
externalSecrets.remoteRefs.app.ENCRYPTION_KEY: path/to/enc
externalSecrets.remoteRefs.app.INTERNAL_API_SECRET: path/to/iapi
externalSecrets.remoteRefs.app.CRON_SECRET: path/to/cron
externalSecrets.remoteRefs.postgresql.password: path/to/pgpw
externalSecrets.remoteRefs.copilot.PORT: path/to/port
externalSecrets.remoteRefs.copilot.SERVICE_NAME: path/to/name
externalSecrets.remoteRefs.copilot.ENVIRONMENT: path/to/env
externalSecrets.remoteRefs.copilot.LOG_LEVEL: path/to/log
externalSecrets.remoteRefs.copilot.AGENT_API_DB_ENCRYPTION_KEY: path/to/agentdbkey
externalSecrets.remoteRefs.copilot.INTERNAL_API_SECRET: path/to/iapi
externalSecrets.remoteRefs.copilot.LICENSE_KEY: path/to/license
externalSecrets.remoteRefs.copilot.SIM_BASE_URL: path/to/baseurl
externalSecrets.remoteRefs.copilot.SIM_AGENT_API_KEY: path/to/agentkey
externalSecrets.remoteRefs.copilot.REDIS_URL: path/to/redis
externalSecrets.remoteRefs.copilot.OPENAI_API_KEY_1: path/to/openai
asserts:
- hasDocuments: { count: 0 }
- it: ESO validator fails when a required copilot key is neither in env nor mapped
template: deployment-copilot.yaml
set:
app.env.BETTER_AUTH_SECRET: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.ENCRYPTION_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.INTERNAL_API_SECRET: x
app.env.CRON_SECRET: x
postgresql.auth.password: xxxxxxxx
copilot.enabled: true
copilot.postgresql.enabled: false
copilot.database.url: "postgresql://x:x@x:5432/x"
externalSecrets.enabled: true
externalSecrets.secretStoreRef.name: sim-store
externalSecrets.remoteRefs.app.BETTER_AUTH_SECRET: path/to/auth
externalSecrets.remoteRefs.app.ENCRYPTION_KEY: path/to/enc
externalSecrets.remoteRefs.app.INTERNAL_API_SECRET: path/to/iapi
externalSecrets.remoteRefs.app.CRON_SECRET: path/to/cron
externalSecrets.remoteRefs.postgresql.password: path/to/pgpw
asserts:
- failedTemplate:
errorPattern: "Required key 'AGENT_API_DB_ENCRYPTION_KEY' is missing"
- it: ESO validator fails when copilot.server.env contains an unmapped key
template: deployment-copilot.yaml
set:
app.env.BETTER_AUTH_SECRET: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.ENCRYPTION_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.INTERNAL_API_SECRET: x
app.env.CRON_SECRET: x
postgresql.auth.password: xxxxxxxx
copilot.enabled: true
copilot.postgresql.enabled: false
copilot.database.url: "postgresql://x:x@x:5432/x"
externalSecrets.enabled: true
externalSecrets.secretStoreRef.name: sim-store
externalSecrets.remoteRefs.app.BETTER_AUTH_SECRET: path/to/auth
externalSecrets.remoteRefs.app.ENCRYPTION_KEY: path/to/enc
externalSecrets.remoteRefs.app.INTERNAL_API_SECRET: path/to/iapi
externalSecrets.remoteRefs.app.CRON_SECRET: path/to/cron
externalSecrets.remoteRefs.postgresql.password: path/to/pgpw
externalSecrets.remoteRefs.copilot.PORT: path/to/port
externalSecrets.remoteRefs.copilot.SERVICE_NAME: path/to/name
externalSecrets.remoteRefs.copilot.ENVIRONMENT: path/to/env
externalSecrets.remoteRefs.copilot.LOG_LEVEL: path/to/log
externalSecrets.remoteRefs.copilot.AGENT_API_DB_ENCRYPTION_KEY: path/to/agentdbkey
externalSecrets.remoteRefs.copilot.INTERNAL_API_SECRET: path/to/iapi
externalSecrets.remoteRefs.copilot.LICENSE_KEY: path/to/license
externalSecrets.remoteRefs.copilot.SIM_BASE_URL: path/to/baseurl
externalSecrets.remoteRefs.copilot.SIM_AGENT_API_KEY: path/to/agentkey
externalSecrets.remoteRefs.copilot.REDIS_URL: path/to/redis
externalSecrets.remoteRefs.copilot.OPENAI_API_KEY_1: path/to/openai
copilot.server.env.UNMAPPED_KEY: "would-go-nowhere"
asserts:
- failedTemplate:
errorPattern: "Key 'UNMAPPED_KEY' is set in copilot.server.env but externalSecrets.enabled=true"
- it: non-ESO mode still requires OPENAI_API_KEY_1 or ANTHROPIC_API_KEY_1 directly in env
template: deployment-copilot.yaml
set:
app.env.BETTER_AUTH_SECRET: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.ENCRYPTION_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
app.env.INTERNAL_API_SECRET: x
app.env.CRON_SECRET: x
postgresql.auth.password: xxxxxxxx
copilot.enabled: true
copilot.postgresql.enabled: false
copilot.database.url: "postgresql://x:x@x:5432/x"
copilot.server.env.AGENT_API_DB_ENCRYPTION_KEY: x
copilot.server.env.INTERNAL_API_SECRET: x
copilot.server.env.LICENSE_KEY: x
copilot.server.env.SIM_BASE_URL: x
copilot.server.env.SIM_AGENT_API_KEY: x
copilot.server.env.REDIS_URL: x
asserts:
- failedTemplate:
errorPattern: "Set at least one of copilot.server.env.OPENAI_API_KEY_1 or copilot.server.env.ANTHROPIC_API_KEY_1"