chore: import upstream snapshot with attribution
Check engine pin consistency / Dockerfile / CI pin consistency (push) Successful in 8s
Sirius CI/CD Pipeline / Detect Changes (push) Successful in 23s
Validate Docker Configuration / Validate Docker Compose Configuration (push) Successful in 47s
Sirius CI/CD Pipeline / Build API (${{ matrix.platform }}) (push) Has been cancelled
Sirius CI/CD Pipeline / Build UI (${{ matrix.platform }}) (push) Has been cancelled
Sirius CI/CD Pipeline / Merge Engine Manifest (push) Has been cancelled
Sirius CI/CD Pipeline / Merge API Manifest (push) Has been cancelled
Sirius CI/CD Pipeline / Merge UI Manifest (push) Has been cancelled
Sirius CI/CD Pipeline / Build Engine (${{ matrix.platform }}) (push) Has been cancelled
Sirius CI/CD Pipeline / Build Infra (${{ matrix.service }}, ${{ matrix.platform }}) (push) Has been cancelled
Sirius CI/CD Pipeline / Merge Infra Manifest (sirius-postgres) (push) Has been cancelled
Sirius CI/CD Pipeline / Merge Infra Manifest (sirius-rabbitmq) (push) Has been cancelled
Sirius CI/CD Pipeline / Merge Infra Manifest (sirius-valkey) (push) Has been cancelled
Sirius CI/CD Pipeline / Integration Test (push) Has been cancelled
Sirius CI/CD Pipeline / Public Stack Contract (push) Has been cancelled
Sirius CI/CD Pipeline / Dispatch Demo Deployment (sirius-demo branch) (push) Has been cancelled
Sirius CI/CD Pipeline / Dispatch Demo Canary (main branch) (push) Has been cancelled
Sirius CI/CD Pipeline / Guard Registry Namespace (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:32:25 +08:00
commit 161ef94b4f
708 changed files with 189571 additions and 0 deletions
@@ -0,0 +1,428 @@
---
title: "Docker Container Deployment Guide"
description: "Complete guide for deploying Sirius using prebuilt container images from GitHub Container Registry"
template: "TEMPLATE.guide"
version: "1.0.0"
last_updated: "2026-04-08"
author: "Development Team"
tags: ["docker", "deployment", "containers", "ghcr", "registry", "production"]
categories: ["deployment", "infrastructure"]
difficulty: "beginner"
prerequisites: ["docker", "docker-compose"]
related_docs:
- "README.terraform-deployment.md"
- "README.development.md"
- "README.docker-architecture.md"
dependencies: ["docker-compose.yaml"]
llm_context: "high"
search_keywords:
[
"docker",
"deployment",
"containers",
"ghcr",
"registry",
"production",
"images",
"compose",
]
---
# Docker Container Deployment Guide
## Purpose
This guide explains how to deploy Sirius using prebuilt container images from GitHub Container Registry (GHCR). This approach provides faster deployments (5-8 minutes vs 20-25 minutes) by eliminating on-instance Docker builds. The guide covers production deployments, image versioning, and fallback strategies.
## When to Use
- **Production deployments** - Deploying Sirius to production environments
- **Demo environments** - Setting up demo instances quickly
- **Staging environments** - Creating isolated testing environments
- **Quick deployments** - When you need fast deployment without building from source
- **CI/CD pipelines** - Automated deployments using prebuilt images
**Avoid when**:
- **Local development** - Use `docker-compose.dev.yaml` for local builds with hot reloading
- **Custom modifications** - When you need to modify source code before deployment
- **Offline environments** - When registry access is unavailable
## How to Use
### Quick Start
```bash
# Clone repository
git clone https://github.com/SiriusScan/Sirius.git
cd Sirius
# Generate startup config and secrets
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
# Deploy with prebuilt images (default)
docker compose up -d
# Or specify a version tag
IMAGE_TAG=v0.4.1 docker compose up -d
# Optional: verify the public GHCR contract before pulling
bash scripts/verify-ghcr-public-access.sh "${IMAGE_TAG:-latest}"
```
### Prerequisites
- **Docker** >= 20.10.0
- **Docker Compose** >= 2.0.0
- **Internet access** to GitHub Container Registry (ghcr.io)
- **Git** (for cloning repository)
## Container Registry Integration
### GitHub Container Registry (GHCR)
Sirius images are automatically built and pushed to GitHub Container Registry on every push to the main branch. Images are available at:
- **UI**: `ghcr.io/siriusscan/sirius-ui:{tag}`
- **API**: `ghcr.io/siriusscan/sirius-api:{tag}`
- **Engine**: `ghcr.io/siriusscan/sirius-engine:{tag}`
- **Postgres**: `ghcr.io/siriusscan/sirius-postgres:{tag}`
- **RabbitMQ**: `ghcr.io/siriusscan/sirius-rabbitmq:{tag}`
- **Valkey**: `ghcr.io/siriusscan/sirius-valkey:{tag}`
`docker-compose.yaml` is the public-image deployment path for Sirius. If an unauthenticated `docker pull` against one of the image refs above returns `unauthorized`, the GHCR public-visibility contract is broken and operators should stop before rollout.
### Image Tagging Strategy
Images are tagged with the following strategy:
| Tag | Description | When Created |
| -------- | ------------------------ | ----------------------- |
| `latest` | Latest main branch build | On every push to main |
| `beta` | Beta release candidate | On main branch pushes |
| `v0.4.1` | Version-specific tag | After the `Publish Release Image Tags` workflow succeeds |
| `dev` | Development builds | On other branch pushes |
| `pr-123` | Pull request builds | On PR creation/updates |
### Image Availability
- **Public images**: No authentication required
- **Multi-architecture**: Images support `linux/amd64` and `linux/arm64`
- **Automatic updates**: Latest images are built automatically by CI/CD
- **Release contract**: A release tag is only valid for operators after the release-tag workflow publishes it, the anonymous GHCR verification step passes, and the public Compose smoke test succeeds. CI validates **`latest`** on every main push (`public-stack-contract` in `ci.yml`); **semver** tags are additionally checked when a GitHub Release is published and on a weekly schedule ([`verify-ghcr-release-tag.yml`](../../../.github/workflows/verify-ghcr-release-tag.yml)).
## Docker Compose Configuration
### Base Configuration (Production)
The default `docker-compose.yaml` uses prebuilt images:
```yaml
services:
sirius-ui:
image: ghcr.io/siriusscan/sirius-ui:${IMAGE_TAG:-latest}
pull_policy: always
# ... other configuration
sirius-api:
image: ghcr.io/siriusscan/sirius-api:${IMAGE_TAG:-latest}
pull_policy: always
# ... other configuration
sirius-engine:
image: ghcr.io/siriusscan/sirius-engine:${IMAGE_TAG:-latest}
pull_policy: always
# ... other configuration
```
### Environment Variables
Control which images to use with the `IMAGE_TAG` environment variable:
```bash
# Use latest images (default)
docker compose up -d
# Use specific version
IMAGE_TAG=v0.4.1 docker compose up -d
# Use beta release
IMAGE_TAG=beta docker compose up -d
# Validate that the selected tag is publicly readable before rollout
bash scripts/verify-ghcr-public-access.sh "${IMAGE_TAG:-latest}"
```
`.env.production.example` leaves `IMAGE_TAG` blank so fresh installer runs inherit the Compose default (`latest`). Pin a release tag only after the publish workflow has validated that all six Sirius images exist for that tag.
### Development Override
For local development with source code changes, use the development override:
```bash
# Build locally with hot reloading
docker compose -f docker-compose.yaml -f docker-compose.dev.yaml up -d --build
```
The `docker-compose.dev.yaml` file overrides the registry images with local builds, enabling:
- Hot reloading
- Volume mounts for source code
- Development-specific environment variables
- Debug logging
## Deployment Workflow
### Standard Production Deployment
1. **Clone repository**:
```bash
git clone https://github.com/SiriusScan/Sirius.git
cd Sirius
```
2. **Configure environment**:
```bash
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
# Optional: pass explicit values
# docker compose -f docker-compose.installer.yaml run --rm sirius-installer --non-interactive --no-print-secrets
```
3. **Deploy services**:
```bash
docker compose up -d
```
4. **Verify deployment**:
```bash
docker compose ps
docker compose logs -f
```
5. **Check health**:
```bash
curl http://localhost:9001/health # API
curl http://localhost:3000/api/health # UI
```
### Maintainer Validation Path
Use the shared validation script to exercise the same public Compose path that operators use:
```bash
# Validate the default public stack (latest)
bash scripts/validate-public-compose-path.sh latest
# Validate a published release tag
bash scripts/validate-public-compose-path.sh v0.4.1
```
### Version-Specific Deployment
Deploy a specific version:
```bash
# Set version tag
export IMAGE_TAG=v0.4.1
# Confirm the compose-rendered GHCR images are public and readable
bash scripts/verify-ghcr-public-access.sh "$IMAGE_TAG"
# Pull and start services
docker compose pull
docker compose up -d
```
### Updating Deployment
To update to the latest version:
```bash
# Pull latest images
docker compose pull
# Restart services with new images
docker compose up -d
```
## Fallback Strategy
### When Registry is Unavailable
If GitHub Container Registry is unavailable or images fail to pull, you can fall back to local builds:
1. **Use the committed source-build override**:
```bash
docker compose -f docker-compose.yaml -f docker-compose.build.yaml up -d --build
```
**Note**: Local builds take significantly longer (20-25 minutes vs 5-8 minutes) and require more system resources.
## Secrets Hardening Overlays
For hardened deployments, Sirius includes optional overlay manifests:
- `docker-compose.secrets.yaml` for Compose secrets mounted at `/run/secrets/*`
- `docker-stack.swarm.yaml` for Swarm stack deployments
Example:
```bash
mkdir -p secrets
printf '%s' "your-postgres-password" > secrets/postgres_password.txt
printf '%s' "your-service-key" > secrets/sirius_api_key.txt
chmod 644 secrets/sirius_api_key.txt
printf '%s' "your-nextauth-secret" > secrets/nextauth_secret.txt
printf '%s' "your-admin-password" > secrets/initial_admin_password.txt
docker compose -f docker-compose.yaml -f docker-compose.secrets.yaml up -d
```
`sirius_api_key.txt` should stay **world-readable** (`644`) so bind-mounted `/run/secrets/sirius_api_key` is readable inside **sirius-api** / **sirius-ui** / **sirius-engine** (non-root UIDs).
## Troubleshooting
### Images Not Pulling
**Problem**: `docker compose pull` fails with authentication or network errors.
**Solutions**:
- Verify internet connectivity: `curl -I https://ghcr.io`
- Check image exists: Visit `https://github.com/SiriusScan/Sirius/pkgs/container/sirius-ui`
- Try pulling manually: `docker pull ghcr.io/siriusscan/sirius-ui:latest`
- Run the contract check: `bash scripts/verify-ghcr-public-access.sh "${IMAGE_TAG:-latest}"`
- If the script reports `Anonymous access denied`, the package is not publicly readable and the GHCR visibility workflow or token scope needs attention
- If the script reports `Manifest missing`, the requested tag was not published and you should verify the release-tag workflow completed successfully
- If the public Compose smoke test fails after pull succeeds, run `bash scripts/validate-public-compose-path.sh "${IMAGE_TAG:-latest}"` to reproduce the operator path and inspect the runtime contract failure
- Use fallback build strategy (see above)
### Wrong Version Deployed
**Problem**: Services are running an unexpected version.
**Solutions**:
- Check current IMAGE_TAG: `echo $IMAGE_TAG`
- Verify image tags: `docker compose images`
- Pull specific version: `IMAGE_TAG=v0.4.1 docker compose pull`
- Restart services: `docker compose up -d`
### Services Not Starting
**Problem**: Containers fail to start after pulling images.
**Solutions**:
- Check logs: `docker compose logs`
- Verify environment variables: `docker compose config`
- Check image compatibility: Ensure architecture matches (amd64/arm64)
- Verify dependencies: Ensure PostgreSQL, RabbitMQ, Valkey are running
### Performance Issues
**Problem**: Deployment is slower than expected.
**Solutions**:
- Check network speed: `docker pull` should be fast on good connections
- Verify image sizes: Large images take longer to pull
- Use specific version tags instead of `latest` for faster pulls
- Consider using image caching strategies
## Best Practices
### Security
- **Use specific version tags** in production (e.g., `v0.4.1`) instead of `latest`
- **Regularly update images** to get security patches
- **Scan images** for vulnerabilities using Docker security scanning
- **Use private registries** for sensitive deployments (if needed)
### Version Management
- **Pin versions** in production environments
- **Test updates** in staging before production
- **Document versions** deployed in each environment
- **Use semantic versioning** for releases
### Performance
- **Pre-pull images** before deployment to reduce startup time
- **Use image caching** to avoid redundant pulls
- **Monitor image sizes** and optimize Dockerfiles if needed
- **Use multi-stage builds** to reduce final image sizes
### Monitoring
- **Track deployment times** to measure improvements
- **Monitor registry availability** and fallback usage
- **Log image versions** deployed for audit trails
- **Alert on deployment failures** for quick response
## Comparison: Registry vs Local Builds
| Aspect | Registry Images | Local Builds |
| --------------------- | ----------------- | -------------------- |
| **Deployment Time** | 5-8 minutes | 20-25 minutes |
| **Resource Usage** | Low (pull only) | High (compilation) |
| **Network Required** | Yes (for pull) | No (after clone) |
| **Customization** | Limited | Full |
| **CI/CD Integration** | Automatic | Manual |
| **Best For** | Production, demos | Development, offline |
## Integration with CI/CD
### GitHub Actions
Images are automatically built and pushed by GitHub Actions on:
- Push to `main` branch → `latest` and `beta` tags
- Manual `Publish Release Image Tags` run → version-specific tag (e.g., `v0.4.1`)
- Pull requests → `pr-{number}` tags
### Deployment Automation
Example GitHub Actions workflow for deployment:
```yaml
name: Deploy Sirius
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy with registry images
run: |
docker compose pull
docker compose up -d
```
## Related Documentation
- [Terraform Deployment Guide](README.terraform-deployment.md) - AWS deployment using Terraform
- [Development Guide](../README.development.md) - Local development setup
- [Docker Architecture Guide](../architecture/README.docker-architecture.md) - Container architecture details
## Support
For issues with container deployment:
1. Check the troubleshooting section above
2. Review Docker logs: `docker compose logs`
3. Verify image availability on GitHub Container Registry
4. Create an issue in the Sirius repository
---
_This guide follows the Sirius Documentation Standard. For questions about documentation structure, see [ABOUT.documentation.md](../ABOUT.documentation.md)._
@@ -0,0 +1,464 @@
---
title: "GitHub Actions Workflow Architecture"
description: "Comprehensive guide to Sirius CI/CD pipeline structure, parallel build jobs, and deployment workflows"
template: "TEMPLATE.documentation-standard"
version: "1.0.0"
last_updated: "2025-11-14"
author: "Sirius Team"
tags: ["ci-cd", "github-actions", "docker", "ghcr", "parallel-builds"]
categories: ["deployment", "automation"]
difficulty: "intermediate"
prerequisites: ["README.docker-container-deployment.md", "README.architecture.md"]
related_docs:
- "README.docker-container-deployment.md"
- "README.terraform-deployment.md"
- "README.cicd.md"
dependencies:
- ".github/workflows/ci.yml"
llm_context: "high"
search_keywords: ["ci-cd", "github-actions", "workflow", "parallel", "build", "deploy", "ghcr", "container-registry"]
---
# GitHub Actions Workflow Architecture
## Purpose
This document describes the Sirius CI/CD pipeline structure, explaining how GitHub Actions workflows orchestrate parallel container builds, validation, testing, and deployment. It's designed for developers maintaining or modifying the CI/CD pipeline and for those troubleshooting build issues.
## When to Use
- **Modifying CI/CD pipeline**: Making changes to build, test, or deployment processes
- **Troubleshooting build failures**: Understanding job dependencies and execution order
- **Adding new services**: Integrating new containers into the build pipeline
- **Optimizing build times**: Identifying parallelization opportunities
- **Debugging deployment issues**: Understanding how code reaches production
## How to Use
### Quick Reference
```bash
# Check workflow status
gh run list --workflow=ci.yml --limit 5
# Watch current run
gh run watch
# View workflow logs
gh run view <run-id> --log
# Manually trigger workflow
gh workflow run ci.yml --ref main
```
### Understanding Workflow Execution
1. **Code Push/PR**: Triggers the workflow automatically
2. **Change Detection**: Determines which services need rebuilding
3. **Parallel Builds**: UI, API, and Engine build concurrently
4. **Integration Tests**: Validates built images work together
5. **Deployment Dispatch**: Triggers downstream deployments
## What It Is
### Workflow Overview
The Sirius CI/CD pipeline is implemented in `.github/workflows/ci.yml` and follows a **parallel build architecture** that significantly reduces build times compared to sequential builds.
**Other workflows** (not shown in the diagram below):
- [`.github/workflows/publish-release-image-tags.yml`](../../../.github/workflows/publish-release-image-tags.yml) — manual retag of all six GHCR images from a source tag (e.g. `latest`) to a release tag (e.g. `v1.0.0`).
- [`.github/workflows/verify-ghcr-release-tag.yml`](../../../.github/workflows/verify-ghcr-release-tag.yml) — on each **published** GitHub Release, on a **weekly** schedule, and via **workflow_dispatch**, runs `scripts/verify-ghcr-public-access.sh` so the release tag cannot drift from what anonymous operators can pull (see [OPERATIONS.md](../../../OPERATIONS.md) GHCR checklist).
**Key characteristics:**
- **Parallel execution**: UI, API, and Engine containers build simultaneously
- **Smart change detection**: Only rebuilds services with code changes
- **Prebuild validation**: Runs lints/tests before expensive Docker builds
- **Multi-arch support**: Builds for both amd64 and arm64 architectures
- **Container registry integration**: Pushes to GitHub Container Registry (GHCR)
### Architecture Diagram
```
┌─────────────────┐
│ detect-changes │ ← Determines what needs rebuilding
└────────┬────────┘
├──────────┬──────────┬──────────┐
▼ ▼ ▼ ▼
┌────────┐ ┌────────┐ ┌──────────┐ │
│build-ui│ │build-api│ │build-engine│ │ ← Run in parallel
└───┬────┘ └───┬────┘ └─────┬────┘ │
│ │ │ │
└──────────┴────────────┴─────────┘
┌────────┐
│ test │ ← Integration testing
└───┬────┘
┬─────────┴─────────┬
▼ ▼
┌───────────────┐ ┌──────────────┐
│dispatch-demo- │ │dispatch-demo-│
│ deployment │ │ canary │
└───────────────┘ └──────────────┘
```
## Workflow Jobs
### 1. detect-changes
**Purpose**: Analyzes commits to determine which services need rebuilding.
**Triggers on**:
- Direct pushes to `main` or `sirius-demo` branches
- Pull requests to `main`
- Repository dispatch events (submodule updates)
**Outputs**:
- `sirius_ui_changes`: Boolean indicating UI code changes
- `sirius_api_changes`: Boolean indicating API code changes
- `sirius_engine_changes`: Boolean indicating Engine code changes
- `submodule_changes`: Boolean indicating submodule updates
**Change detection logic**:
```yaml
# UI changes: Any file in sirius-ui/
# API changes: Any file in sirius-api/
# Engine changes: Any file in sirius-engine/ or rabbitmq/
# Global changes: Dockerfile, docker-compose, .github/ → rebuild all
```
### 2. build-ui (Parallel)
**Purpose**: Validates and builds the Next.js UI container.
**Runs when**: `detect-changes.outputs.sirius_ui_changes == 'true'`
**Steps**:
1. **Generate metadata**: Determines image tag (`latest`, `beta`, `pr-*`)
2. **Validate UI code**: Runs `npm ci && npm run lint`
3. **Set up Docker Buildx**: Configures multi-arch builds
4. **Log in to GHCR**: Authenticates with container registry
5. **Build and push**: Creates and pushes amd64/arm64 images
**Output**: `image_tag` (e.g., `latest`, `beta`, `pr-123`)
**Build platforms**: `linux/amd64`, `linux/arm64`
**Validation timing**: ~2-3 minutes (lint before Docker build)
### 3. build-api (Parallel)
**Purpose**: Validates and builds the Go API container.
**Runs when**: `detect-changes.outputs.sirius_api_changes == 'true'`
**Steps**:
1. **Generate metadata**: Determines image tag and submodule SHAs
2. **Set up Go**: Installs Go 1.24
3. **Validate API code**: Runs `go mod download && go test ./...`
4. **Set up Docker Buildx**: Configures multi-arch builds
5. **Log in to GHCR**: Authenticates with container registry
6. **Build and push**: Creates and pushes amd64/arm64 images with submodule refs
**Output**: `image_tag`
**Build args**: `GO_API_COMMIT_SHA` (for go-api submodule)
**Validation timing**: ~1-2 minutes (tests before Docker build)
### 4. build-engine (Parallel)
**Purpose**: Validates and builds the Go Engine container.
**Runs when**: `detect-changes.outputs.sirius_engine_changes == 'true'`
**Steps**:
1. **Generate metadata**: Determines image tag and all submodule SHAs
2. **Set up Go**: Installs Go 1.24
3. **Validate Engine code**: Runs `go mod download && go test ./...`
4. **Set up Docker Buildx**: Configures multi-arch builds
5. **Log in to GHCR**: Authenticates with container registry
6. **Build and push**: Creates and pushes amd64/arm64 images with all submodule refs
**Output**: `image_tag`
**Build args**:
- `GO_API_COMMIT_SHA`
- `APP_SCANNER_COMMIT_SHA`
- `APP_TERMINAL_COMMIT_SHA`
- `SIRIUS_NSE_COMMIT_SHA`
- `APP_AGENT_COMMIT_SHA`
**Validation timing**: ~1-2 minutes (tests before Docker build)
### 5. test
**Purpose**: Validates that built containers work together in an integrated environment.
**Runs when**: At least one build job succeeds
**Dependencies**: `[detect-changes, build-ui, build-api, build-engine]`
**Steps**:
1. **Determine image tag**: Uses output from whichever build job ran
2. **Create test environment**: Generates `docker-compose.test.yml` with fresh images
3. **Start infrastructure**: Launches Postgres, RabbitMQ, Valkey
4. **Start application services**: Launches only the services that were built
5. **Run smoke tests**: Validates services are running and responsive
6. **Cleanup**: Tears down test environment
**Test configuration**:
- Uses in-memory Postgres (`tmpfs`) for speed
- Isolated test database (`sirius_test`)
- Debug-level logging for troubleshooting
### 6. dispatch-demo-deployment
**Purpose**: Triggers deployment to the demo environment when `sirius-demo` branch updates.
**Runs when**: Push to `sirius-demo` branch after successful build + test
**Dependencies**: `[detect-changes, build-ui, build-api, build-engine, test]`
**Sends to**: `SiriusScan/sirius-demo` repository with event type `sirius-demo-updated`
**Payload includes**:
- Source repo/branch/SHA
- Triggering actor
- Commit message
### 7. dispatch-demo-canary
**Purpose**: Triggers demo rebuild on every `main` branch push as a deployment canary.
**Runs when**: Push to `main` branch after successful build + test
**Dependencies**: `[detect-changes, build-ui, build-api, build-engine, test]`
**Sends to**: `SiriusScan/sirius-demo` repository with event type `sirius-main-updated`
**Purpose of canary**: Catches bad commits to `main` by immediately deploying to demo environment
## Image Tagging Strategy
### Tag Types
**`latest`**:
- Pushed on every `main` branch commit
- Also tagged as `beta` simultaneously
- Used by default in `docker-compose.yaml`
**`beta`**:
- Alias for `latest` (same image)
- Explicitly labeled for beta testing
**`pr-{number}`**:
- Unique tag for pull request builds
- Enables testing PRs in isolation
**`dev`**:
- Fallback for other branches/events
### Tag Determination
```bash
# Pull request
TAG="pr-123"
# Push to main or repository_dispatch
TAG="latest"
also_tag_beta="true"
# Other events
TAG="dev"
```
## Prebuild Validation
Each build job runs validation **before** Docker builds to fail fast and save time.
### UI Validation
```bash
cd sirius-ui
npm ci # Install dependencies
npm run lint # ESLint validation
# Docker build only if lint succeeds
```
**Typical duration**: 2-3 minutes
**Catches**: Import errors, syntax issues, unused variables
### API Validation
```bash
cd sirius-api
go mod download # Download dependencies
go test ./... -v # Run tests
# Docker build only if tests pass
```
**Typical duration**: 1-2 minutes
**Catches**: Compilation errors, failing tests, import issues
### Engine Validation
```bash
cd sirius-engine
go mod download # Download dependencies
go test ./... -v # Run tests
# Docker build only if tests pass
```
**Typical duration**: 1-2 minutes
**Catches**: Compilation errors, failing tests, integration issues
## Timing Expectations
### Before Parallelization (Sequential Builds)
| Phase | Duration |
|-------|----------|
| Change detection | ~30s |
| Build UI | ~15-20 min |
| Build API | ~20-25 min |
| Build Engine | ~30-40 min |
| Test | ~5 min |
| **Total** | **~70-90 min** |
### After Parallelization (Current)
| Phase | Duration |
|-------|----------|
| Change detection | ~30s |
| Prebuild validation (all parallel) | ~2-3 min |
| Build UI, API, Engine (all parallel) | ~30-40 min (longest wins) |
| Test | ~5 min |
| **Total** | **~40-50 min** |
**Time savings**: ~40-60% reduction (30-40 minutes faster)
## Authentication & Secrets
### Required Secrets
**`GHCR_PUSH_USER`**: GitHub username that generated the PAT
**`GHCR_PUSH_TOKEN`**: GitHub Personal Access Token with scopes:
- `write:packages` (push images)
- `read:packages` (pull existing layers)
- `delete:packages` (cleanup, optional)
- `repo` (access repository context)
**`GITHUB_TOKEN`**: Automatically provided by GitHub Actions
- Used for repository dispatch events
- Has limited package permissions (read-only)
### Setting Up Secrets
```bash
# Repository secrets (Sirius repo)
Settings → Security → Secrets and variables → Actions
# Add GHCR_PUSH_USER (your GitHub username)
Name: GHCR_PUSH_USER
Value: your-username
# Add GHCR_PUSH_TOKEN (PAT with package scopes)
Name: GHCR_PUSH_TOKEN
Value: ghp_xxxxxxxxxxxx
```
## Troubleshooting
### Build Fails with "denied: installation not allowed"
**Cause**: Default `GITHUB_TOKEN` doesn't have package creation permission.
**Fix**: Use PAT-based authentication (already implemented with `GHCR_PUSH_USER`/`GHCR_PUSH_TOKEN`).
### Build Fails with "403 Forbidden" during push
**Cause**: PAT missing required scopes or package visibility is restricted.
**Fix**:
1. Verify PAT has `write:packages` and `read:packages` scopes
2. Check package is public: `https://github.com/orgs/SiriusScan/packages/container/sirius-{ui,api,engine}/settings`
3. Set visibility to Public (one-time change)
### Test job fails to pull images
**Cause**: Image tag mismatch or build job didn't complete.
**Fix**:
1. Check build job outputs: `needs.build-{ui,api,engine}.outputs.image_tag`
2. Verify images exist in GHCR: `https://github.com/orgs/SiriusScan/packages`
3. Check GHCR authentication in test job
### Dispatch jobs don't trigger downstream
**Cause**: Missing `GITHUB_TOKEN` permission or incorrect event type.
**Fix**:
1. Verify `GITHUB_TOKEN` has workflow permissions
2. Check downstream repo workflow listens for correct event type
3. Confirm payload format matches expectations
### Builds take longer than expected
**Expected timings** (parallel mode):
- Prebuild validation: 2-3 min
- Docker builds: 30-40 min (longest service)
- Integration test: 5 min
**Investigate if**:
- Build cache not working (check `cache-from: type=gha`)
- Network issues downloading dependencies
- Jobs running sequentially instead of parallel
## Best Practices
### When Modifying Workflows
1. **Test in PR first**: Use `pr-*` tags to test changes without affecting `latest`
2. **Preserve outputs**: Build jobs must output `image_tag` for downstream jobs
3. **Use `always()`**: Downstream jobs should use `always()` with result checks
4. **Validate locally**: Use `act` (GitHub Actions local runner) when possible
5. **Check syntax**: Run `actionlint` before committing
### Adding New Services
1. **Create build job**: Copy pattern from `build-ui/api/engine`
2. **Add change detection**: Update `detect-changes` job logic
3. **Update test job**: Add service to `docker-compose.test.yml`
4. **Update dispatch dependencies**: Include new job in `needs` array
### Optimizing Build Times
**Already implemented**:
- Parallel builds (saves 40-60%)
- Prebuild validation (fails fast)
- Docker layer caching (`cache-from: type=gha`)
- Multi-arch builds in one step
**Future opportunities**:
- Cache Go dependencies between runs
- Use `npm ci --prefer-offline` for UI builds
- Split test job into parallel service-specific tests
## Related Documentation
- **[README.docker-container-deployment.md](README.docker-container-deployment.md)**: Container registry deployment guide
- **[README.terraform-deployment.md](../operations/README.terraform-deployment.md)**: Terraform-based infrastructure deployment
- **[README.cicd.md](../architecture/README.cicd.md)**: CI/CD architecture overview
- **[README.docker-architecture.md](../architecture/README.docker-architecture.md)**: Docker multi-stage builds and architecture
## Additional Resources
- **GitHub Actions Docs**: https://docs.github.com/en/actions
- **Docker Buildx**: https://docs.docker.com/buildx/
- **GHCR Documentation**: https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry