siriusscan--sirius/documentation/dev

title, description, template, version, last_updated, author, tags, categories, difficulty, prerequisites, related_docs, dependencies, llm_context, search_keywords

title	description	template	version	last_updated	author	tags	categories	difficulty	prerequisites	related_docs	dependencies	llm_context	search_keywords
Sirius Development Setup	Comprehensive guide for setting up and using the Sirius development environment, including standard and extended development modes with local repository integration.	TEMPLATE.documentation-standard	1.0.0	2026-04-07	Development Team	development setup docker workflow	development setup	intermediate	docker git	README.container-testing.md ABOUT.documentation.md	docker-compose.yaml scripts/dev-setup.sh	medium	development setup docker compose local development workflow

Sirius Development Setup

Quick Start

New in v1.0.0: Use the improved environment switching system for easier development.

# Generate/merge required .env values once (installer-first)
docker compose -f docker-compose.installer.yaml run --rm sirius-installer

# Modern development workflow (recommended)
./scripts/switch-env.sh dev

# Legacy development (uses built-in repositories)
./scripts/dev-setup.sh start

# Extended development (with local repository mounts)
./scripts/dev-setup.sh init           # Create local overrides
./scripts/dev-setup.sh start-extended # Start with local mounts

Note

: For the complete developer experience, see README.developer-guide.md for comprehensive development workflows and best practices.

Development Modes

🎯 Standard Development

Best for: UI/API development, testing, most development tasks

./scripts/dev-setup.sh start

• Uses built-in repositories from Docker images
• No local repository setup required
• All services start with go run in development mode
• Live reloading for UI changes
• Uses secrets generated by docker-compose.installer.yaml (.env) for service auth/DB consistency

🔧 Extended Development

Best for: Working on scanner, terminal, or agent code

# 1. Set up local repositories (one-time)
mkdir -p ../minor-projects && cd ../minor-projects
git clone https://github.com/SiriusScan/app-scanner.git
git clone https://github.com/SiriusScan/app-terminal.git
git clone https://github.com/SiriusScan/app-agent.git
cd ../Sirius

# 2. Initialize local overrides
./scripts/dev-setup.sh init

# 3. Edit docker-compose.local.yaml (uncomment what you need)
nano docker-compose.local.yaml

# 4. Start extended development
./scripts/dev-setup.sh start-extended

Hot reloading in extended mode (engine services):

Each minor-project ships its own .air.toml (app-agent/.air.toml, app-terminal/.air.toml, app-scanner/.air.toml). When GO_ENV=development (the default for docker-compose.dev.yaml), start-enhanced.sh checks for .air.toml in the bind-mounted source and uses Air for live rebuilds; if .air.toml is absent (or air is missing from the image) it falls back to plain go run. The matrix:

Service	Bind mount path in container	Live-reload tool	Entry point
app-agent	/app-agent	air	cmd/server/main.go
app-terminal	/app-terminal	air	cmd/main.go
app-scanner	/app-scanner	air (CGO_ENABLED=1)	main.go

Notes:

• Changes to bind-mounted source are picked up automatically by Air; you should see a rebuild log line in docker logs sirius-engine within ~1s of saving.
• app-scanner requires CGO and libpcap; the container ships both, so the in-container Air build works out of the box.
• The orphan sirius-engine/.air.toml that previously lived in the engine image was removed in the April 2026 dev-mode overhaul; per-service .air.toml is the only supported configuration.
• Changes are not tracked by the main Sirius repository (the bind mounts are git-ignored).

When Air isn't enough — hot-swap from local source:

Sometimes you want to validate a change against a production-mode engine without waiting for a full container rebuild (e.g. you're testing a binary against docker-compose.prod.yaml, or you want to confirm a release-style build before bumping a pin). Use the root Makefile targets:

make engine-mode                # show whether the running engine is dev or prod
make agent-hot-swap-from-local
make terminal-hot-swap-from-local
make scanner-hot-swap-from-local      # CGO; fails cleanly if host lacks libpcap
make engine-rebuild-from-local        # all three, best-effort

What they do: cross-compile the binary on the host (linux/$(uname -m) by default; override with HOST_GOARCH=amd64), docker cp it into the engine container at the production binary path (e.g. /app-agent-src/server), and restart the engine.

Important guard rails:

• The targets refuse to run in dev mode (GO_ENV=development). In dev, /app-terminal and /app-scanner are bind-mounted, so docker cp would write through to your host repo; and Air owns the live binary at ./tmp/main, so the swapped production-path binary wouldn't even be executed. The right loop in dev is edit source → Air rebuilds. Override with FORCE_HOT_SWAP=1 only if you know what you want.
• Scanner cross-compile may fail on macOS hosts. The scanner uses cgo + libpcap; the macOS toolchain can't satisfy linux cgo headers. The target prints the actual cgo error and a copy-pasteable in-container build command. This is expected and not a Makefile bug.
• These targets are an inner-loop convenience, not a release path. Once the change is good, push it to the relevant minor-project and bump the engine pin per README.engine-component-pinning.md. The CI pipeline is the single source of truth for shipped images.

File Structure

Sirius/
├── docker-compose.yaml                 # Base configuration
├── docker-compose.override.yaml        # 🔒 Committed: Safe development defaults
├── docker-compose.local.example.yaml   # 🔒 Committed: Template for local overrides
├── docker-compose.local.yaml           # 🚫 Git-ignored: Your personal overrides
└── scripts/dev-setup.sh               # 🔒 Committed: Development helper

Safety Features

🛡️ Prevents Accidental Commits

• docker-compose.local.yaml is git-ignored
• CI/CD validates that volume mounts stay commented in docker-compose.override.yaml
• Pre-commit hook auto-fixes uncommented volume mounts

🔧 Developer-Friendly

• Easy setup with ./scripts/dev-setup.sh init
• Template file shows all available options
• Helper script for common development tasks

Available Commands

./scripts/dev-setup.sh init              # Create local overrides from template
./scripts/dev-setup.sh start             # Standard development mode
./scripts/dev-setup.sh start-extended    # Extended development with local repos
./scripts/dev-setup.sh stop              # Stop all services
./scripts/dev-setup.sh status            # Show container status
./scripts/dev-setup.sh logs [service]    # Show logs
./scripts/dev-setup.sh shell <service>   # Open shell in container
./scripts/dev-setup.sh clean             # Clean containers and volumes

Troubleshooting

Permission denied (Linux, Docker dev bind mounts)

docker-compose.dev.yaml runs sirius-ui and sirius-api as UID/GID 1001 (matching the image users). If you cloned the repo as root or files under sirius-ui/ / sirius-api/ are root-owned, processes inside the container get EACCES when Next.js or go build writes into mounted directories.

Preferred: fix ownership on the host (adjust path to your checkout):

sudo chown -R 1001:1001 sirius-ui sirius-api
# If you mount minor-projects into those containers:
sudo chown -R 1001:1001 ../minor-projects/go-api ../minor-projects/app-system-monitor ../minor-projects/app-administrator

Quick dev-only workaround: in .env set:

SIRIUS_DEV_CONTAINER_UID=0
SIRIUS_DEV_CONTAINER_GID=0

Then recreate containers: docker compose -f docker-compose.yaml -f docker-compose.dev.yaml up --build -d --force-recreate. This runs those two services as root inside the container (acceptable for a local lab, not for production images).

Volume Mounts Not Working

# Check if local file exists
ls -la docker-compose.local.yaml

# Check if repositories exist
ls -la ../minor-projects/

# Verify you're using start-extended
./scripts/dev-setup.sh start-extended

Git Commits Being Rejected

Your CI/CD is preventing commits with uncommented volume mounts:

# Fix automatically
git add docker-compose.override.yaml
git commit  # Pre-commit hook will fix and re-stage

# Or fix manually - comment out volume mounts with #
nano docker-compose.override.yaml

Services Not Starting

# Ensure installer-generated .env exists and is up to date
docker compose -f docker-compose.installer.yaml run --rm sirius-installer

# Check container status
./scripts/dev-setup.sh status

# View logs for specific service
./scripts/dev-setup.sh logs sirius-engine

# Restart clean
./scripts/dev-setup.sh stop
./scripts/dev-setup.sh clean
./scripts/dev-setup.sh start

Best Practices

1. Use standard mode for most development work
2. Only use extended mode when working on engine/scanner/terminal/agent code
3. Never commit docker-compose.local.yaml
4. Always test your changes work in standard mode before committing
5. Keep local overrides minimal - only uncomment what you're actively developing

Migration from Old Setup

If you were previously editing docker-compose.override.yaml directly:

# 1. Reset override file to clean state
git checkout docker-compose.override.yaml

# 2. Set up new local overrides
./scripts/dev-setup.sh init

# 3. Move your customizations to docker-compose.local.yaml
nano docker-compose.local.yaml

# 4. Start with new system
./scripts/dev-setup.sh start-extended

CI/CD Integration

The repository includes automatic validation:

• GitHub Actions: Validates docker-compose files on PRs
• Pre-commit Hook: Auto-fixes volume mount issues
• Git Ignore: Prevents local files from being committed

This ensures the repository stays clean and deployments are predictable.