siriusscan--sirius/documentation/dev/README.development.md

---
title: "Sirius Development Setup"
description: "Comprehensive guide for setting up and using the Sirius development environment, including standard and extended development modes with local repository integration."
template: "TEMPLATE.documentation-standard"
version: "1.0.0"
last_updated: "2026-04-07"
author: "Development Team"
tags: ["development", "setup", "docker", "workflow"]
categories: ["development", "setup"]
difficulty: "intermediate"
prerequisites: ["docker", "git"]
related_docs:
  - "README.container-testing.md"
  - "ABOUT.documentation.md"
dependencies:
  - "docker-compose.yaml"
  - "scripts/dev-setup.sh"
llm_context: "medium"
search_keywords:
  ["development setup", "docker compose", "local development", "workflow"]
---

# Sirius Development Setup

## Quick Start

**New in v1.0.0**: Use the improved environment switching system for easier development.

```bash
# Generate/merge required .env values once (installer-first)
docker compose -f docker-compose.installer.yaml run --rm sirius-installer

# Modern development workflow (recommended)
./scripts/switch-env.sh dev

# Legacy development (uses built-in repositories)
./scripts/dev-setup.sh start

# Extended development (with local repository mounts)
./scripts/dev-setup.sh init           # Create local overrides
./scripts/dev-setup.sh start-extended # Start with local mounts
```

> **Note**: For the complete developer experience, see [README.developer-guide.md](README.developer-guide.md) for comprehensive development workflows and best practices.

## Development Modes

### 🎯 Standard Development

**Best for: UI/API development, testing, most development tasks**

```bash
./scripts/dev-setup.sh start
```

- Uses built-in repositories from Docker images
- No local repository setup required
- All services start with `go run` in development mode
- Live reloading for UI changes
- Uses secrets generated by `docker-compose.installer.yaml` (`.env`) for service auth/DB consistency

### 🔧 Extended Development

**Best for: Working on scanner, terminal, or agent code**

```bash
# 1. Set up local repositories (one-time)
mkdir -p ../minor-projects && cd ../minor-projects
git clone https://github.com/SiriusScan/app-scanner.git
git clone https://github.com/SiriusScan/app-terminal.git
git clone https://github.com/SiriusScan/app-agent.git
cd ../Sirius

# 2. Initialize local overrides
./scripts/dev-setup.sh init

# 3. Edit docker-compose.local.yaml (uncomment what you need)
nano docker-compose.local.yaml

# 4. Start extended development
./scripts/dev-setup.sh start-extended
```

**Hot reloading in extended mode (engine services):**

Each minor-project ships its own `.air.toml` (`app-agent/.air.toml`, `app-terminal/.air.toml`, `app-scanner/.air.toml`). When `GO_ENV=development` (the default for `docker-compose.dev.yaml`), `start-enhanced.sh` checks for `.air.toml` in the bind-mounted source and uses [Air](https://github.com/air-verse/air) for live rebuilds; if `.air.toml` is absent (or `air` is missing from the image) it falls back to plain `go run`. The matrix:

| Service | Bind mount path in container | Live-reload tool | Entry point |
| --- | --- | --- | --- |
| `app-agent` | `/app-agent` | `air` | `cmd/server/main.go` |
| `app-terminal` | `/app-terminal` | `air` | `cmd/main.go` |
| `app-scanner` | `/app-scanner` | `air` (`CGO_ENABLED=1`) | `main.go` |

Notes:

- Changes to bind-mounted source are picked up automatically by Air; you should see a rebuild log line in `docker logs sirius-engine` within ~1s of saving.
- `app-scanner` requires CGO and `libpcap`; the container ships both, so the in-container Air build works out of the box.
- The orphan `sirius-engine/.air.toml` that previously lived in the engine image was removed in the April 2026 dev-mode overhaul; per-service `.air.toml` is the only supported configuration.
- Changes are not tracked by the main Sirius repository (the bind mounts are git-ignored).

**When Air isn't enough — hot-swap from local source:**

Sometimes you want to validate a change against a *production-mode* engine without waiting for a full container rebuild (e.g. you're testing a binary against `docker-compose.prod.yaml`, or you want to confirm a release-style build before bumping a pin). Use the root `Makefile` targets:

```bash
make engine-mode                # show whether the running engine is dev or prod
make agent-hot-swap-from-local
make terminal-hot-swap-from-local
make scanner-hot-swap-from-local      # CGO; fails cleanly if host lacks libpcap
make engine-rebuild-from-local        # all three, best-effort
```

What they do: cross-compile the binary on the host (`linux/$(uname -m)` by default; override with `HOST_GOARCH=amd64`), `docker cp` it into the engine container at the production binary path (e.g. `/app-agent-src/server`), and restart the engine.

Important guard rails:

- **The targets refuse to run in dev mode** (`GO_ENV=development`). In dev, `/app-terminal` and `/app-scanner` are bind-mounted, so `docker cp` would write through to your host repo; and Air owns the live binary at `./tmp/main`, so the swapped production-path binary wouldn't even be executed. The right loop in dev is *edit source → Air rebuilds*. Override with `FORCE_HOT_SWAP=1` only if you know what you want.
- **Scanner cross-compile may fail on macOS hosts**. The scanner uses `cgo` + `libpcap`; the macOS toolchain can't satisfy linux cgo headers. The target prints the actual `cgo` error and a copy-pasteable in-container build command. This is expected and not a Makefile bug.
- **These targets are an inner-loop convenience, not a release path.** Once the change is good, push it to the relevant minor-project and bump the engine pin per [README.engine-component-pinning.md](architecture/README.engine-component-pinning.md). The CI pipeline is the single source of truth for shipped images.

## File Structure

```
Sirius/
├── docker-compose.yaml                 # Base configuration
├── docker-compose.override.yaml        # 🔒 Committed: Safe development defaults
├── docker-compose.local.example.yaml   # 🔒 Committed: Template for local overrides
├── docker-compose.local.yaml           # 🚫 Git-ignored: Your personal overrides
└── scripts/dev-setup.sh               # 🔒 Committed: Development helper
```

## Safety Features

### 🛡️ Prevents Accidental Commits

- `docker-compose.local.yaml` is git-ignored
- CI/CD validates that volume mounts stay commented in `docker-compose.override.yaml`
- Pre-commit hook auto-fixes uncommented volume mounts

### 🔧 Developer-Friendly

- Easy setup with `./scripts/dev-setup.sh init`
- Template file shows all available options
- Helper script for common development tasks

## Available Commands

```bash
./scripts/dev-setup.sh init              # Create local overrides from template
./scripts/dev-setup.sh start             # Standard development mode
./scripts/dev-setup.sh start-extended    # Extended development with local repos
./scripts/dev-setup.sh stop              # Stop all services
./scripts/dev-setup.sh status            # Show container status
./scripts/dev-setup.sh logs [service]    # Show logs
./scripts/dev-setup.sh shell <service>   # Open shell in container
./scripts/dev-setup.sh clean             # Clean containers and volumes
```

## Troubleshooting

### Permission denied (Linux, Docker dev bind mounts)

`docker-compose.dev.yaml` runs **sirius-ui** and **sirius-api** as UID/GID **1001** (matching the image users). If you cloned the repo as **root** or files under `sirius-ui/` / `sirius-api/` are root-owned, processes inside the container get **EACCES** when Next.js or `go build` writes into mounted directories.

**Preferred:** fix ownership on the host (adjust path to your checkout):

```bash
sudo chown -R 1001:1001 sirius-ui sirius-api
# If you mount minor-projects into those containers:
sudo chown -R 1001:1001 ../minor-projects/go-api ../minor-projects/app-system-monitor ../minor-projects/app-administrator
```

**Quick dev-only workaround:** in `.env` set:

```bash
SIRIUS_DEV_CONTAINER_UID=0
SIRIUS_DEV_CONTAINER_GID=0
```

Then recreate containers: `docker compose -f docker-compose.yaml -f docker-compose.dev.yaml up --build -d --force-recreate`. This runs those two services as root inside the container (acceptable for a local lab, not for production images).

### Volume Mounts Not Working

```bash
# Check if local file exists
ls -la docker-compose.local.yaml

# Check if repositories exist
ls -la ../minor-projects/

# Verify you're using start-extended
./scripts/dev-setup.sh start-extended
```

### Git Commits Being Rejected

Your CI/CD is preventing commits with uncommented volume mounts:

```bash
# Fix automatically
git add docker-compose.override.yaml
git commit  # Pre-commit hook will fix and re-stage

# Or fix manually - comment out volume mounts with #
nano docker-compose.override.yaml
```

### Services Not Starting

```bash
# Ensure installer-generated .env exists and is up to date
docker compose -f docker-compose.installer.yaml run --rm sirius-installer

# Check container status
./scripts/dev-setup.sh status

# View logs for specific service
./scripts/dev-setup.sh logs sirius-engine

# Restart clean
./scripts/dev-setup.sh stop
./scripts/dev-setup.sh clean
./scripts/dev-setup.sh start
```

## Best Practices

1. **Use standard mode** for most development work
2. **Only use extended mode** when working on engine/scanner/terminal/agent code
3. **Never commit** `docker-compose.local.yaml`
4. **Always test** your changes work in standard mode before committing
5. **Keep local overrides minimal** - only uncomment what you're actively developing

## Migration from Old Setup

If you were previously editing `docker-compose.override.yaml` directly:

```bash
# 1. Reset override file to clean state
git checkout docker-compose.override.yaml

# 2. Set up new local overrides
./scripts/dev-setup.sh init

# 3. Move your customizations to docker-compose.local.yaml
nano docker-compose.local.yaml

# 4. Start with new system
./scripts/dev-setup.sh start-extended
```

## CI/CD Integration

The repository includes automatic validation:

- **GitHub Actions**: Validates docker-compose files on PRs
- **Pre-commit Hook**: Auto-fixes volume mount issues
- **Git Ignore**: Prevents local files from being committed

This ensures the repository stays clean and deployments are predictable.