chore: import upstream snapshot with attribution

2026-07-13 12:32:25 +08:00
commit 161ef94b4f
708 changed files with 189571 additions and 0 deletions
@@ -0,0 +1,260 @@
+---
+title: "Sirius Development Setup"
+description: "Comprehensive guide for setting up and using the Sirius development environment, including standard and extended development modes with local repository integration."
+template: "TEMPLATE.documentation-standard"
+version: "1.0.0"
+last_updated: "2026-04-07"
+author: "Development Team"
+tags: ["development", "setup", "docker", "workflow"]
+categories: ["development", "setup"]
+difficulty: "intermediate"
+prerequisites: ["docker", "git"]
+related_docs:
+  - "README.container-testing.md"
+  - "ABOUT.documentation.md"
+dependencies:
+  - "docker-compose.yaml"
+  - "scripts/dev-setup.sh"
+llm_context: "medium"
+search_keywords:
+  ["development setup", "docker compose", "local development", "workflow"]
+---
+
+# Sirius Development Setup
+
+## Quick Start
+
+**New in v1.0.0**: Use the improved environment switching system for easier development.
+
+```bash
+# Generate/merge required .env values once (installer-first)
+docker compose -f docker-compose.installer.yaml run --rm sirius-installer
+
+# Modern development workflow (recommended)
+./scripts/switch-env.sh dev
+
+# Legacy development (uses built-in repositories)
+./scripts/dev-setup.sh start
+
+# Extended development (with local repository mounts)
+./scripts/dev-setup.sh init           # Create local overrides
+./scripts/dev-setup.sh start-extended # Start with local mounts
+```
+
+> **Note**: For the complete developer experience, see [README.developer-guide.md](README.developer-guide.md) for comprehensive development workflows and best practices.
+
+## Development Modes
+
+### 🎯 Standard Development
+
+**Best for: UI/API development, testing, most development tasks**
+
+```bash
+./scripts/dev-setup.sh start
+```
+
+- Uses built-in repositories from Docker images
+- No local repository setup required
+- All services start with `go run` in development mode
+- Live reloading for UI changes
+- Uses secrets generated by `docker-compose.installer.yaml` (`.env`) for service auth/DB consistency
+
+### 🔧 Extended Development
+
+**Best for: Working on scanner, terminal, or agent code**
+
+```bash
+# 1. Set up local repositories (one-time)
+mkdir -p ../minor-projects && cd ../minor-projects
+git clone https://github.com/SiriusScan/app-scanner.git
+git clone https://github.com/SiriusScan/app-terminal.git
+git clone https://github.com/SiriusScan/app-agent.git
+cd ../Sirius
+
+# 2. Initialize local overrides
+./scripts/dev-setup.sh init
+
+# 3. Edit docker-compose.local.yaml (uncomment what you need)
+nano docker-compose.local.yaml
+
+# 4. Start extended development
+./scripts/dev-setup.sh start-extended
+```
+
+**Hot reloading in extended mode (engine services):**
+
+Each minor-project ships its own `.air.toml` (`app-agent/.air.toml`, `app-terminal/.air.toml`, `app-scanner/.air.toml`). When `GO_ENV=development` (the default for `docker-compose.dev.yaml`), `start-enhanced.sh` checks for `.air.toml` in the bind-mounted source and uses [Air](https://github.com/air-verse/air) for live rebuilds; if `.air.toml` is absent (or `air` is missing from the image) it falls back to plain `go run`. The matrix:
+
+| Service | Bind mount path in container | Live-reload tool | Entry point |
+| --- | --- | --- | --- |
+| `app-agent` | `/app-agent` | `air` | `cmd/server/main.go` |
+| `app-terminal` | `/app-terminal` | `air` | `cmd/main.go` |
+| `app-scanner` | `/app-scanner` | `air` (`CGO_ENABLED=1`) | `main.go` |
+
+Notes:
+
+- Changes to bind-mounted source are picked up automatically by Air; you should see a rebuild log line in `docker logs sirius-engine` within ~1s of saving.
+- `app-scanner` requires CGO and `libpcap`; the container ships both, so the in-container Air build works out of the box.
+- The orphan `sirius-engine/.air.toml` that previously lived in the engine image was removed in the April 2026 dev-mode overhaul; per-service `.air.toml` is the only supported configuration.
+- Changes are not tracked by the main Sirius repository (the bind mounts are git-ignored).
+
+**When Air isn't enough — hot-swap from local source:**
+
+Sometimes you want to validate a change against a *production-mode* engine without waiting for a full container rebuild (e.g. you're testing a binary against `docker-compose.prod.yaml`, or you want to confirm a release-style build before bumping a pin). Use the root `Makefile` targets:
+
+```bash
+make engine-mode                # show whether the running engine is dev or prod
+make agent-hot-swap-from-local
+make terminal-hot-swap-from-local
+make scanner-hot-swap-from-local      # CGO; fails cleanly if host lacks libpcap
+make engine-rebuild-from-local        # all three, best-effort
+```
+
+What they do: cross-compile the binary on the host (`linux/$(uname -m)` by default; override with `HOST_GOARCH=amd64`), `docker cp` it into the engine container at the production binary path (e.g. `/app-agent-src/server`), and restart the engine.
+
+Important guard rails:
+
+- **The targets refuse to run in dev mode** (`GO_ENV=development`). In dev, `/app-terminal` and `/app-scanner` are bind-mounted, so `docker cp` would write through to your host repo; and Air owns the live binary at `./tmp/main`, so the swapped production-path binary wouldn't even be executed. The right loop in dev is *edit source → Air rebuilds*. Override with `FORCE_HOT_SWAP=1` only if you know what you want.
+- **Scanner cross-compile may fail on macOS hosts**. The scanner uses `cgo` + `libpcap`; the macOS toolchain can't satisfy linux cgo headers. The target prints the actual `cgo` error and a copy-pasteable in-container build command. This is expected and not a Makefile bug.
+- **These targets are an inner-loop convenience, not a release path.** Once the change is good, push it to the relevant minor-project and bump the engine pin per [README.engine-component-pinning.md](architecture/README.engine-component-pinning.md). The CI pipeline is the single source of truth for shipped images.
+
+## File Structure
+
+```
+Sirius/
+├── docker-compose.yaml                 # Base configuration
+├── docker-compose.override.yaml        # 🔒 Committed: Safe development defaults
+├── docker-compose.local.example.yaml   # 🔒 Committed: Template for local overrides
+├── docker-compose.local.yaml           # 🚫 Git-ignored: Your personal overrides
+└── scripts/dev-setup.sh               # 🔒 Committed: Development helper
+```
+
+## Safety Features
+
+### 🛡️ Prevents Accidental Commits
+
+- `docker-compose.local.yaml` is git-ignored
+- CI/CD validates that volume mounts stay commented in `docker-compose.override.yaml`
+- Pre-commit hook auto-fixes uncommented volume mounts
+
+### 🔧 Developer-Friendly
+
+- Easy setup with `./scripts/dev-setup.sh init`
+- Template file shows all available options
+- Helper script for common development tasks
+
+## Available Commands
+
+```bash
+./scripts/dev-setup.sh init              # Create local overrides from template
+./scripts/dev-setup.sh start             # Standard development mode
+./scripts/dev-setup.sh start-extended    # Extended development with local repos
+./scripts/dev-setup.sh stop              # Stop all services
+./scripts/dev-setup.sh status            # Show container status
+./scripts/dev-setup.sh logs [service]    # Show logs
+./scripts/dev-setup.sh shell <service>   # Open shell in container
+./scripts/dev-setup.sh clean             # Clean containers and volumes
+```
+
+## Troubleshooting
+
+### Permission denied (Linux, Docker dev bind mounts)
+
+`docker-compose.dev.yaml` runs **sirius-ui** and **sirius-api** as UID/GID **1001** (matching the image users). If you cloned the repo as **root** or files under `sirius-ui/` / `sirius-api/` are root-owned, processes inside the container get **EACCES** when Next.js or `go build` writes into mounted directories.
+
+**Preferred:** fix ownership on the host (adjust path to your checkout):
+
+```bash
+sudo chown -R 1001:1001 sirius-ui sirius-api
+# If you mount minor-projects into those containers:
+sudo chown -R 1001:1001 ../minor-projects/go-api ../minor-projects/app-system-monitor ../minor-projects/app-administrator
+```
+
+**Quick dev-only workaround:** in `.env` set:
+
+```bash
+SIRIUS_DEV_CONTAINER_UID=0
+SIRIUS_DEV_CONTAINER_GID=0
+```
+
+Then recreate containers: `docker compose -f docker-compose.yaml -f docker-compose.dev.yaml up --build -d --force-recreate`. This runs those two services as root inside the container (acceptable for a local lab, not for production images).
+
+### Volume Mounts Not Working
+
+```bash
+# Check if local file exists
+ls -la docker-compose.local.yaml
+
+# Check if repositories exist
+ls -la ../minor-projects/
+
+# Verify you're using start-extended
+./scripts/dev-setup.sh start-extended
+```
+
+### Git Commits Being Rejected
+
+Your CI/CD is preventing commits with uncommented volume mounts:
+
+```bash
+# Fix automatically
+git add docker-compose.override.yaml
+git commit  # Pre-commit hook will fix and re-stage
+
+# Or fix manually - comment out volume mounts with #
+nano docker-compose.override.yaml
+```
+
+### Services Not Starting
+
+```bash
+# Ensure installer-generated .env exists and is up to date
+docker compose -f docker-compose.installer.yaml run --rm sirius-installer
+
+# Check container status
+./scripts/dev-setup.sh status
+
+# View logs for specific service
+./scripts/dev-setup.sh logs sirius-engine
+
+# Restart clean
+./scripts/dev-setup.sh stop
+./scripts/dev-setup.sh clean
+./scripts/dev-setup.sh start
+```
+
+## Best Practices
+
+1. **Use standard mode** for most development work
+2. **Only use extended mode** when working on engine/scanner/terminal/agent code
+3. **Never commit** `docker-compose.local.yaml`
+4. **Always test** your changes work in standard mode before committing
+5. **Keep local overrides minimal** - only uncomment what you're actively developing
+
+## Migration from Old Setup
+
+If you were previously editing `docker-compose.override.yaml` directly:
+
+```bash
+# 1. Reset override file to clean state
+git checkout docker-compose.override.yaml
+
+# 2. Set up new local overrides
+./scripts/dev-setup.sh init
+
+# 3. Move your customizations to docker-compose.local.yaml
+nano docker-compose.local.yaml
+
+# 4. Start with new system
+./scripts/dev-setup.sh start-extended
+```
+
+## CI/CD Integration
+
+The repository includes automatic validation:
+
+- **GitHub Actions**: Validates docker-compose files on PRs
+- **Pre-commit Hook**: Auto-fixes volume mount issues
+- **Git Ignore**: Prevents local files from being committed
+
+This ensures the repository stays clean and deployments are predictable.