siriusscan--sirius/documentation/dev/apps/agent/README.agent-system.md

title, description, template, version, last_updated, author, tags, categories, difficulty, prerequisites, related_docs, dependencies, llm_context, search_keywords

title	description	template	version	last_updated	author	tags	categories	difficulty	prerequisites	related_docs	dependencies	llm_context	search_keywords
Agent System Architecture	Comprehensive architecture documentation for the Sirius distributed agent system, including gRPC communication, template synchronization, and vulnerability detection	TEMPLATE.documentation-standard	1.0.0	2025-10-25	Sirius Platform Team	agent grpc architecture distributed-system template-system vulnerability-detection	architecture agent-system distributed-systems	advanced	Basic understanding of gRPC and Protocol Buffers Familiarity with Go programming language Understanding of distributed system concepts Knowledge of Docker and containerization	README.docker-architecture.md README.development.md ../README.developer-guide.md ../apps/README.agent-template-api.md	app-agent/ sirius-api/ proto/hello/	high	agent grpc distributed template vulnerability detection synchronization bidirectional-stream protobuf

Agent System Architecture

Overview

The Sirius Agent System is a distributed architecture for managing remote security agents that perform vulnerability detection and system scanning across heterogeneous environments. The system uses gRPC bidirectional streaming for real-time communication, a template-based detection engine, and a sophisticated synchronization mechanism for distributing vulnerability signatures.

System Components

┌──────────────────────────────────────────────────────┐
│                  Sirius Platform                      │
│  ┌────────────────┐      ┌────────────────┐         │
│  │  sirius-api    │◄────►│  sirius-ui     │         │
│  │  (Go/Fiber)    │      │  (Next.js)     │         │
│  └────────┬───────┘      └────────────────┘         │
│           │                                           │
│           │ RabbitMQ (Command Queue)                 │
│           ▼                                           │
│  ┌────────────────┐                                  │
│  │  Agent Server  │◄──────── Valkey/Redis           │
│  │  (gRPC)        │         (Template Storage)       │
│  └────────┬───────┘                                  │
│           │ Bidirectional gRPC Stream                │
│           │                                           │
└───────────┼───────────────────────────────────────────┘
            │
    ┌───────┴───────┬──────────┬──────────┐
    │               │          │          │
    ▼               ▼          ▼          ▼
┌────────┐    ┌────────┐  ┌────────┐  ┌────────┐
│ Agent  │    │ Agent  │  │ Agent  │  │ Agent  │
│ Linux  │    │ Win    │  │ macOS  │  │ Remote │
└────────┘    └────────┘  └────────┘  └────────┘

Core Architecture

1. Agent Server

Location: app-agent/cmd/server/ and app-agent/internal/server/

Purpose: Central hub for agent management, command distribution, and template synchronization.

Server Components

type Server struct {
    // Core services
    logger  *zap.Logger
    config  *config.ServerConfig
    server  *grpc.Server

    // Agent management
    agentsMutex sync.RWMutex
    agents      map[string]pb.HelloService_ConnectStreamServer

    // Command tracking
    commandsMutex sync.RWMutex
    commands      map[string]*CommandStatus

    // Template system
    templateManager *ServerTemplateManager
    valkeyClient    valkey.Client

    // Response storage
    responseStore store.ResponseStore

    // Pending command correlation
    pendingCommandsMutex sync.Mutex
    pendingCommands      map[string]string
}

Key Responsibilities

1. Agent Connection Management

   • Accept and maintain bidirectional gRPC streams
   • Track connected agents with unique IDs
   • Handle agent disconnections gracefully
   • Monitor agent health via heartbeats

2. Command Distribution

   • Listen to RabbitMQ for commands from UI
   • Route commands to appropriate agents
   • Track command execution status
   • Store results in Valkey for UI retrieval

3. Template Synchronization

   • Sync templates from GitHub repository
   • Store templates in Valkey
   • Distribute templates to agents
   • Handle custom template uploads

4. State Management

   • Maintain agent connection state
   • Track command execution lifecycle
   • Coordinate template versioning
   • Manage concurrent access safely

2. Agent Client

Location: app-agent/cmd/agent/ and app-agent/internal/agent/

Purpose: Remote endpoint that executes commands and performs vulnerability scans.

Agent Components

type Agent struct {
    // gRPC communication
    logger    *zap.Logger
    config    *config.AgentConfig
    conn      *grpc.ClientConn
    client    pb.HelloServiceClient
    stream    pb.HelloService_ConnectStreamClient

    // Agent information
    startTime time.Time
    agentInfo commands.AgentInfo

    // Scripting support
    powerShellPath   string
    scriptingEnabled bool

    // Template management
    syncManager *templateagent.AgentSyncManager
}

Key Responsibilities

1. Server Communication

   • Establish bidirectional gRPC stream
   • Send periodic heartbeats with system metrics
   • Receive and process server messages
   • Handle connection failures and reconnection

2. Command Execution

   • Process internal commands (status, help, scan)
   • Execute shell commands via PowerShell/bash
   • Capture command output and errors
   • Report execution results to server

3. Template Synchronization

   • Request template updates from server
   • Receive and cache templates locally
   • Validate template checksums
   • Maintain local template manifest

4. Vulnerability Scanning

   • Load and parse cached templates
   • Execute template detection steps
   • Evaluate detection logic
   • Report scan results

Communication Protocol

gRPC Service Definition

service HelloService {
  // Health check
  rpc Ping(PingRequest) returns (PingResponse) {}

  // Bidirectional streaming for real-time communication
  rpc ConnectStream(stream AgentMessage) returns (stream ServerMessage) {}
}

enum MessageType {
  UNKNOWN = 0;
  HEARTBEAT = 1;              // Agent → Server: Health status
  COMMAND = 2;                // Server → Agent: Execute command
  RESULT = 3;                 // Agent → Server: Command result
  ACKNOWLEDGMENT = 4;         // Server → Agent: Confirm receipt
  TEMPLATE_UPDATE = 5;        // Server → Agent: Template data
  TEMPLATE_SYNC_REQUEST = 6;  // Agent → Server: Request templates
}

Message Flow Patterns

Command Execution Flow

1. UI/API → RabbitMQ
   └─ Queue: agent_commands
   └─ Message: {action, command, agentId, userId, timestamp}

2. Server consumes from RabbitMQ
   └─ Validates agent is connected
   └─ Sends ACK to agent_response queue
   └─ Generates commandId = "agentId:timestamp"
   └─ Stores in pendingCommands map
   └─ Forwards to agent via gRPC stream

3. Agent receives COMMAND message
   └─ Checks internal command registry
   └─ If not found, executes as shell command
   └─ Captures stdout, stderr, exit code
   └─ Sends RESULT message to server

4. Server receives RESULT message
   └─ Looks up commandId from pendingCommands
   └─ Creates CommandResponse object
   └─ Stores in Valkey: key = "cmd:result:{commandId}"
   └─ Logs to file
   └─ Sends ACKNOWLEDGMENT to agent

5. UI/API polls Valkey
   └─ Retrieves result using commandId
   └─ Displays to user

Template Sync Flow

1. Agent sends TEMPLATE_SYNC_REQUEST
   └─ Includes lastSync timestamp
   └─ Sent via gRPC stream

2. Server processes sync request
   └─ Queries Valkey for templates updated after lastSync
   └─ Builds TemplateManifest (metadata only)
   └─ Sends manifest as first message

3. Server streams templates
   └─ For each template:
       ├─ Retrieves content from Valkey
       ├─ Calculates checksum
       ├─ Creates TEMPLATE_UPDATE message
       └─ Sends via stream

4. Agent receives templates
   └─ For each TEMPLATE_UPDATE:
       ├─ Writes to cache directory (atomic)
       ├─ Verifies checksum
       ├─ Updates local manifest
       └─ Ready for scanning

5. Agent completes sync
   └─ Updates lastSync timestamp
   └─ Templates available for execution

Heartbeat Flow

Agent (every 30 seconds):
  └─ Collects system metrics (CPU, memory)
  └─ Sends HEARTBEAT message
  └─ Includes timestamp, metrics

Server receives HEARTBEAT:
  └─ Updates agent last-seen time
  └─ Logs metrics (debug level)
  └─ No response needed (fire-and-forget)

Template System Architecture

Template Structure

The template system uses YAML-based vulnerability definitions that are platform-agnostic and module-driven.

Template Definition

id: apache-outdated
info:
  name: Outdated Apache HTTP Server
  author: security-team
  severity: high
  description: Detects Apache versions with known CVEs
  cve:
    - CVE-2021-44228
  tags: [apache, web-server, cve]
  version: "1.0"

detection:
  logic: all # all (AND) or any (OR)
  steps:
    - type: version-cmd
      platforms: [linux, darwin]
      weight: 1.0
      config:
        command: "httpd -v"
        version_regex: "Apache/(\\d+\\.\\d+\\.\\d+)"
        vulnerable_versions:
          - "< 2.4.52"

    - type: file-content
      platforms: [linux, darwin]
      weight: 0.8
      config:
        path: "/etc/apache2/apache2.conf"
        patterns: ["ServerTokens.*Full"]

Template Execution Engine

Location: app-agent/internal/template/executor/

┌─────────────────────────────────────────────┐
│         Template Executor                    │
├─────────────────────────────────────────────┤
│                                              │
│  1. Filter Steps by Platform                │
│     └─ Current OS: linux/darwin/windows     │
│                                              │
│  2. Execute Steps Sequentially              │
│     ├─ Get module from registry             │
│     ├─ Create timeout context               │
│     ├─ Execute module                       │
│     └─ Collect results                      │
│                                              │
│  3. Evaluate Detection Logic                │
│     ├─ logic: all → All steps match (AND)   │
│     └─ logic: any → Any step matches (OR)   │
│                                              │
│  4. Calculate Confidence                    │
│     ├─ logic: all → Min(weights)            │
│     └─ logic: any → Max(weights)            │
│                                              │
│  5. Build Result                            │
│     └─ {matched, confidence, steps, errors} │
│                                              │
└─────────────────────────────────────────────┘

Detection Modules

Detection modules are pluggable components that implement specific detection mechanisms.

Module Interface

type Module interface {
    Name() string
    Execute(ctx context.Context, config map[string]interface{}) (*ModuleResult, error)
}

type ModuleResult struct {
    Matched  bool
    Evidence map[string]interface{}
    Error    string
}

Available Modules

Module	Type	Purpose	Platforms
version-cmd	Command	Parse version from command	All
file-hash	File System	Check file SHA256 hash	All
file-content	File System	Search file for patterns	All
registry-key	Windows	Check registry keys/values	Windows only
service-check	System	Verify service exists/runs	All
config-check	Configuration	Parse and validate configs	All
script-exec	Custom	Execute custom detection	Platform-specific

Module Registration

// Module implementation
type FileHashModule struct{}

func (m *FileHashModule) Name() string {
    return "file-hash"
}

func (m *FileHashModule) Execute(ctx context.Context, config map[string]interface{}) (*ModuleResult, error) {
    // Implementation
}

// Registration (in init())
func init() {
    registry.Register(&FileHashModule{})
}

Template Storage and Distribution

Server-Side Storage (Valkey)

Template Keys Structure:

  template:manifest
    └─ Global manifest JSON with all template metadata

  template:meta:<template-id>
    └─ Individual template metadata JSON
    └─ {id, version, checksum, size, severity, platforms, etc.}

  template:standard:<template-id>
    └─ Standard template YAML content
    └─ From sirius-agent-modules GitHub repository

  template:custom:<template-id>
    └─ Custom template YAML content
    └─ User-uploaded via UI

Agent-Side Cache

Cache Directory Structure:

Windows:  C:\ProgramData\Sirius\templates\
Linux:    /var/lib/sirius/templates/
macOS:    /Library/Application Support/Sirius/templates/

Contents:
  ├── cache-manifest.json          # Local cache metadata
  │   └─ {version, lastSync, templates{}, statistics{}}
  │
  ├── standard/                    # Standard templates
  │   ├── apache-outdated.yaml
  │   ├── nginx-vuln.yaml
  │   └── ssh-weak-ciphers.yaml
  │
  └── custom/                      # Custom templates
      └── org-policy-check.yaml

Template Synchronization Manager

Server Side (internal/server/template_manager.go):

type ServerTemplateManager struct {
    valkeyClient valkey.Client
    storage      *templatevalkey.ValKeyTemplateStorage
    githubSync   *templatevalkey.GitHubSyncManager
    logger       *zap.Logger
    config       *TemplateConfig
    server       *Server
}

// Key Methods:
// - SyncFromGitHub() → Pull from sirius-agent-modules
// - GetTemplatesForSync() → Prepare templates for agent
// - StoreCustomTemplate() → Handle user uploads
// - ValidateTemplate() → Security and syntax checks

Agent Side (internal/template/agent/sync_manager.go):

type AgentSyncManager struct {
    cacheDir   string
    logger     *zap.Logger
    serverURL  string
    agentID    string
    grpcStream pb.HelloService_ConnectStreamClient
}

// Key Methods:
// - SyncFromServer() → Request template sync
// - HandleTemplateUpdate() → Process incoming templates
// - LoadTemplates() → Load cached templates for scanning
// - ValidateCache() → Verify template checksums

Command System

Command Architecture

The agent supports both internal commands (built into the agent) and shell commands (executed via PowerShell/bash).

┌─────────────────────────────────────────────┐
│         Command Dispatcher                   │
├─────────────────────────────────────────────┤
│                                              │
│  1. Receive Command String                  │
│     └─ From server via gRPC stream          │
│                                              │
│  2. Parse Command                           │
│     ├─ Extract command name                 │
│     └─ Parse arguments                      │
│                                              │
│  3. Check Command Registry                  │
│     ├─ Internal command? → Execute handler  │
│     └─ Not found? → Fall back to shell      │
│                                              │
│  4. Execute                                 │
│     ├─ Internal: Call Go function           │
│     └─ Shell: Execute via PowerShell/bash   │
│                                              │
│  5. Capture Result                          │
│     ├─ stdout, stderr                       │
│     ├─ exit code                            │
│     └─ execution time                       │
│                                              │
│  6. Send Result to Server                   │
│     └─ RESULT message via gRPC stream       │
│                                              │
└─────────────────────────────────────────────┘

Internal Commands

Command	Purpose	Example
help	List available commands	help
status	Agent status and capabilities	status
scan	Execute vulnerability scan	scan --targets=192.168.1.0/24
template list	List cached templates	template list
template sync	Trigger template synchronization	template sync
templatescan	Run specific template	templatescan --template=apache-001

Command Registration

// Command handler signature
type CommandHandler func(ctx context.Context, info AgentInfo, args []string) (string, error)

// Registration
func Register(name string, handler CommandHandler) {
    commandsMutex.Lock()
    defer commandsMutex.Unlock()
    commands[name] = handler
}

// Dispatch
func Dispatch(ctx context.Context, info AgentInfo, commandString string) (string, error) {
    cmdName, args := parseCommandLine(commandString)

    handler, exists := commands[cmdName]
    if !exists {
        return "", ErrUnknownCommand
    }

    return handler(ctx, info, args)
}

Shell Command Execution

When a command is not found in the internal registry, the agent attempts to execute it as a shell command.

PowerShell Execution (Windows)

func ExecuteScript(ctx context.Context, psPath string, scriptContent string) (stdout, stderr string, exitCode int, err error) {
    // Create temp script file
    tempFile := filepath.Join(os.TempDir(), fmt.Sprintf("sirius-script-%d.ps1", time.Now().UnixNano()))
    os.WriteFile(tempFile, []byte(scriptContent), 0600)
    defer os.Remove(tempFile)

    // Execute via PowerShell
    cmd := exec.CommandContext(ctx, psPath, "-NoProfile", "-ExecutionPolicy", "Bypass", "-File", tempFile)

    // Capture output
    var stdoutBuf, stderrBuf bytes.Buffer
    cmd.Stdout = &stdoutBuf
    cmd.Stderr = &stderrBuf

    err = cmd.Run()

    return stdoutBuf.String(), stderrBuf.String(), cmd.ProcessState.ExitCode(), err
}

Bash Execution (Linux/macOS)

func ExecuteCommand(ctx context.Context, command string) (stdout, stderr string, exitCode int, err error) {
    cmd := exec.CommandContext(ctx, "/bin/bash", "-c", command)

    var stdoutBuf, stderrBuf bytes.Buffer
    cmd.Stdout = &stdoutBuf
    cmd.Stderr = &stderrBuf

    err = cmd.Run()

    return stdoutBuf.String(), stderrBuf.String(), cmd.ProcessState.ExitCode(), err
}

Data Storage Architecture

Valkey (Redis) Schema

The system uses Valkey for distributed state management and caching.

Key Patterns

Command Results:
  cmd:result:{agentId}:{timestamp}
    └─ CommandResponse JSON
    └─ TTL: 1 hour
    └─ Fields: {commandId, agentId, command, status, output, error, exitCode}

Template Storage:
  template:manifest
    └─ Global manifest with statistics

  template:meta:{templateId}
    └─ Template metadata

  template:standard:{templateId}
    └─ Standard template YAML

  template:custom:{templateId}
    └─ Custom template YAML

Agent Metadata (future):
  agent:meta:{agentId}
    └─ Agent information
    └─ {id, hostname, platform, version, capabilities, lastSeen}

RabbitMQ Message Queues

Queue Definitions

agent_commands
  └─ Purpose: Commands from UI/API to agents
  └─ Format: JSON
  └─ Schema: {action, command, agentId, userId, timestamp, target}
  └─ Consumers: Agent Server

agent_response
  └─ Purpose: Acknowledgments and responses to UI/API
  └─ Format: JSON
  └─ Schema: {success, message, output, error}
  └─ Consumers: sirius-api

Message Examples

Command Message:

{
  "action": "",
  "command": "systeminfo",
  "agentId": "",
  "userId": "user-123",
  "timestamp": "1698765432000",
  "target": {
    "type": "agent",
    "id": "agent-001"
  }
}

Response Message:

{
  "success": true,
  "message": "Command received, forwarding to agent",
  "output": "",
  "error": ""
}

Security Architecture

Authentication and Authorization

Agent Authentication

1. Agent Registration:
   ├─ Agent ID generated on first startup
   ├─ ID stored in agent config
   └─ Server validates agent ID on connection

2. gRPC Metadata:
   ├─ agent_id: Unique identifier
   ├─ scripting_enabled: Capability flag
   └─ Future: JWT tokens, TLS certificates

3. Stream Authorization:
   ├─ Server maintains whitelist of allowed agent IDs
   ├─ Validates agent ID on ConnectStream
   └─ Rejects unauthorized connections

Command Authorization

1. Command Source Validation:
   ├─ Commands only accepted from RabbitMQ
   ├─ Server validates target agent exists
   └─ Server checks agent is connected

2. Agent-Side Validation:
   ├─ Only processes commands from established stream
   ├─ Validates command format
   └─ Rejects malformed commands

3. Future Enhancements:
   ├─ Role-based access control (RBAC)
   ├─ Command whitelisting per agent
   └─ Audit logging of all commands

Template Security

Template Validation

// Server-side validation before storage
func (tm *ServerTemplateManager) ValidateTemplate(template *types.Template, content []byte) error {
    // 1. Syntax validation
    if err := parser.ParseTemplate(template); err != nil {
        return fmt.Errorf("syntax error: %w", err)
    }

    // 2. Security scan - dangerous patterns
    dangerousPatterns := []string{
        "eval(", "exec(", "system(",
        "shell_exec(", "passthru(",
    }
    for _, pattern := range dangerousPatterns {
        if strings.Contains(string(content), pattern) {
            return fmt.Errorf("dangerous pattern detected: %s", pattern)
        }
    }

    // 3. Script injection check
    scriptPatterns := []string{
        "<script", "javascript:", "vbscript:",
        "onload=", "onerror=",
    }
    for _, pattern := range scriptPatterns {
        if strings.Contains(strings.ToLower(string(content)), pattern) {
            return fmt.Errorf("script injection pattern: %s", pattern)
        }
    }

    // 4. Size limit
    if len(content) > MaxTemplateSize {
        return fmt.Errorf("template exceeds size limit")
    }

    return nil
}

Checksum Verification

1. Server calculates SHA256 on storage:
   └─ checksum = sha256(templateContent)

2. Checksum included in TEMPLATE_UPDATE:
   └─ {templateId, version, checksum, content}

3. Agent verifies on receipt:
   ├─ Calculate sha256(received content)
   ├─ Compare with provided checksum
   └─ Reject if mismatch

Network Security

1. gRPC Transport Security:
   ├─ TLS encryption (configurable)
   ├─ Certificate validation
   └─ Mutual TLS support (future)

2. RabbitMQ Security:
   ├─ Authentication required
   ├─ Per-queue permissions
   └─ TLS connections

3. Valkey Security:
   ├─ Password authentication
   ├─ Network isolation
   └─ Redis ACLs (future)

Deployment Architecture

Container Architecture

┌─────────────────────────────────────────────────────┐
│                Docker Network: sirius                │
│                                                       │
│  ┌──────────────┐  ┌──────────────┐  ┌───────────┐ │
│  │   sirius-ui  │  │  sirius-api  │  │  postgres │ │
│  │  (Next.js)   │  │  (Go/Fiber)  │  │           │ │
│  │  Port: 3000  │  │  Port: 8080  │  │ Port:5432 │ │
│  └──────────────┘  └──────┬───────┘  └───────────┘ │
│                            │                         │
│  ┌──────────────┐  ┌──────▼───────┐  ┌───────────┐ │
│  │   valkey     │  │  rabbitmq    │  │ sirius-   │ │
│  │  (Redis)     │  │              │  │ engine    │ │
│  │  Port: 6379  │  │ Port: 5672   │  │ (agent)   │ │
│  └──────────────┘  └──────────────┘  └───────────┘ │
│                                                       │
│  ┌─────────────────────────────────────────────────┐│
│  │         Agent Server (gRPC)                      ││
│  │         Port: 50051                              ││
│  │  Mounts: app-agent/ → /app-agent                ││
│  └─────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────┘
                            │
                            │ gRPC (external)
                            │
            ┌───────────────┴────────────────┐
            │                                 │
            ▼                                 ▼
    ┌──────────────┐                 ┌──────────────┐
    │ Remote Agent │                 │ Remote Agent │
    │  (Linux)     │                 │  (Windows)   │
    │              │                 │              │
    │ External IP  │                 │ External IP  │
    └──────────────┘                 └──────────────┘

Agent Server Deployment

Development Mode (sirius-engine container):

services:
  sirius-engine:
    build:
      context: sirius-engine
      dockerfile: Dockerfile
    volumes:
      - ./app-agent:/app-agent # Dev mode hot-reload
    ports:
      - "50051:50051" # gRPC port
    environment:
      - VALKEY_ADDRESS=valkey:6379
      - RABBITMQ_URL=amqp://guest:guest@rabbitmq:5672/
    depends_on:
      - valkey
      - rabbitmq

Production Mode:

services:
  agent-server:
    image: sirius-agent-server:latest
    ports:
      - "50051:50051"
    environment:
      - SERVER_ADDRESS=0.0.0.0:50051
      - VALKEY_ADDRESS=valkey:6379
      - RABBITMQ_URL=amqp://user:pass@rabbitmq:5672/
      - LOG_LEVEL=info
    restart: unless-stopped

Remote Agent Deployment

Linux Agent (systemd)

[Unit]
Description=Sirius Security Agent
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=sirius-agent
Group=sirius-agent
Environment="AGENT_ID=agent-linux-001"
Environment="SERVER_ADDRESS=server.example.com:50051"
Environment="ENABLE_SCRIPTING=true"
ExecStart=/usr/local/bin/sirius-agent
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Windows Agent (Service)

# Install as Windows Service
New-Service -Name "SiriusAgent" `
    -BinaryPathName "C:\Program Files\Sirius\sirius-agent.exe" `
    -DisplayName "Sirius Security Agent" `
    -StartupType Automatic `
    -Description "Sirius distributed security agent"

# Set environment variables
[Environment]::SetEnvironmentVariable("AGENT_ID", "agent-win-001", "Machine")
[Environment]::SetEnvironmentVariable("SERVER_ADDRESS", "server.example.com:50051", "Machine")
[Environment]::SetEnvironmentVariable("ENABLE_SCRIPTING", "true", "Machine")

# Start service
Start-Service -Name "SiriusAgent"

macOS Agent (launchd)

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.sirius.agent</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/sirius-agent</string>
    </array>
    <key>EnvironmentVariables</key>
    <dict>
        <key>AGENT_ID</key>
        <string>agent-mac-001</string>
        <key>SERVER_ADDRESS</key>
        <string>server.example.com:50051</string>
        <key>ENABLE_SCRIPTING</key>
        <string>true</string>
    </dict>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
</dict>
</plist>

Performance Considerations

Scalability

Horizontal Scaling

Server Scaling:
  ├─ Multiple agent server instances behind load balancer
  ├─ gRPC load balancing strategies
  ├─ Shared state in Valkey
  └─ RabbitMQ message distribution

Agent Scaling:
  ├─ Thousands of agents per server instance
  ├─ Efficient stream multiplexing
  ├─ Minimal memory footprint per agent
  └─ Connection pooling

Vertical Scaling

Server Resources:
  ├─ CPU: Moderate (gRPC is efficient)
  ├─ Memory: 2GB base + 5MB per 1000 agents
  ├─ Network: High bandwidth for template distribution
  └─ Storage: Minimal (state in Valkey)

Agent Resources:
  ├─ CPU: Low (idle), High (scanning)
  ├─ Memory: 50-100MB base
  ├─ Disk: 100MB for template cache
  └─ Network: Low (heartbeats), High (template sync)

Optimization Strategies

Template Caching

Agent-Side:
  ├─ Templates cached on disk
  ├─ Only sync changed templates
  ├─ Checksum-based validation
  └─ Last-sync timestamp tracking

Server-Side:
  ├─ Templates stored in Valkey
  ├─ Manifest cached in memory
  ├─ Incremental sync support
  └─ Compression for transfer (future)

Connection Management

gRPC Streams:
  ├─ Single bidirectional stream per agent
  ├─ Stream multiplexing (HTTP/2)
  ├─ Automatic reconnection with backoff
  └─ Heartbeat-based connection validation

Command Execution:
  ├─ Async command processing
  ├─ Non-blocking stream operations
  ├─ Context-based cancellation
  └─ Timeout enforcement

Monitoring and Observability

Metrics

Server Metrics:
  ├─ agent.connections.total (gauge)
  ├─ agent.connections.active (gauge)
  ├─ commands.executed.total (counter)
  ├─ commands.duration (histogram)
  ├─ templates.synced.total (counter)
  └─ grpc.stream.errors (counter)

Agent Metrics:
  ├─ agent.uptime (gauge)
  ├─ agent.heartbeat.sent (counter)
  ├─ commands.received (counter)
  ├─ scans.executed (counter)
  ├─ templates.cached (gauge)
  └─ memory.usage (gauge)

Logging

Structured Logging (zap):

  Server:
    ├─ agent.connected: {agentId, timestamp}
    ├─ command.sent: {commandId, agentId, command}
    ├─ command.completed: {commandId, exitCode, duration}
    └─ template.synced: {templateId, agentId}

  Agent:
    ├─ connection.established: {serverId, timestamp}
    ├─ command.received: {command, timestamp}
    ├─ command.executed: {command, exitCode, duration}
    └─ template.updated: {templateId, checksum}

Health Checks

Server Health Endpoint:
  GET /health
  └─ {
      "status": "healthy",
      "agents": {
        "total": 42,
        "connected": 40
      },
      "services": {
        "valkey": "healthy",
        "rabbitmq": "healthy"
      }
    }

Agent Health Check:
  ├─ Heartbeat interval: 30 seconds
  ├─ Server considers agent unhealthy after: 90 seconds
  └─ Agent reconnects automatically

Error Handling and Resilience

Connection Failures

Agent Reconnection Strategy:
  1. Detect connection loss (stream error)
  2. Wait with exponential backoff:
     ├─ 1st retry: 1 second
     ├─ 2nd retry: 2 seconds
     ├─ 3rd retry: 4 seconds
     └─ Max: 60 seconds
  3. Attempt reconnection
  4. Resume from last sync point
  5. Resend pending commands

Command Failures

Server Handling:
  ├─ Command timeout: 5 minutes
  ├─ Store error in CommandResponse
  ├─ Mark command as failed
  └─ Notify UI via Valkey

Agent Handling:
  ├─ Capture all errors
  ├─ Include stderr in result
  ├─ Return non-zero exit code
  └─ Continue processing other commands

Template Sync Failures

Graceful Degradation:
  ├─ Agent continues using cached templates
  ├─ Retry sync on next request
  ├─ Log sync errors for investigation
  └─ Alert on repeated failures

Future Enhancements

Planned Features

1. Enhanced Security

   • Mutual TLS authentication
   • JWT-based authorization
   • Command RBAC system
   • Encrypted template storage

2. Advanced Template Features

   • Template versioning system
   • Template dependency resolution
   • Conditional step execution
   • Dynamic module loading

3. Scalability Improvements

   • Agent clustering support
   • Multi-region deployment
   • Template CDN distribution
   • Result streaming for large scans

4. Monitoring & Analytics

   • Real-time dashboards
   • Agent performance metrics
   • Template effectiveness tracking
   • Automated alerting system

LLM Context

High-Level Architecture Understanding

This document provides a comprehensive overview of the Sirius Agent System architecture. Key areas for AI assistants to understand:

1. Bidirectional gRPC Communication: The system uses persistent streams for real-time agent communication, not request-response patterns.

2. Template-Based Detection: Vulnerability detection is driven by YAML templates with pluggable modules, not hardcoded logic.

3. Distributed State Management: Valkey stores templates and results; RabbitMQ handles command distribution; gRPC handles real-time communication.

4. Agent Autonomy: Agents cache templates locally and can operate semi-independently, syncing periodically with the server.

5. Security-First Design: Multiple validation layers for templates, commands, and connections ensure system integrity.

Code Navigation

When working with this system:

• Server code: app-agent/internal/server/
• Agent code: app-agent/internal/agent/
• Template system: app-agent/internal/template/
• Detection modules: app-agent/internal/detect/
• Protocol definitions: app-agent/proto/hello/

Common Development Tasks

• Adding detection modules: Implement Module interface, register in internal/modules/registry
• Creating templates: Follow YAML structure, test with templatescan command
• Extending protocol: Update .proto file, regenerate with ./scripts/generate_proto.sh
• Debugging communication: Enable gRPC logging, check server/agent logs, inspect Valkey keys

---

Related Documentation:

• Docker Architecture - Container deployment details
• Developer Guide - Development workflow
• Agent Template API - API integration

For Developers: See Remote Agent Engineer Role for detailed implementation guidance.

Last Updated: October 25, 2025 Version: 1.0.0 Maintained By: Sirius Platform Team