chore: import upstream snapshot with attribution

2026-07-13 13:28:13 +08:00
commit 0878425be3
1160 changed files with 491311 additions and 0 deletions
@@ -0,0 +1,688 @@
+# Docker Client Package Documentation
+
+## Table of Contents
+
+- [Overview](#overview)
+- [Architecture](#architecture)
+- [Configuration](#configuration)
+- [Core Interfaces](#core-interfaces)
+- [Container Lifecycle Management](#container-lifecycle-management)
+- [Security and Isolation](#security-and-isolation)
+- [Integration with PentAGI](#integration-with-pentagi)
+- [Usage Examples](#usage-examples)
+- [Error Handling](#error-handling)
+- [Best Practices](#best-practices)
+
+## Overview
+
+The Docker client package (`backend/pkg/docker`) provides a secure and isolated containerized environment for PentAGI's AI agents to execute penetration testing operations. This package serves as a wrapper around the official Docker SDK, offering specialized functionality for managing containers that AI agents use to perform security testing tasks.
+
+### Key Features
+
+- **Secure Isolation**: All operations are performed in sandboxed Docker containers with complete isolation
+- **AI Agent Integration**: Specifically designed to support AI agent workflows and terminal operations
+- **Container Lifecycle Management**: Comprehensive container creation, execution, and cleanup
+- **Port Management**: Automatic port allocation for flow-specific containers
+- **File Operations**: Safe file transfer, path metadata lookup, and non-recursive directory listing between host and containers
+- **Network Isolation**: Configurable network policies for security
+- **Resource Management**: Memory and CPU limits for controlled execution
+- **Volume Management**: Persistent and temporary storage solutions
+
+### Role in PentAGI Ecosystem
+
+The Docker client is a critical component that enables PentAGI's core promise of secure, isolated penetration testing. It provides the foundation for:
+
+- **Terminal Access**: AI agents execute commands in isolated environments
+- **Tool Execution**: Professional pentesting tools run in dedicated containers
+- **File Management**: Secure file operations and artifact storage
+- **Environment Preparation**: Dynamic container setup based on task requirements
+- **Resource Cleanup**: Automatic cleanup of completed or failed operations
+
+## Architecture
+
+### Core Components
+
+The Docker client package consists of several key components:
+
+```
+backend/pkg/docker/
+├── client.go          # Main Docker client implementation
+└── (future files)     # Additional Docker utilities
+```
+
+### Key Constants and Configuration
+
+```go
+const WorkFolderPathInContainer = "/work"              // Standard working directory in containers
+const BaseContainerPortsNumber = 28000                // Starting port number for dynamic allocation
+const defaultImage = "debian:latest"                  // Fallback image if custom image fails
+const containerPortsNumber = 2                        // Number of ports allocated per container
+const limitContainerPortsNumber = 2000                // Maximum port range for allocation
+const containerListWorkers = 20                       // Parallel stat workers for directory listing
+```
+
+### Port Allocation Strategy
+
+PentAGI uses a deterministic port allocation algorithm to ensure each flow gets unique, predictable ports:
+
+```go
+func GetPrimaryContainerPorts(flowID int64) []int {
+    ports := make([]int, containerPortsNumber)
+    for i := 0; i < containerPortsNumber; i++ {
+        delta := (int(flowID)*containerPortsNumber + i) % limitContainerPortsNumber
+        ports[i] = BaseContainerPortsNumber + delta
+    }
+    return ports
+}
+```
+
+This ensures that:
+- Each flow gets consistent port numbers across restarts
+- Port conflicts are avoided between different flows
+- Ports are within a controlled range (28000-30000)
+
+## Configuration
+
+### Environment Variables
+
+The Docker client is configured through several environment variables defined in the main configuration:
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `DOCKER_HOST` | `unix:///var/run/docker.sock` | Docker daemon connection |
+| `DOCKER_INSIDE` | `false` | Whether PentAGI communicates with host Docker daemon from containers |
+| `DOCKER_NET_ADMIN` | `false` | Whether PentAGI grants the primary container NET_ADMIN capability for advanced networking. |
+| `DOCKER_SOCKET` | `/var/run/docker.sock` | Path to Docker socket on host |
+| `DOCKER_NETWORK` | | Docker network for container communication (bridge mode) or `host` for host network mode |
+| `DOCKER_PUBLIC_IP` | `0.0.0.0` | Public IP for port binding (bridge mode only) |
+| `DOCKER_WORK_DIR` | | Custom work directory path on host |
+| `DOCKER_DEFAULT_IMAGE` | `debian:latest` | Fallback image if AI-selected image fails |
+| `DOCKER_DEFAULT_IMAGE_FOR_PENTEST` | `vxcontrol/kali-linux` | Default Docker image for penetration testing tasks |
+| `DATA_DIR` | `./data` | Local data directory for file operations |
+
+### Configuration Structure
+
+```go
+type Config struct {
+    // Docker (terminal) settings
+    DockerInside                 bool   `env:"DOCKER_INSIDE" envDefault:"false"`
+    DockerNetAdmin               bool   `env:"DOCKER_NET_ADMIN" envDefault:"false"`
+    DockerSocket                 string `env:"DOCKER_SOCKET"`
+    DockerNetwork                string `env:"DOCKER_NETWORK"`
+    DockerPublicIP               string `env:"DOCKER_PUBLIC_IP" envDefault:"0.0.0.0"`
+    DockerWorkDir                string `env:"DOCKER_WORK_DIR"`
+    DockerDefaultImage           string `env:"DOCKER_DEFAULT_IMAGE" envDefault:"debian:latest"`
+    DockerDefaultImageForPentest string `env:"DOCKER_DEFAULT_IMAGE_FOR_PENTEST" envDefault:"vxcontrol/kali-linux"`
+    DataDir                      string `env:"DATA_DIR" envDefault:"./data"`
+}
+```
+
+### NET_ADMIN Capability Configuration
+
+The `DOCKER_NET_ADMIN` option controls whether PentAGI containers are granted the `NET_ADMIN` Linux capability, which provides advanced networking permissions essential for many penetration testing operations.
+
+#### Network Administration Capabilities
+
+When `DOCKER_NET_ADMIN=true`, containers receive the following networking capabilities:
+
+- **Network Interface Management**: Create, modify, and delete network interfaces
+- **Routing Control**: Manipulate routing tables and network routes
+- **Firewall Rules**: Configure iptables, netfilter, and other firewall systems
+- **Traffic Shaping**: Implement QoS (Quality of Service) and bandwidth controls
+- **Bridge Operations**: Create and manage network bridges
+- **VLAN Configuration**: Set up and modify VLAN configurations
+- **Packet Capture**: Enhanced access to raw sockets and packet capture mechanisms
+
+#### Security Implications
+
+**Enabling NET_ADMIN (`DOCKER_NET_ADMIN=true`)**:
+- **Benefits**: Enables full-featured network penetration testing tools
+- **Risks**: Containers can potentially modify host network configuration
+- **Use Cases**: Network scanning, traffic interception, custom routing setups
+- **Tools Enabled**: Advanced nmap features, tcpdump, wireshark, custom networking tools
+
+**Disabling NET_ADMIN (`DOCKER_NET_ADMIN=false`)**:
+- **Benefits**: Enhanced security isolation from host networking
+- **Limitations**: Some advanced networking tools may not function fully (nmap)
+- **Use Cases**: Application-level testing, web security assessment
+- **Recommended**: For environments where network-level testing is not required
+
+#### Container Capability Assignment
+
+The NET_ADMIN capability is applied differently based on container type and configuration:
+
+```go
+// Primary containers (when DOCKER_NET_ADMIN=true)
+hostConfig := &container.HostConfig{
+    CapAdd: []string{"NET_RAW", "NET_ADMIN"},  // Full networking capabilities
+    // ... other configurations
+}
+
+// Primary containers (when DOCKER_NET_ADMIN=false)
+hostConfig := &container.HostConfig{
+    CapAdd: []string{"NET_RAW"},  // Basic raw socket access only
+    // ... other configurations
+}
+```
+
+### Docker-in-Docker Support
+
+PentAGI supports running inside Docker containers while still managing other containers. This is controlled by the `DOCKER_INSIDE` setting:
+
+- **`DOCKER_INSIDE=false`**: PentAGI runs on host, manages containers directly
+- **`DOCKER_INSIDE=true`**: PentAGI runs in container, mounts Docker socket to manage sibling containers
+
+### Network Configuration
+
+PentAGI supports two network modes for container isolation:
+
+#### Bridge Network Mode (Default)
+
+When `DOCKER_NETWORK` is set to a custom network name (e.g., `pentagi-network`), containers are connected to an isolated bridge network:
+- **Isolated Communication**: Containers communicate only within the defined network
+- **Port Mapping**: Container ports are mapped to host ports for external access
+- **Service Discovery**: Enables internal DNS-based service discovery
+- **Enhanced Security**: Network-level isolation from other containers
+
+#### Host Network Mode
+
+When `DOCKER_NETWORK` is set to the special value `host`, containers use the host's network stack directly:
+- **Direct Network Access**: Container shares the host's network interfaces
+- **No Port Mapping**: Ports are directly accessible on host interfaces (no NAT)
+- **Performance**: Eliminates network virtualization overhead
+- **Use Cases**: Advanced network testing, raw packet manipulation, network monitoring
+
+**Security Consideration**: Host network mode reduces isolation. Use only when necessary for penetration testing tasks requiring direct host network access.
+
+## Core Interfaces
+
+### DockerClient Interface
+
+The main interface defines all Docker operations available to PentAGI components:
+
+```go
+type DockerClient interface {
+    // Container lifecycle management
+    RunContainer(ctx context.Context, containerName string, containerType database.ContainerType,
+        flowID int64, config *container.Config, hostConfig *container.HostConfig) (database.Container, error)
+    StopContainer(ctx context.Context, containerID string, dbID int64) error
+    RemoveContainer(ctx context.Context, containerID string, dbID int64) error
+    IsContainerRunning(ctx context.Context, containerID string) (bool, error)
+
+    // Command execution
+    ContainerExecCreate(ctx context.Context, container string, config container.ExecOptions) (container.ExecCreateResponse, error)
+    ContainerExecAttach(ctx context.Context, execID string, config container.ExecAttachOptions) (types.HijackedResponse, error)
+    ContainerExecInspect(ctx context.Context, execID string) (container.ExecInspect, error)
+
+    // File operations
+    ContainerStatPath(ctx context.Context, containerID string, path string) (container.PathStat, error)
+    ListContainerDir(ctx context.Context, containerID string, dirPath string) ([]container.PathStat, error)
+    CopyToContainer(ctx context.Context, containerID string, dstPath string, content io.Reader, options container.CopyToContainerOptions) error
+    CopyFromContainer(ctx context.Context, containerID string, srcPath string) (io.ReadCloser, container.PathStat, error)
+
+    // Utility methods
+    Cleanup(ctx context.Context) error
+    GetDefaultImage() string
+}
+```
+
+### Implementation Structure
+
+```go
+type dockerClient struct {
+    db       database.Querier     // Database for container state management
+    logger   *logrus.Logger       // Structured logging
+    dataDir  string               // Local data directory
+    hostDir  string               // Host-mapped data directory
+    client   *client.Client       // Docker SDK client
+    inside   bool                 // Running inside Docker
+    defImage string               // Default fallback image
+    socket   string               // Docker socket path
+    network  string               // Docker network name
+    publicIP string               // Public IP for port binding
+}
+```
+
+## Container Lifecycle Management
+
+### Container Creation Process
+
+The `RunContainer` method handles the complete container creation workflow:
+
+1. **Preparation**:
+   - Creates flow-specific work directory
+   - Generates unique container name
+   - Records container in database with "starting" status
+
+2. **Image Management**:
+   - Attempts to pull requested image
+   - Falls back to default image if pull fails
+   - Updates database with actual image used
+
+3. **Container Configuration**:
+   - Sets hostname based on container name hash
+   - Configures working directory to `/work`
+   - Sets up restart policy (`on-failure`, maximum 5 retries)
+   - Configures logging (JSON driver with rotation)
+
+4. **Storage Setup**:
+   - Creates dedicated volume or bind mount
+   - Mounts work directory to `/work` in container
+   - Optionally mounts Docker socket for Docker-in-Docker
+
+5. **Network and Ports**:
+   - **Bridge Mode**: Assigns flow-specific ports using deterministic algorithm, binds to public IP
+   - **Host Mode** (`DOCKER_NETWORK=host`): Uses host network stack, skips port bindings
+   - Connects to specified Docker network (unless host mode)
+
+6. **Container Startup**:
+   - Creates container with all configurations
+   - Starts container
+   - Updates database status to "running"
+
+### Example Container Configuration
+
+```go
+containerConfig := &container.Config{
+    Image:      "kali:latest",                    // AI-selected or default image
+    Hostname:   "a1b2c3d4",                      // Generated from container name
+    WorkingDir: "/work",                         // Standard working directory
+    Entrypoint: []string{"tail", "-f", "/dev/null"}, // Keep container running
+    ExposedPorts: nat.PortSet{
+        "28000/tcp": {},                         // Flow-specific ports
+        "28001/tcp": {},
+    },
+}
+
+hostConfig := &container.HostConfig{
+    CapAdd: []string{"NET_RAW"},                 // Required capabilities for network tools
+    RestartPolicy: container.RestartPolicy{
+        Name:              "on-failure",         // Restart failed containers only
+        MaximumRetryCount: 5,
+    },
+    Binds: []string{
+        "/host/data/flow-123:/work",            // Work directory mount
+        "/var/run/docker.sock:/var/run/docker.sock", // Docker socket (if inside Docker)
+    },
+    PortBindings: nat.PortMap{
+        "28000/tcp": []nat.PortBinding{{HostIP: "0.0.0.0", HostPort: "28000"}},
+        "28001/tcp": []nat.PortBinding{{HostIP: "0.0.0.0", HostPort: "28001"}},
+    },
+}
+```
+
+### Container States and Transitions
+
+PentAGI tracks container states in the database:
+
+- **`Starting`**: Container creation in progress
+- **`Running`**: Container is active and available
+- **`Stopped`**: Container has been stopped but not removed
+- **`Failed`**: Container creation or startup failed
+- **`Deleted`**: Container has been removed
+
+### Container Naming Convention
+
+Containers follow a specific naming pattern for easy identification:
+
+```go
+func PrimaryTerminalName(flowID int64) string {
+    return fmt.Sprintf("pentagi-terminal-%d", flowID)
+}
+```
+
+This creates names like `pentagi-terminal-123` for flow ID 123, making it easy to:
+- Identify containers belonging to specific flows
+- Perform flow-based cleanup operations
+- Debug container-related issues
+
+### Cleanup Operations
+
+The `Cleanup` method performs comprehensive cleanup:
+
+1. **Flow State Assessment**:
+   - Identifies flows that should be terminated
+   - Marks incomplete flows as failed
+   - Preserves running flows that should continue
+
+2. **Container Cleanup**:
+   - Stops all containers for terminated flows
+   - Removes stopped containers and their volumes
+   - Updates database to reflect current state
+
+3. **Parallel Processing**:
+   - Uses goroutines for concurrent container deletion
+   - Ensures cleanup doesn't block system operation
+
+## Security and Isolation
+
+### Container Security Model
+
+PentAGI implements a multi-layered security approach for container isolation:
+
+#### Network Isolation
+- **Custom Networks**: Containers run in dedicated Docker networks
+- **Port Control**: Only specific ports are exposed to the host
+- **Host Protection**: Container cannot access host network by default
+
+#### File System Isolation
+- **Read-Only Root**: Base container filesystem is immutable
+- **Controlled Mounts**: Only specific directories are writable
+- **Volume Separation**: Each flow gets isolated storage space
+
+#### Capability Management
+```go
+hostConfig := &container.HostConfig{
+    CapAdd: []string{"NET_RAW"},  // Required for network scanning tools
+    // Other dangerous capabilities are not granted
+}
+```
+
+#### Process Isolation
+- **User Namespaces**: Containers run with isolated user space
+- **PID Isolation**: Container processes are isolated from host
+- **Resource Limits**: Memory and CPU usage are controlled
+
+### Security Best Practices Implemented
+
+1. **Image Validation**: All images are pulled and verified before use
+2. **Fallback Strategy**: Safe default image used if custom image fails
+3. **State Tracking**: All container operations are logged and monitored
+4. **Automatic Cleanup**: Failed or abandoned containers are automatically removed
+5. **Socket Security**: Docker socket is only mounted when explicitly required
+
+## Integration with PentAGI
+
+### Tool Integration
+
+The Docker client integrates with PentAGI's tool system to provide terminal access:
+
+```go
+type terminal struct {
+    flowID       int64
+    containerID  int64
+    containerLID string
+    dockerClient docker.DockerClient
+    tlp          TermLogProvider
+}
+```
+
+The terminal tool uses the Docker client for:
+- **Command Execution**: Running shell commands in isolated containers
+- **File Operations**: Reading and writing files safely
+- **Result Capture**: Collecting command output and artifacts
+
+### Flow File Integration
+
+Flow files are managed by the REST API in `pkg/server/services/flow_files.go` and use Docker client file APIs for synchronization with the running primary container.
+
+PentAGI keeps two different storage areas for flow files:
+
+- **Local cache**: `{DATA_DIR}/flow-{id}-data/uploads` and `{DATA_DIR}/flow-{id}-data/container`
+- **Container workspace**: `/work` inside the primary container
+
+This separation is intentional. It supports both single-node deployments and remote worker-node deployments where the backend host filesystem is not the same filesystem used by Docker workers.
+
+The current behavior is:
+
+- User uploads are saved to the local cache under `uploads/`.
+- If the primary container is running, uploaded files are pushed best-effort to `/work/uploads`.
+- When the primary container starts or is reused, cached uploads are synchronized into `/work/uploads`; the cache is the source of truth.
+- Files pulled from the container are stored under `container/` using their normalized full container path, for example:
+  - `/etc/nginx/nginx.conf` -> `container/etc/nginx/nginx.conf`
+  - `/work/test.md` -> `container/work/test.md`
+- Deleting cached upload files is allowed even when the container is not running. The next container start will resynchronize `/work/uploads` from cache.
+
+The flow files API also exposes a non-recursive live container directory listing endpoint. It uses `ContainerStatPath` to determine whether the requested path is a file or directory:
+
+- If the path is a file, it returns that file metadata directly.
+- If the path is a directory, it calls `ListContainerDir`.
+- If the path is omitted, it defaults to `/work`.
+
+### Provider Integration
+
+The provider system uses Docker client for environment preparation:
+
+```go
+// In providers.go
+type flowProvider struct {
+    // ... other fields
+    docker    docker.DockerClient
+    publicIP  string
+}
+```
+
+Providers use the Docker client to:
+- **Image Selection**: AI agents choose appropriate container images
+- **Environment Setup**: Prepare containers for specific tasks
+- **Resource Management**: Allocate and deallocate containers as needed
+
+### Database Integration
+
+Container states are persisted in the PostgreSQL database:
+
+```sql
+-- Container state tracking
+CREATE TABLE containers (
+    id SERIAL PRIMARY KEY,
+    flow_id INTEGER REFERENCES flows(id),
+    name VARCHAR NOT NULL,
+    image VARCHAR NOT NULL,
+    status container_status NOT NULL,
+    local_id VARCHAR,
+    local_dir VARCHAR,
+    created_at TIMESTAMP DEFAULT NOW(),
+    updated_at TIMESTAMP DEFAULT NOW()
+);
+```
+
+### Observability Integration
+
+All Docker operations are instrumented with:
+- **Structured Logging**: JSON logs with context and metadata
+- **Error Tracking**: Comprehensive error capture and reporting
+- **Performance Metrics**: Container creation and execution timing
+- **Resource Monitoring**: CPU, memory, and network usage tracking
+
+## Usage Examples
+
+### Basic Container Creation
+
+```go
+// Initialize Docker client
+dockerClient, err := docker.NewDockerClient(ctx, db, cfg)
+if err != nil {
+    return fmt.Errorf("failed to create docker client: %w", err)
+}
+
+// Create container for a flow
+containerName := docker.PrimaryTerminalName(flowID)
+container, err := dockerClient.RunContainer(
+    ctx,
+    containerName,
+    database.ContainerTypePrimary,
+    flowID,
+    &container.Config{
+        Image:      "kali:latest",
+        Entrypoint: []string{"tail", "-f", "/dev/null"},
+    },
+    &container.HostConfig{
+        CapAdd: []string{"NET_RAW", "NET_ADMIN"},
+    },
+)
+```
+
+### Command Execution
+
+```go
+// Execute command in container
+createResp, err := dockerClient.ContainerExecCreate(ctx, containerName, container.ExecOptions{
+    Cmd:          []string{"sh", "-c", "nmap -sS 192.168.1.1"},
+    AttachStdout: true,
+    AttachStderr: true,
+    WorkingDir:   "/work",
+    Tty:          true,
+})
+
+// Attach to execution
+resp, err := dockerClient.ContainerExecAttach(ctx, createResp.ID, container.ExecAttachOptions{
+    Tty: true,
+})
+
+// Read output
+output, err := io.ReadAll(resp.Reader)
+```
+
+### File Operations
+
+```go
+// Write file to container
+content := "#!/bin/bash\necho 'Hello from container'"
+archive := createTarArchive("script.sh", content)
+err := dockerClient.CopyToContainer(ctx, containerID, "/work", archive, container.CopyToContainerOptions{})
+
+// Read file from container
+reader, stats, err := dockerClient.CopyFromContainer(ctx, containerID, "/work/results.txt")
+defer reader.Close()
+
+// Extract content from tar
+content := extractFromTar(reader)
+
+// Stat a file or directory in the container
+stat, err := dockerClient.ContainerStatPath(ctx, containerID, "/work/results.txt")
+
+// List direct entries in a container directory
+entries, err := dockerClient.ListContainerDir(ctx, containerID, "/work")
+```
+
+### Container Directory Listing
+
+`ListContainerDir` performs a non-recursive directory listing inside a running container:
+
+1. Uses `ContainerStatPath` to verify that `dirPath` exists and is a directory.
+2. Executes `ls -1 -- <dirPath>` inside the container to get direct entry names.
+3. Calls `ContainerStatPath` for every entry to return Docker `container.PathStat` metadata.
+4. Runs entry stat calls through `pkg/queue` with `containerListWorkers = 20` workers to reduce latency for large directories.
+
+The method returns `[]container.PathStat`. The caller is responsible for joining the returned entry name with the requested base path when it needs full paths.
+
+If `dirPath` is empty, it defaults to `WorkFolderPathInContainer` (`/work`).
+
+### Cleanup and Resource Management
+
+```go
+// Check if container is running
+isRunning, err := dockerClient.IsContainerRunning(ctx, containerID)
+
+// Stop container
+err = dockerClient.StopContainer(ctx, containerID, dbID)
+
+// Remove container and volumes
+err = dockerClient.RemoveContainer(ctx, containerID, dbID)
+
+// Global cleanup (usually called on startup)
+err = dockerClient.Cleanup(ctx)
+```
+
+### Error Handling
+
+```go
+// The client implements comprehensive error handling
+container, err := dockerClient.RunContainer(ctx, name, containerType, flowID, config, hostConfig)
+if err != nil {
+    // Errors include:
+    // - Image pull failures (handled with fallback)
+    // - Container creation failures
+    // - Network configuration issues
+    // - Database update failures
+
+    // The client automatically:
+    // - Updates database with failure status
+    // - Cleans up partially created resources
+    // - Logs detailed error information
+
+    return fmt.Errorf("container creation failed: %w", err)
+}
+```
+
+## Error Handling
+
+### Error Categories
+
+The Docker client handles several categories of errors:
+
+1. **Docker Daemon Errors**:
+   - Connection failures to Docker daemon
+   - API version mismatches
+   - Permission issues
+
+2. **Image-Related Errors**:
+   - Image pull failures (network, authentication)
+   - Invalid image names or tags
+   - Image compatibility issues
+
+3. **Container Runtime Errors**:
+   - Container creation failures
+   - Container startup issues
+   - Resource allocation problems
+
+4. **Network and Storage Errors**:
+   - Port binding conflicts
+   - Volume mount failures
+   - Network configuration issues
+
+### Error Recovery Strategies
+
+1. **Image Fallback**:
+   ```go
+   if err := dc.pullImage(ctx, config.Image); err != nil {
+       logger.WithError(err).Warnf("failed to pull image '%s', using default", config.Image)
+       config.Image = dc.defImage
+       // Retry with default image
+   }
+   ```
+
+2. **Container Cleanup**:
+   ```go
+   if containerCreationFails {
+       defer updateContainerInfo(database.ContainerStatusFailed, containerID)
+       // Clean up any partially created resources
+   }
+   ```
+
+3. **State Synchronization**:
+   - Database state always reflects actual container state
+   - Failed operations are marked appropriately
+   - Orphaned resources are cleaned up automatically
+
+## Best Practices
+
+### Resource Management
+- Always use the `Cleanup()` method on application startup
+- Monitor container resource usage through observability tools
+- Set appropriate timeouts for long-running operations
+- Use deterministic port allocation to avoid conflicts
+
+### Security Considerations
+- Regularly update base images used for containers
+- Minimize capabilities granted to containers
+- Use dedicated networks for container communication
+- Monitor and audit all container operations
+
+### Development and Debugging
+- Use structured logging for all Docker operations
+- Implement comprehensive error handling with context
+- Test container operations in isolated environments
+- Use the ftester utility for debugging specific operations
+
+### Performance Optimization
+- Reuse containers when possible instead of creating new ones
+- Implement efficient cleanup to prevent resource leaks
+- Use appropriate container restart policies
+- Monitor container startup times and optimize configurations
+
+### Integration Guidelines
+- Always use the DockerClient interface instead of direct Docker SDK calls
+- Integrate with PentAGI's database for state management
+- Use the provided logging and observability infrastructure
+- Follow the established naming conventions for containers