wehub-resource-sync
4b6817381b
CI (OpenClaw E2E) / openclaw test (push) Has been cancelled
CI / coverage-report (push) Has been cancelled
CI / test-kubernetes (push) Has been cancelled
CI / should-run-thorough (push) Has been cancelled
CI / test-thorough (cloudwatch-demo) (push) Has been cancelled
CI / test-thorough (flink-ecs) (push) Has been cancelled
CI / test-thorough (upstream-lambda) (push) Has been cancelled
CI / test-thorough (prefect-ecs-fargate) (push) Has been cancelled
Release / build-binaries (zip, opensre.exe, onefile, windows-latest, windows-x64) (push) Has been cancelled
Benchmark image — build + push to ECR (any adapter) / build + push (push) Has been cancelled
CI / quality (ubuntu-latest) (push) Has been cancelled
CI / test (tools-runtime) (push) Has been cancelled
CI / test (e2e-general) (push) Has been cancelled
CI / test (cli-runtime) (push) Has been cancelled
CI / test (e2e-provider-and-openclaw) (push) Has been cancelled
CI / test (integrations-and-misc) (push) Has been cancelled
Release / verify (push) Has been cancelled
Release / build-python-dist (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-15-intel, darwin-x64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-latest, darwin-arm64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04, linux-x64) (push) Has been cancelled
Release / publish-release (push) Has been cancelled
Release / publish-main-release (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-checks (no-LLM) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-live shard ${{ matrix.shard_index }} (push) Has been cancelled
Release / prepare (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04-arm, linux-arm64) (push) Has been cancelled
Synthetic Deterministic Tests / Synthetic offline (deterministic) (push) Has been cancelled
44 lines
2.3 KiB
YAML
44 lines
2.3 KiB
YAML
root_cause_category: infrastructure
|
|
equivalent_root_cause_categories:
|
|
- failover_event
|
|
- multi_az_failover_health_check
|
|
- failover
|
|
required_keywords:
|
|
- failover
|
|
- Multi-AZ
|
|
- health check failure
|
|
- primary evidence source
|
|
- workload resumed normally
|
|
- failover initiated
|
|
- failover in progress
|
|
- failover completed
|
|
- instance available
|
|
required_evidence_sources:
|
|
- aws_rds_events
|
|
optimal_trajectory:
|
|
- query_grafana_logs
|
|
- query_grafana_metrics
|
|
- query_grafana_alert_rules
|
|
- describe_rds_instance
|
|
- describe_rds_events
|
|
max_investigation_loops: 4
|
|
model_response: |
|
|
ROOT_CAUSE: Based on the RDS event timeline (primary evidence source), a Multi-AZ automatic failover occurred on payments-prod, triggered by a health check failure on the primary host. This was an infrastructure event rather than resource exhaustion, and the system recovered and workload resumed normally after failover completion.
|
|
ROOT_CAUSE_CATEGORY: infrastructure
|
|
|
|
VALIDATED_CLAIMS:
|
|
- DatabaseConnections dropped to 0 at 08:05Z and recovered fully by 08:07Z. [evidence: aws_cloudwatch_metrics]
|
|
- CPUUtilization and WriteIOPS both dropped to near-zero during the same window, confirming the instance was unavailable during failover rather than suffering a connection pool issue. [evidence: aws_cloudwatch_metrics]
|
|
- RDS events show the full failover sequence: failover initiated (08:04:18Z) -> failover in progress (08:04:21Z) -> failover completed (08:04:58Z) -> instance available (08:05:04Z). [evidence: aws_rds_events]
|
|
- Performance Insights shows no anomalous queries before or after the failover window, and the system recovered and workload resumed normally afterward. [evidence: aws_performance_insights]
|
|
|
|
NON_VALIDATED_CLAIMS:
|
|
- Application connection retry logic should be reviewed to minimize reconnection time after future failovers.
|
|
|
|
CAUSAL_CHAIN:
|
|
- AWS RDS detected a health check failure on the primary instance in us-east-1a.
|
|
- Multi-AZ automatic failover was initiated, promoting the standby in us-east-1b to primary.
|
|
- The DNS endpoint was updated to point clients to the promoted standby host.
|
|
- A brief outage occurred during the failover window while existing database connections dropped and clients reconnected.
|
|
- The instance became available again, and the system recovered and workload resumed normally.
|