wehub-resource-sync
4b6817381b
CI (OpenClaw E2E) / openclaw test (push) Has been cancelled
CI / coverage-report (push) Has been cancelled
CI / test-kubernetes (push) Has been cancelled
CI / should-run-thorough (push) Has been cancelled
CI / test-thorough (cloudwatch-demo) (push) Has been cancelled
CI / test-thorough (flink-ecs) (push) Has been cancelled
CI / test-thorough (upstream-lambda) (push) Has been cancelled
CI / test-thorough (prefect-ecs-fargate) (push) Has been cancelled
Release / build-binaries (zip, opensre.exe, onefile, windows-latest, windows-x64) (push) Has been cancelled
Benchmark image — build + push to ECR (any adapter) / build + push (push) Has been cancelled
CI / quality (ubuntu-latest) (push) Has been cancelled
CI / test (tools-runtime) (push) Has been cancelled
CI / test (e2e-general) (push) Has been cancelled
CI / test (cli-runtime) (push) Has been cancelled
CI / test (e2e-provider-and-openclaw) (push) Has been cancelled
CI / test (integrations-and-misc) (push) Has been cancelled
Release / verify (push) Has been cancelled
Release / build-python-dist (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-15-intel, darwin-x64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-latest, darwin-arm64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04, linux-x64) (push) Has been cancelled
Release / publish-release (push) Has been cancelled
Release / publish-main-release (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-checks (no-LLM) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-live shard ${{ matrix.shard_index }} (push) Has been cancelled
Release / prepare (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04-arm, linux-arm64) (push) Has been cancelled
Synthetic Deterministic Tests / Synthetic offline (deterministic) (push) Has been cancelled
51 lines
1.4 KiB
YAML
51 lines
1.4 KiB
YAML
# Cloud-OpsBench v1 — DeepSeek slice (deepseek-v3.2).
|
|
#
|
|
# Provider slice of cloudopsbench_v1.yml. Use this when running just the
|
|
# DeepSeek model (no ANTHROPIC_API_KEY / OPENAI_API_KEY required). The
|
|
# full grid (all 4 paper models in one run) is still
|
|
# cloudopsbench_v1.yml — keep that for the publication-grade comparison
|
|
# once all provider credits are available.
|
|
#
|
|
# Sibling slices:
|
|
# - cloudopsbench_v1_anthropic.yml (claude-4-sonnet)
|
|
# - cloudopsbench_v1_openai.yml (gpt-4o + gpt-5)
|
|
#
|
|
# Required env at run time: DEEPSEEK_API_KEY. The llm_dispatch LLMSpec
|
|
# routes DeepSeek through opensre's OpenAI-compatible client with the
|
|
# DeepSeek base_url; no separate provider integration required.
|
|
#
|
|
# Run with --dev first to verify the chain, then drop --dev for production:
|
|
# uv run python -m tests.benchmarks._framework.cli run \
|
|
# tests/benchmarks/cloudopsbench/configs/cloudopsbench_v1_deepseek.yml --dev
|
|
|
|
benchmark: cloudopsbench
|
|
|
|
modes:
|
|
- opensre+llm
|
|
|
|
llms:
|
|
- deepseek-v3.2
|
|
|
|
model_versions:
|
|
deepseek-v3.2: deepseek-chat-v3.2
|
|
|
|
runs_per_case: 3
|
|
workers: 4
|
|
cost_budget_usd: 500.0
|
|
seed: 42
|
|
|
|
output_dir: .bench-results/cloudopsbench_v1_deepseek/
|
|
|
|
pre_registration_path: tests/benchmarks/cloudopsbench/configs/preregistrations/cloudopsbench_v1.yml
|
|
|
|
filters:
|
|
limit: 30
|
|
seen_shape: [true]
|
|
systems: []
|
|
fault_categories: []
|
|
|
|
report_formats:
|
|
- json
|
|
- markdown
|
|
- html
|