chore: import upstream snapshot with attribution
CI (OpenClaw E2E) / openclaw test (push) Has been cancelled
CI / coverage-report (push) Has been cancelled
CI / test-kubernetes (push) Has been cancelled
CI / should-run-thorough (push) Has been cancelled
CI / test-thorough (cloudwatch-demo) (push) Has been cancelled
CI / test-thorough (flink-ecs) (push) Has been cancelled
CI / test-thorough (upstream-lambda) (push) Has been cancelled
CI / test-thorough (prefect-ecs-fargate) (push) Has been cancelled
Release / build-binaries (zip, opensre.exe, onefile, windows-latest, windows-x64) (push) Has been cancelled
Benchmark image — build + push to ECR (any adapter) / build + push (push) Has been cancelled
CI / quality (ubuntu-latest) (push) Has been cancelled
CI / test (tools-runtime) (push) Has been cancelled
CI / test (e2e-general) (push) Has been cancelled
CI / test (cli-runtime) (push) Has been cancelled
CI / test (e2e-provider-and-openclaw) (push) Has been cancelled
CI / test (integrations-and-misc) (push) Has been cancelled
Release / verify (push) Has been cancelled
Release / build-python-dist (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-15-intel, darwin-x64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-latest, darwin-arm64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04, linux-x64) (push) Has been cancelled
Release / publish-release (push) Has been cancelled
Release / publish-main-release (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-checks (no-LLM) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-live shard ${{ matrix.shard_index }} (push) Has been cancelled
Release / prepare (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04-arm, linux-arm64) (push) Has been cancelled
Synthetic Deterministic Tests / Synthetic offline (deterministic) (push) Has been cancelled
@@ -0,0 +1,92 @@
|
||||
name: Bug Report
|
||||
description: Report something that isn't working
|
||||
title: "[BUG] "
|
||||
labels:
|
||||
- bug
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
> **Security notice:** Never paste real API keys, tokens, or passwords in this form. Replace them with placeholders like `glsa_***`, `sk-***`, or `your-token-here` before submitting. Tokens posted here are publicly visible and cannot be fully deleted from GitHub's history.
|
||||
|
||||
Thanks for the report. Please include **observed** behavior and repro steps — guesses make triage slower.
|
||||
- type: textarea
|
||||
id: summary
|
||||
attributes:
|
||||
label: Summary
|
||||
description: One-line description of the bug. Be specific (e.g. agent fails when `.env` contains `=` in values). Avoid vague phrases like "something doesn't work."
|
||||
placeholder: "Example: Agent fails to parse config when .env vars contain special characters"
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: expected_actual
|
||||
attributes:
|
||||
label: Expected vs actual behavior
|
||||
description: What should happen, and what happens instead? Include exact errors, exit codes, or unexpected output where relevant.
|
||||
placeholder: |
|
||||
**Expected:** …
|
||||
|
||||
**Actual:** …
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: reproduce
|
||||
attributes:
|
||||
label: Steps to reproduce
|
||||
description: Minimal steps that consistently trigger the bug (commands, config snippets — redact secrets).
|
||||
placeholder: |
|
||||
1.
|
||||
2.
|
||||
3.
|
||||
validations:
|
||||
required: true
|
||||
- type: dropdown
|
||||
id: reproducible
|
||||
attributes:
|
||||
label: Can you reproduce it consistently?
|
||||
options:
|
||||
- "Yes"
|
||||
- "No"
|
||||
- "Sometimes"
|
||||
validations:
|
||||
required: true
|
||||
- type: dropdown
|
||||
id: frequency
|
||||
attributes:
|
||||
label: How often does it occur?
|
||||
options:
|
||||
- Every time
|
||||
- Intermittent
|
||||
- Under specific conditions
|
||||
validations:
|
||||
required: true
|
||||
- type: dropdown
|
||||
id: os
|
||||
attributes:
|
||||
label: Operating system
|
||||
description: Pick the closest match.
|
||||
options:
|
||||
- macOS
|
||||
- Linux
|
||||
- Windows
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: logs
|
||||
attributes:
|
||||
label: Logs and error output
|
||||
description: Full error message and relevant logs. Redact API keys, tokens, and passwords.
|
||||
placeholder: |
|
||||
```
|
||||
[paste error / logs]
|
||||
```
|
||||
validations:
|
||||
required: false
|
||||
- type: textarea
|
||||
id: context
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: Screenshots, related issues (`#123`), what you were trying to do when this happened.
|
||||
validations:
|
||||
required: false
|
||||
@@ -0,0 +1,8 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: "Questions or Support"
|
||||
url: https://github.com/Tracer-Cloud/open-sre-agent/discussions
|
||||
about: "Ask questions or get help in Discussions"
|
||||
- name: "Security Issue"
|
||||
url: https://github.com/Tracer-Cloud/open-sre-agent/security/policy
|
||||
about: "Report a security vulnerability safely"
|
||||
@@ -0,0 +1,22 @@
|
||||
name: Feature Request
|
||||
description: Suggest a new feature or capability
|
||||
title: "[FEATURE] "
|
||||
labels:
|
||||
- enhancement
|
||||
- pending triage
|
||||
body:
|
||||
- type: textarea
|
||||
id: problem
|
||||
attributes:
|
||||
label: Problem statement
|
||||
description: Why is this needed? User pain, limitation, or gap today (not the solution yet).
|
||||
placeholder: "Example: Users can't export agent logs to external systems without custom scripts."
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: solution
|
||||
attributes:
|
||||
label: Proposed solution
|
||||
description: How should it work? Commands, APIs, UX, inputs/outputs, example snippets.
|
||||
validations:
|
||||
required: true
|
||||
@@ -0,0 +1,20 @@
|
||||
name: Improvement
|
||||
description: Suggest a refactor, optimization, or quality improvement
|
||||
title: "[IMPROVEMENT] "
|
||||
labels:
|
||||
- enhancement
|
||||
body:
|
||||
- type: textarea
|
||||
id: current
|
||||
attributes:
|
||||
label: Current state
|
||||
description: How does it work now? Link to code (`file:line`), snippets, or behavior/architecture.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: desired
|
||||
attributes:
|
||||
label: Desired state
|
||||
description: What should change, why it is better (metrics if possible), and any architectural notes.
|
||||
validations:
|
||||
required: true
|
||||
@@ -0,0 +1,51 @@
|
||||
Fixes #
|
||||
|
||||
<!-- Add issue number above -->
|
||||
|
||||
#### Describe the changes you have made in this PR -
|
||||
|
||||
### Demo/Screenshot for feature changes and bug fixes -
|
||||
<!-- Include at least one proof of the change: UI screenshot, terminal screenshot/log snippet, short video/GIF, or equivalent demo output. -->
|
||||
<!-- Do not add code diff here -->
|
||||
|
||||
---
|
||||
|
||||
## Code Understanding and AI Usage
|
||||
|
||||
**Did you use AI assistance (ChatGPT, Claude, Copilot, etc.) to write any part of this code?**
|
||||
- [ ] No, I wrote all the code myself
|
||||
- [ ] Yes, I used AI assistance (continue below)
|
||||
|
||||
**If you used AI assistance:**
|
||||
- [ ] I have reviewed every single line of the AI-generated code
|
||||
- [ ] I can explain the purpose and logic of each function/component I added
|
||||
- [ ] I have tested edge cases and understand how the code handles them
|
||||
- [ ] I have modified the AI output to follow this project's coding standards and conventions
|
||||
|
||||
**Explain your implementation approach:**
|
||||
<!--
|
||||
Describe in your own words:
|
||||
- What problem does your code solve?
|
||||
- What alternative approaches did you consider?
|
||||
- Why did you choose this specific implementation?
|
||||
- What are the key functions/components and what do they do?
|
||||
|
||||
This helps reviewers understand your thought process and ensures you understand the code.
|
||||
-->
|
||||
|
||||
---
|
||||
|
||||
## Checklist before requesting a review
|
||||
- [ ] I have added proper PR title and linked to the issue
|
||||
- [ ] I have performed a self-review of my code
|
||||
- [ ] **I can explain the purpose of every function, class, and logic block I added**
|
||||
- [ ] I understand why my changes work and have tested them thoroughly
|
||||
- [ ] I have considered potential edge cases and how my code handles them
|
||||
- [ ] If it is a core feature, I have added thorough tests
|
||||
- [ ] My code follows the project's style guidelines and conventions
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
Note: Please check **Allow edits from maintainers** if you would like us to assist in the PR.
|
||||
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 1.2 MiB |
|
After Width: | Height: | Size: 2.9 MiB |
|
After Width: | Height: | Size: 817 KiB |
|
After Width: | Height: | Size: 2.1 MiB |
|
After Width: | Height: | Size: 476 KiB |
|
After Width: | Height: | Size: 4.5 MiB |
|
After Width: | Height: | Size: 374 KiB |
|
After Width: | Height: | Size: 438 KiB |
|
After Width: | Height: | Size: 1.1 MiB |
|
After Width: | Height: | Size: 890 B |
@@ -0,0 +1,237 @@
|
||||
"""Enforce forbidden *direct* import edges between top-level packages.
|
||||
|
||||
Unlike import-linter (which flags transitive chains), this checker looks at
|
||||
**module-top-level** and **nested** (function/class-body) ``import`` /
|
||||
``from … import`` statements. Module-top-level uses the same AST walk as
|
||||
``check_import_cycles``; nested imports close the loophole where lazy
|
||||
``surfaces.*`` imports bypass the module-level graph.
|
||||
|
||||
Used by ``make check-imports`` (and ``check_imports``) alongside
|
||||
import-linter's config contract.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
_CI_DIR = Path(__file__).resolve().parent
|
||||
if str(_CI_DIR) not in sys.path:
|
||||
sys.path.insert(0, str(_CI_DIR))
|
||||
|
||||
from check_import_cycles import ( # noqa: E402
|
||||
_build_graph,
|
||||
_nested_imports,
|
||||
discover_first_party_roots,
|
||||
module_from_path,
|
||||
)
|
||||
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
# ``source_prefix -> forbidden destination roots`` for direct imports only.
|
||||
# Enforces the layering contract documented in ``surfaces/__init__.py``:
|
||||
# "Nothing first-party may import from surfaces/". Adds an explicit bound
|
||||
# on ``platform``, ``core``, ``gateway`` so the surfaces ban
|
||||
# is CI-enforced, not just doc-described.
|
||||
_FORBIDDEN_DIRECT: dict[str, frozenset[str]] = {
|
||||
"platform": frozenset({"surfaces"}),
|
||||
"core": frozenset({"surfaces"}),
|
||||
"gateway": frozenset({"surfaces"}),
|
||||
"integrations": frozenset({"tools", "surfaces"}),
|
||||
"tools": frozenset({"surfaces"}),
|
||||
}
|
||||
|
||||
# Known direct violations being burned down — remove entries as fixes land.
|
||||
# Format: ``"source.module -> dest.module"`` (exact modules from the graph).
|
||||
_BASELINE_IGNORES: frozenset[str] = frozenset(
|
||||
{
|
||||
# Gateway hosts the interactive_shell runtime — pre-existing reuse
|
||||
# to be burned down by extracting shared runtime primitives out of
|
||||
# ``surfaces/interactive_shell/`` and into a layer below ``surfaces``.
|
||||
"gateway.storage.session.resolver -> surfaces.interactive_shell.runtime.context",
|
||||
# tools/interactive_shell action tools reach UP into surfaces/interactive_shell
|
||||
# for runtime / command_registry / UI primitives. Clears when the action
|
||||
# tools themselves are refactored to be UI-agnostic (e.g. return
|
||||
# "approval-required" sentinels instead of calling execution_confirm
|
||||
# directly) so the surface owns its own confirmation UX.
|
||||
"tools.interactive_shell.actions.cli_command -> surfaces.interactive_shell.runtime.subprocess_runner",
|
||||
# These shell action tools type against the shell ``Session`` (investigation_launch
|
||||
# reads session.terminal.background_mode_enabled). Clears when they are made
|
||||
# session-core-agnostic so they no longer import the shell session type.
|
||||
"tools.interactive_shell.actions.investigation -> surfaces.interactive_shell.session",
|
||||
"tools.interactive_shell.actions.sample_alert -> surfaces.interactive_shell.session",
|
||||
"tools.interactive_shell.shared.investigation_launch -> surfaces.interactive_shell.session",
|
||||
"tools.interactive_shell.actions.llm_provider -> surfaces.interactive_shell.command_registry",
|
||||
"tools.interactive_shell.actions.llm_provider -> surfaces.interactive_shell.ui.execution_confirm",
|
||||
"tools.interactive_shell.actions.slash -> surfaces.interactive_shell.command_registry",
|
||||
"tools.interactive_shell.actions.slash -> surfaces.interactive_shell.command_registry.slash_catalog",
|
||||
"tools.interactive_shell.actions.slash -> surfaces.interactive_shell.ui",
|
||||
"tools.interactive_shell.actions.slash -> surfaces.interactive_shell.ui.execution_confirm",
|
||||
"tools.interactive_shell.actions.slash -> surfaces.interactive_shell.utils.telemetry.turn_outcome",
|
||||
"tools.interactive_shell.actions.task_cancel -> surfaces.interactive_shell.command_registry",
|
||||
"tools.interactive_shell.actions.task_cancel -> surfaces.interactive_shell.runtime",
|
||||
"tools.interactive_shell.actions.task_cancel -> surfaces.interactive_shell.ui.execution_confirm",
|
||||
"tools.interactive_shell.implementation.claude_code_executor -> surfaces.interactive_shell.runtime",
|
||||
"tools.interactive_shell.implementation.claude_code_executor -> surfaces.interactive_shell.runtime.subprocess_runner.task_streaming",
|
||||
"tools.interactive_shell.implementation.claude_code_executor -> surfaces.interactive_shell.ui",
|
||||
"tools.interactive_shell.implementation.claude_code_executor -> surfaces.interactive_shell.ui.execution_confirm",
|
||||
"tools.interactive_shell.implementation.claude_code_executor -> surfaces.interactive_shell.utils.error_handling.exception_reporting",
|
||||
"tools.interactive_shell.shell.runner -> surfaces.interactive_shell.runtime",
|
||||
"tools.interactive_shell.shell.runner -> surfaces.interactive_shell.runtime.subprocess_runner.task_streaming",
|
||||
"tools.interactive_shell.shell.runner -> surfaces.interactive_shell.ui",
|
||||
"tools.interactive_shell.shell.runner -> surfaces.interactive_shell.ui.execution_confirm",
|
||||
"tools.interactive_shell.shell.runner -> surfaces.interactive_shell.utils.error_handling.exception_reporting",
|
||||
"tools.interactive_shell.synthetic.runner -> surfaces.interactive_shell.runtime",
|
||||
"tools.interactive_shell.synthetic.runner -> surfaces.interactive_shell.runtime.subprocess_runner.task_streaming",
|
||||
"tools.interactive_shell.synthetic.runner -> surfaces.interactive_shell.ui",
|
||||
"tools.interactive_shell.synthetic.runner -> surfaces.interactive_shell.ui.execution_confirm",
|
||||
"tools.interactive_shell.synthetic.runner -> surfaces.interactive_shell.utils.error_handling.exception_reporting",
|
||||
}
|
||||
)
|
||||
|
||||
# Function/class-body lazy imports that bypass the module-level graph.
|
||||
# Format matches ``_BASELINE_IGNORES``; burn down by moving shared code
|
||||
# below ``surfaces/`` or making tools UI-agnostic.
|
||||
_NESTED_BASELINE_IGNORES: frozenset[str] = frozenset(
|
||||
{
|
||||
"tools.interactive_shell.actions.investigation -> surfaces.interactive_shell.runtime.background.runner",
|
||||
"tools.interactive_shell.actions.investigation -> surfaces.interactive_shell.runtime.investigation_adapter",
|
||||
"tools.interactive_shell.actions.sample_alert -> surfaces.interactive_shell.runtime.background.runner",
|
||||
"tools.interactive_shell.actions.sample_alert -> surfaces.interactive_shell.runtime.investigation_adapter",
|
||||
"tools.interactive_shell.actions.llm_provider -> surfaces.cli.wizard.config",
|
||||
# Nested only: runner.py also has a module-level ui import (see _BASELINE_IGNORES).
|
||||
"tools.interactive_shell.shell.runner -> surfaces.interactive_shell.ui",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DirectViolation:
|
||||
source: str
|
||||
target: str
|
||||
|
||||
@property
|
||||
def edge(self) -> str:
|
||||
return f"{self.source} -> {self.target}"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class NestedViolation:
|
||||
source: str
|
||||
target: str
|
||||
lineno: int
|
||||
|
||||
@property
|
||||
def edge(self) -> str:
|
||||
return f"{self.source} -> {self.target}"
|
||||
|
||||
|
||||
def _source_root(module: str) -> str:
|
||||
return module.split(".", 1)[0]
|
||||
|
||||
|
||||
def find_direct_violations(
|
||||
graph: dict[str, set[str]],
|
||||
*,
|
||||
forbidden: dict[str, frozenset[str]] | None = None,
|
||||
baseline_ignores: frozenset[str] | None = None,
|
||||
) -> list[DirectViolation]:
|
||||
rules = forbidden or _FORBIDDEN_DIRECT
|
||||
ignores = baseline_ignores if baseline_ignores is not None else _BASELINE_IGNORES
|
||||
violations: list[DirectViolation] = []
|
||||
|
||||
for source_module, targets in sorted(graph.items()):
|
||||
source_root = _source_root(source_module)
|
||||
forbidden_roots = rules.get(source_root)
|
||||
if not forbidden_roots:
|
||||
continue
|
||||
for target_module in sorted(targets):
|
||||
target_root = _source_root(target_module)
|
||||
if target_root not in forbidden_roots:
|
||||
continue
|
||||
edge = DirectViolation(source_module, target_module)
|
||||
if edge.edge in ignores:
|
||||
continue
|
||||
violations.append(edge)
|
||||
return violations
|
||||
|
||||
|
||||
def find_nested_direct_violations(
|
||||
root: Path,
|
||||
first_party_roots: tuple[str, ...],
|
||||
*,
|
||||
forbidden: dict[str, frozenset[str]] | None = None,
|
||||
baseline_ignores: frozenset[str] | None = None,
|
||||
) -> list[NestedViolation]:
|
||||
rules = forbidden or _FORBIDDEN_DIRECT
|
||||
ignores = baseline_ignores if baseline_ignores is not None else _NESTED_BASELINE_IGNORES
|
||||
roots = frozenset(first_party_roots)
|
||||
violations: list[NestedViolation] = []
|
||||
|
||||
for pkg in first_party_roots:
|
||||
if pkg not in rules:
|
||||
continue
|
||||
pkg_path = root / pkg
|
||||
if not pkg_path.exists():
|
||||
continue
|
||||
for py in pkg_path.rglob("*.py"):
|
||||
if "__pycache__" in py.parts:
|
||||
continue
|
||||
source_module = module_from_path(root, py)
|
||||
source_root = _source_root(source_module)
|
||||
forbidden_roots = rules.get(source_root)
|
||||
if not forbidden_roots:
|
||||
continue
|
||||
source = py.read_text(encoding="utf-8")
|
||||
for target_module, lineno in _nested_imports(source, first_party_roots=roots):
|
||||
target_root = _source_root(target_module)
|
||||
if target_root not in forbidden_roots:
|
||||
continue
|
||||
violation = NestedViolation(source_module, target_module, lineno)
|
||||
if violation.edge in ignores:
|
||||
continue
|
||||
violations.append(violation)
|
||||
|
||||
return sorted(violations, key=lambda item: (item.source, item.lineno, item.target))
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
del argv
|
||||
root = _REPO_ROOT
|
||||
first_party_roots = discover_first_party_roots(root)
|
||||
graph = _build_graph(root, first_party_roots)
|
||||
module_violations = find_direct_violations(graph)
|
||||
nested_violations = find_nested_direct_violations(root, first_party_roots)
|
||||
|
||||
if not module_violations and not nested_violations:
|
||||
print(
|
||||
"No forbidden direct import edges found "
|
||||
f"(module baseline: {len(_BASELINE_IGNORES)}, "
|
||||
f"nested baseline: {len(_NESTED_BASELINE_IGNORES)})."
|
||||
)
|
||||
return 0
|
||||
|
||||
if module_violations:
|
||||
print(f"FAIL: {len(module_violations)} forbidden module-level direct import edge(s):")
|
||||
for violation in module_violations:
|
||||
print(f" {violation.edge}")
|
||||
|
||||
if nested_violations:
|
||||
if module_violations:
|
||||
print()
|
||||
print(f"FAIL: {len(nested_violations)} forbidden nested direct import edge(s):")
|
||||
for violation in nested_violations:
|
||||
print(f" {violation.edge} (line {violation.lineno})")
|
||||
|
||||
print(
|
||||
"\nFix by moving shared code to a lower layer (platform/common, core/contracts) "
|
||||
"or add a temporary baseline entry in .github/ci/check_direct_imports.py "
|
||||
"with a linked issue — do not use function-level lazy imports to hide "
|
||||
"new direct edges."
|
||||
)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
@@ -0,0 +1,342 @@
|
||||
"""Detect first-party module-load import cycles via Tarjan's SCC.
|
||||
|
||||
Walks every first-party Python module in the repo, builds an import
|
||||
graph from **top-level** ``import`` / ``from ... import`` statements
|
||||
only (function-level lazy imports are intentional runtime breaks and
|
||||
not counted), then reports any strongly-connected component of size
|
||||
> 1, plus any single-module self-loop.
|
||||
|
||||
Used by ``make check-imports`` (via ``check_imports``) locally and by CI.
|
||||
|
||||
Exit codes:
|
||||
0 — zero cycles found
|
||||
1 — at least one cycle found (output lists every SCC + its edges)
|
||||
|
||||
|
||||
How to break a cycle
|
||||
====================
|
||||
|
||||
Two patterns work. Pick by what consumers do with the name.
|
||||
|
||||
1. ``import pkg.sub as sub`` (preferred when consumers monkeypatch)
|
||||
|
||||
Switch ::
|
||||
|
||||
from pkg.sub import name # binds ``name`` at import time
|
||||
...
|
||||
name(args) # uses the bound reference
|
||||
|
||||
to ::
|
||||
|
||||
import pkg.sub as sub # imports the submodule directly
|
||||
...
|
||||
sub.name(args) # attribute lookup at call time
|
||||
|
||||
Both break the static cycle (no edge from the consumer back to the
|
||||
``pkg`` package). The second form keeps attribute-lookup semantics,
|
||||
which matters whenever consumers monkeypatch ``pkg.sub.name`` in
|
||||
tests — the patched attribute IS looked up at each call. The first
|
||||
form binds the name at import time and ignores later patching.
|
||||
|
||||
2. Port / Protocol (when the cycle crosses architectural layers)
|
||||
|
||||
When the cycle is ``layerA <-> layerB`` (e.g. analytics ↔ sentry,
|
||||
integrations ↔ services), neither side should depend on the other
|
||||
directly. Extract a third module — a ``Protocol`` or a small
|
||||
abstract dataclass — that both depend on, and inject the concrete
|
||||
implementation at startup. See the verifier plugin-registry
|
||||
refactor (``integrations/verification/registry.py``) for the
|
||||
canonical example.
|
||||
|
||||
Avoid ``from pkg import sub`` (where ``sub`` is a submodule of ``pkg``)
|
||||
inside that ``pkg`` itself or any of its children — that's the form
|
||||
this script flags. It triggers ``pkg``'s ``__init__`` even when you
|
||||
just want the submodule, and re-export patterns in ``__init__`` close
|
||||
the loop.
|
||||
|
||||
Function-local imports are NOT flagged. They're a legitimate Python
|
||||
pattern for startup-cost deferral (heavy modules in click subcommand
|
||||
bodies), optional dependencies, and conditional / platform-specific
|
||||
code paths. Use sparingly — keep top-level the default — and comment
|
||||
the *why* so future readers don't mistake them for cycle workarounds.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
from collections.abc import Iterable
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
|
||||
# Directories at the repo root that are never first-party import roots.
|
||||
_SKIP_ROOT_DIRS = frozenset(
|
||||
{
|
||||
".git",
|
||||
".github",
|
||||
".pytest_cache",
|
||||
".ruff_cache",
|
||||
".venv",
|
||||
"__pycache__",
|
||||
"docs",
|
||||
"opensre.egg-info",
|
||||
"packaging",
|
||||
"tests",
|
||||
"venv",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def discover_first_party_roots(repo_root: Path | None = None) -> tuple[str, ...]:
|
||||
"""Return top-level package names that contain importable Python code."""
|
||||
root = repo_root or Path(__file__).resolve().parents[2]
|
||||
names: list[str] = []
|
||||
for child in sorted(root.iterdir()):
|
||||
if not child.is_dir() or child.name.startswith("."):
|
||||
continue
|
||||
if child.name in _SKIP_ROOT_DIRS:
|
||||
continue
|
||||
if not any(child.rglob("*.py")):
|
||||
continue
|
||||
names.append(child.name)
|
||||
return tuple(names)
|
||||
|
||||
|
||||
def _top_level_imports(source: str, *, first_party_roots: frozenset[str]) -> set[str]:
|
||||
"""Return first-party module paths imported at the module top level.
|
||||
|
||||
Function-bodies, class-bodies, conditional / try-except wrappers all
|
||||
count as top-level if they are direct module statements — the only
|
||||
imports skipped are those nested **inside a function or class body**.
|
||||
A lazy ``from X import Y`` inside a function does not deadlock at
|
||||
module load, so it should not be flagged as a cycle.
|
||||
"""
|
||||
try:
|
||||
tree = ast.parse(source)
|
||||
except SyntaxError:
|
||||
return set()
|
||||
|
||||
names: set[str] = set()
|
||||
|
||||
def _add(module_path: str) -> None:
|
||||
top = module_path.split(".", 1)[0]
|
||||
if top in first_party_roots:
|
||||
names.add(module_path)
|
||||
|
||||
def _walk_top(body: Iterable[ast.stmt]) -> None:
|
||||
for node in body:
|
||||
if isinstance(node, ast.Import):
|
||||
for alias in node.names:
|
||||
_add(alias.name)
|
||||
elif isinstance(node, ast.ImportFrom):
|
||||
if node.level or not node.module:
|
||||
continue
|
||||
_add(node.module)
|
||||
elif isinstance(node, ast.If):
|
||||
_walk_top(node.body)
|
||||
_walk_top(node.orelse)
|
||||
elif isinstance(node, ast.Try | ast.TryStar):
|
||||
# ``ast.TryStar`` (Python 3.11+, ``try/except*``) shares
|
||||
# the same handler/orelse/finalbody shape as ``ast.Try``.
|
||||
_walk_top(node.body)
|
||||
for handler in node.handlers:
|
||||
_walk_top(handler.body)
|
||||
_walk_top(node.orelse)
|
||||
_walk_top(node.finalbody)
|
||||
elif isinstance(node, ast.With | ast.AsyncWith):
|
||||
_walk_top(node.body)
|
||||
|
||||
_walk_top(tree.body)
|
||||
return names
|
||||
|
||||
|
||||
def _nested_imports(source: str, *, first_party_roots: frozenset[str]) -> list[tuple[str, int]]:
|
||||
"""Return ``(target_module, lineno)`` for imports inside function/class bodies.
|
||||
|
||||
Complements ``_top_level_imports``: catches lazy imports that bypass the
|
||||
module-level direct-edge checker while still being layering violations.
|
||||
"""
|
||||
try:
|
||||
tree = ast.parse(source)
|
||||
except SyntaxError:
|
||||
return []
|
||||
|
||||
results: list[tuple[str, int]] = []
|
||||
|
||||
def _add(module_path: str, lineno: int) -> None:
|
||||
top = module_path.split(".", 1)[0]
|
||||
if top in first_party_roots:
|
||||
results.append((module_path, lineno))
|
||||
|
||||
def _walk_nested(body: Iterable[ast.stmt], *, nested: bool) -> None:
|
||||
for node in body:
|
||||
if isinstance(node, ast.Import):
|
||||
if nested:
|
||||
for alias in node.names:
|
||||
_add(alias.name, node.lineno)
|
||||
elif isinstance(node, ast.ImportFrom):
|
||||
if nested and not node.level and node.module:
|
||||
_add(node.module, node.lineno)
|
||||
elif isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef | ast.ClassDef):
|
||||
_walk_nested(node.body, nested=True)
|
||||
elif isinstance(node, ast.If):
|
||||
_walk_nested(node.body, nested=nested)
|
||||
_walk_nested(node.orelse, nested=nested)
|
||||
elif isinstance(node, ast.Try | ast.TryStar):
|
||||
_walk_nested(node.body, nested=nested)
|
||||
for handler in node.handlers:
|
||||
_walk_nested(handler.body, nested=nested)
|
||||
_walk_nested(node.orelse, nested=nested)
|
||||
_walk_nested(node.finalbody, nested=nested)
|
||||
elif isinstance(node, ast.With | ast.AsyncWith):
|
||||
_walk_nested(node.body, nested=nested)
|
||||
elif isinstance(node, ast.For | ast.AsyncFor | ast.While):
|
||||
_walk_nested(node.body, nested=nested)
|
||||
_walk_nested(node.orelse, nested=nested)
|
||||
elif isinstance(node, ast.Match):
|
||||
for case in node.cases:
|
||||
_walk_nested(case.body, nested=nested)
|
||||
|
||||
_walk_nested(tree.body, nested=False)
|
||||
return results
|
||||
|
||||
|
||||
def module_from_path(root: Path, py: Path) -> str:
|
||||
"""Resolve a repo-relative ``.py`` path to its dotted module name."""
|
||||
module = ".".join(py.with_suffix("").relative_to(root).parts)
|
||||
return module.removesuffix(".__init__")
|
||||
|
||||
|
||||
def _build_graph(root: Path, first_party_roots: tuple[str, ...]) -> dict[str, set[str]]:
|
||||
"""Build the first-party module-level import graph rooted at ``root``."""
|
||||
roots = frozenset(first_party_roots)
|
||||
graph: dict[str, set[str]] = defaultdict(set)
|
||||
for pkg in first_party_roots:
|
||||
pkg_path = root / pkg
|
||||
if not pkg_path.exists():
|
||||
continue
|
||||
for py in pkg_path.rglob("*.py"):
|
||||
if "__pycache__" in py.parts:
|
||||
continue
|
||||
module = module_from_path(root, py)
|
||||
source = py.read_text(encoding="utf-8")
|
||||
graph[module].update(_top_level_imports(source, first_party_roots=roots))
|
||||
return graph
|
||||
|
||||
|
||||
def _tarjan_sccs(graph: dict[str, set[str]]) -> list[list[str]]:
|
||||
"""Return every strongly-connected component of size > 1, plus any
|
||||
single-module self-loop."""
|
||||
index: dict[str, int] = {}
|
||||
lowlink: dict[str, int] = {}
|
||||
on_stack: dict[str, bool] = {}
|
||||
stack: list[str] = []
|
||||
sccs: list[list[str]] = []
|
||||
counter = [0]
|
||||
|
||||
def strongconnect(v: str) -> None:
|
||||
index[v] = counter[0]
|
||||
lowlink[v] = counter[0]
|
||||
counter[0] += 1
|
||||
stack.append(v)
|
||||
on_stack[v] = True
|
||||
|
||||
for w in graph.get(v, ()):
|
||||
if w not in index:
|
||||
strongconnect(w)
|
||||
lowlink[v] = min(lowlink[v], lowlink[w])
|
||||
elif on_stack.get(w):
|
||||
lowlink[v] = min(lowlink[v], index[w])
|
||||
|
||||
if lowlink[v] == index[v]:
|
||||
component: list[str] = []
|
||||
while True:
|
||||
w = stack.pop()
|
||||
on_stack[w] = False
|
||||
component.append(w)
|
||||
if w == v:
|
||||
break
|
||||
# ``while True`` above guarantees ``component`` is non-empty here.
|
||||
if len(component) > 1 or component[0] in graph.get(component[0], ()):
|
||||
sccs.append(component)
|
||||
|
||||
sys.setrecursionlimit(10000)
|
||||
for vertex in list(graph.keys()):
|
||||
if vertex not in index:
|
||||
strongconnect(vertex)
|
||||
|
||||
return sccs
|
||||
|
||||
|
||||
def _format_scc(scc: list[str], graph: dict[str, set[str]]) -> str:
|
||||
"""Format an SCC for human-readable output: members + edges within."""
|
||||
members = sorted(scc)
|
||||
in_scc = set(scc)
|
||||
is_self_loop = len(scc) == 1
|
||||
edges: list[str] = []
|
||||
for module in members:
|
||||
for target in sorted(graph.get(module, ())):
|
||||
if target not in in_scc:
|
||||
continue
|
||||
# Single-module self-loops have only the self-edge; show it
|
||||
# explicitly so the developer knows which import closes the
|
||||
# loop. Multi-module SCCs hide self-edges to keep the diff
|
||||
# focused on the cross-module edges that close the cycle.
|
||||
if target == module and not is_self_loop:
|
||||
continue
|
||||
edges.append(f" {module} -> {target}")
|
||||
|
||||
lines = [f" Modules ({len(scc)}):"]
|
||||
lines.extend(f" - {m}" for m in members)
|
||||
if is_self_loop and not edges:
|
||||
lines.append(" Self-import detected (module imports itself at top level).")
|
||||
elif edges:
|
||||
lines.append(" Edges within SCC:")
|
||||
lines.extend(edges)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
del argv
|
||||
root = Path(__file__).resolve().parents[2]
|
||||
first_party_roots = discover_first_party_roots(root)
|
||||
graph = _build_graph(root, first_party_roots)
|
||||
sccs = _tarjan_sccs(graph)
|
||||
|
||||
if not sccs:
|
||||
print(
|
||||
f"No import cycles found across {len(graph)} first-party modules "
|
||||
f"({len(first_party_roots)} roots)."
|
||||
)
|
||||
return 0
|
||||
|
||||
print(f"FAIL: {len(sccs)} import cycle(s) found across {len(graph)} first-party modules.")
|
||||
for i, scc in enumerate(sorted(sccs, key=lambda s: -len(s)), 1):
|
||||
print(f"\n## SCC #{i} ({len(scc)} module{'s' if len(scc) > 1 else ''}):")
|
||||
print(_format_scc(scc, graph))
|
||||
print(
|
||||
"\nTo break a cycle, prefer:\n"
|
||||
" import pkg.sub as sub\n"
|
||||
" ...\n"
|
||||
" sub.name(args)\n"
|
||||
"over:\n"
|
||||
" from pkg.sub import name\n"
|
||||
" ...\n"
|
||||
" name(args)\n"
|
||||
"\n"
|
||||
"Both fix the static cycle; only the first keeps attribute-lookup\n"
|
||||
"semantics so tests that monkeypatch ``pkg.sub.name`` still work.\n"
|
||||
"\n"
|
||||
"For cross-layer cycles, introduce a Protocol/port both sides\n"
|
||||
"depend on (see ``integrations/verification/registry.py`` for the\n"
|
||||
"canonical example).\n"
|
||||
"\n"
|
||||
"Full pattern reference: docstring at the top of this script."
|
||||
)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
@@ -0,0 +1,99 @@
|
||||
"""Run all import-graph quality checks in one command.
|
||||
|
||||
Orchestrates, in order:
|
||||
|
||||
1. **Cycles** — module-load SCCs (``check_import_cycles``)
|
||||
2. **Layers** — ``config`` independence via import-linter (``.importlinter``)
|
||||
3. **Direct edges** — forbidden top-level imports (``check_direct_imports``)
|
||||
|
||||
Used by ``make check-imports``, ``make check``, and CI.
|
||||
|
||||
Exit codes:
|
||||
0 — all checks passed
|
||||
1 — one or more checks failed (each section prints its own detail)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
from collections.abc import Callable, Sequence
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
_CI_DIR = Path(__file__).resolve().parent
|
||||
if str(_CI_DIR) not in sys.path:
|
||||
sys.path.insert(0, str(_CI_DIR))
|
||||
|
||||
from check_direct_imports import main as check_direct_imports # noqa: E402
|
||||
from check_import_cycles import main as check_import_cycles # noqa: E402
|
||||
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ImportCheck:
|
||||
name: str
|
||||
run: Callable[[], int]
|
||||
|
||||
|
||||
def _run_importlinter(*, config: Path | None = None) -> int:
|
||||
lint_imports = Path(sys.executable).with_name("lint-imports")
|
||||
if not lint_imports.is_file():
|
||||
print(
|
||||
"lint-imports not found — install dev deps (import-linter package).",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
command = [str(lint_imports)]
|
||||
if config is not None:
|
||||
command.extend(["--config", str(config)])
|
||||
completed = subprocess.run(command, cwd=_REPO_ROOT, check=False)
|
||||
return int(completed.returncode)
|
||||
|
||||
|
||||
def import_checks(*, strict_layers: bool = False) -> Sequence[ImportCheck]:
|
||||
layer_config = _REPO_ROOT / ".importlinter.strict" if strict_layers else None
|
||||
return (
|
||||
ImportCheck("Import cycles (Tarjan SCC)", check_import_cycles),
|
||||
ImportCheck(
|
||||
"Import layers (import-linter)",
|
||||
lambda: _run_importlinter(config=layer_config),
|
||||
),
|
||||
ImportCheck("Forbidden direct import edges (module + nested)", check_direct_imports),
|
||||
)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
args = list(argv or [])
|
||||
strict_layers = False
|
||||
if args == ["--strict"]:
|
||||
strict_layers = True
|
||||
args = []
|
||||
if args:
|
||||
print(f"Unknown arguments: {' '.join(args)}", file=sys.stderr)
|
||||
print("Usage: check_imports.py [--strict]", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
checks = import_checks(strict_layers=strict_layers)
|
||||
failures: list[str] = []
|
||||
|
||||
for index, check in enumerate(checks, start=1):
|
||||
print(f"=== [{index}/{len(checks)}] {check.name} ===")
|
||||
exit_code = check.run()
|
||||
if exit_code != 0:
|
||||
failures.append(check.name)
|
||||
print()
|
||||
|
||||
if not failures:
|
||||
print(f"All {len(checks)} import checks passed.")
|
||||
return 0
|
||||
|
||||
print(f"FAIL: {len(failures)} import check(s) failed:")
|
||||
for name in failures:
|
||||
print(f" - {name}")
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
@@ -0,0 +1,194 @@
|
||||
"""Run the live-LLM turn scenario suite sharded across local processes.
|
||||
|
||||
This mirrors the CI ``turn-live`` job
|
||||
(``.github/workflows/interactive-shell-live.yml``): each shard sets
|
||||
``TURN_SHARD_TOTAL`` / ``TURN_SHARD_INDEX`` and runs the live pytest selection
|
||||
``-m live_llm -k "test_live_turn_execution_oracle or test_live_action_planning"``
|
||||
against ``tests/core/agent/test_turn_scenarios.py``.
|
||||
|
||||
The suite is IO-bound (it waits on real LLM API calls), so running all shards
|
||||
concurrently finishes in roughly one shard's wall time instead of the serial
|
||||
total. Each shard runs as its own pytest process; per-shard output is streamed
|
||||
to a log file and the exit codes are aggregated into a final summary.
|
||||
|
||||
Usage:
|
||||
|
||||
uv run python .github/ci/run_live_turn_shards.py # all 8 shards
|
||||
uv run python .github/ci/run_live_turn_shards.py --shards 4
|
||||
uv run python .github/ci/run_live_turn_shards.py --indexes 0,3
|
||||
uv run python .github/ci/run_live_turn_shards.py -- -x # extra pytest args
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import IO
|
||||
|
||||
_TARGET = "tests/core/agent/test_turn_scenarios.py"
|
||||
_K_EXPR = "test_live_turn_execution_oracle or test_live_action_planning"
|
||||
_LOG_DIR = Path(".turn-shard-logs")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ShardResult:
|
||||
index: int
|
||||
exit_code: int
|
||||
duration_s: float
|
||||
log_path: Path
|
||||
|
||||
|
||||
def _parse_args(argv: list[str]) -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"--shards",
|
||||
type=int,
|
||||
default=int(os.getenv("TURN_SHARD_TOTAL", "8")),
|
||||
help="Total number of shards to split the suite into (default: 8).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--indexes",
|
||||
type=str,
|
||||
default="",
|
||||
help="Comma-separated subset of shard indexes to run (default: all).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--workers-per-shard",
|
||||
type=str,
|
||||
default="auto",
|
||||
help="pytest-xdist worker count per shard, passed to -n (default: auto).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--provider",
|
||||
type=str,
|
||||
default=os.getenv("LLM_PROVIDER", "openai"),
|
||||
help="LLM provider for the live run (default: $LLM_PROVIDER or openai).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"pytest_args",
|
||||
nargs="*",
|
||||
help="Extra args forwarded to each pytest shard (after a -- separator).",
|
||||
)
|
||||
return parser.parse_args(argv)
|
||||
|
||||
|
||||
def _resolve_indexes(shards: int, indexes: str) -> list[int]:
|
||||
if shards < 1:
|
||||
raise SystemExit("--shards must be >= 1")
|
||||
if not indexes.strip():
|
||||
return list(range(shards))
|
||||
selected = sorted({int(part) for part in indexes.split(",") if part.strip()})
|
||||
out_of_range = [i for i in selected if i < 0 or i >= shards]
|
||||
if out_of_range:
|
||||
raise SystemExit(f"shard indexes out of range for --shards {shards}: {out_of_range}")
|
||||
return selected
|
||||
|
||||
|
||||
def _build_command(workers: str, pytest_args: list[str]) -> list[str]:
|
||||
return [
|
||||
sys.executable,
|
||||
"-m",
|
||||
"pytest",
|
||||
"-n",
|
||||
workers,
|
||||
"-v",
|
||||
"-m",
|
||||
"live_llm",
|
||||
_TARGET,
|
||||
"-k",
|
||||
_K_EXPR,
|
||||
*pytest_args,
|
||||
]
|
||||
|
||||
|
||||
def _shard_env(*, shard_total: int, shard_index: int, provider: str) -> dict[str, str]:
|
||||
env = os.environ.copy()
|
||||
env["TURN_SHARD_TOTAL"] = str(shard_total)
|
||||
env["TURN_SHARD_INDEX"] = str(shard_index)
|
||||
env["LLM_PROVIDER"] = provider
|
||||
env.setdefault("OPENSRE_DISABLE_KEYRING", "1")
|
||||
env.setdefault("PYTHONUTF8", "1")
|
||||
return env
|
||||
|
||||
|
||||
def _run_shards(
|
||||
*, shard_total: int, indexes: list[int], workers: str, provider: str, pytest_args: list[str]
|
||||
) -> list[ShardResult]:
|
||||
_LOG_DIR.mkdir(parents=True, exist_ok=True)
|
||||
command = _build_command(workers, pytest_args)
|
||||
print(f"Launching {len(indexes)} shard(s) of {shard_total} (provider={provider}):")
|
||||
print(f" {' '.join(command)}\n")
|
||||
|
||||
started: dict[int, tuple[subprocess.Popen[bytes], float, Path, IO[bytes]]] = {}
|
||||
for shard_index in indexes:
|
||||
log_path = _LOG_DIR / f"shard-{shard_index}.log"
|
||||
log_handle: IO[bytes] = log_path.open("wb")
|
||||
process = subprocess.Popen(
|
||||
command,
|
||||
env=_shard_env(shard_total=shard_total, shard_index=shard_index, provider=provider),
|
||||
stdout=log_handle,
|
||||
stderr=subprocess.STDOUT,
|
||||
)
|
||||
started[shard_index] = (process, time.monotonic(), log_path, log_handle)
|
||||
print(f" shard {shard_index} -> pid {process.pid}, log {log_path}")
|
||||
|
||||
print("\nWaiting for shards to finish...\n")
|
||||
results: list[ShardResult] = []
|
||||
pending = set(started)
|
||||
while pending:
|
||||
for shard_index in sorted(pending):
|
||||
process, start_time, log_path, log_handle = started[shard_index]
|
||||
code = process.poll()
|
||||
if code is None:
|
||||
continue
|
||||
log_handle.close()
|
||||
duration = time.monotonic() - start_time
|
||||
status = "PASS" if code == 0 else f"FAIL (exit {code})"
|
||||
print(f" shard {shard_index} {status} in {duration:.0f}s")
|
||||
results.append(
|
||||
ShardResult(
|
||||
index=shard_index,
|
||||
exit_code=code,
|
||||
duration_s=duration,
|
||||
log_path=log_path,
|
||||
)
|
||||
)
|
||||
pending.discard(shard_index)
|
||||
if pending:
|
||||
time.sleep(2.0)
|
||||
return sorted(results, key=lambda r: r.index)
|
||||
|
||||
|
||||
def _print_summary(results: list[ShardResult]) -> int:
|
||||
failures = [r for r in results if r.exit_code != 0]
|
||||
print("\n==================== live turn shard summary ====================")
|
||||
for result in results:
|
||||
status = "PASS" if result.exit_code == 0 else f"FAIL (exit {result.exit_code})"
|
||||
print(f" shard {result.index}: {status} [{result.duration_s:.0f}s] {result.log_path}")
|
||||
if failures:
|
||||
print(f"\n{len(failures)} shard(s) failed. Inspect the logs above.")
|
||||
return 1
|
||||
print("\nAll shards passed.")
|
||||
return 0
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
args = _parse_args(sys.argv[1:] if argv is None else argv)
|
||||
indexes = _resolve_indexes(args.shards, args.indexes)
|
||||
results = _run_shards(
|
||||
shard_total=args.shards,
|
||||
indexes=indexes,
|
||||
workers=args.workers_per_shard,
|
||||
provider=args.provider,
|
||||
pytest_args=list(args.pytest_args),
|
||||
)
|
||||
return _print_summary(results)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,120 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run pytest targets relevant to files changed on this branch.
|
||||
|
||||
Usage
|
||||
-----
|
||||
make test-scope
|
||||
make test-scope ARGS=--dry-run
|
||||
python .github/ci/run_test_scope.py [--dry-run] [--base <ref>]
|
||||
|
||||
Exit codes mirror pytest: 0 = all pass, non-zero = failure or config error.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_CI_DIR = Path(__file__).resolve().parent
|
||||
if str(_CI_DIR) not in sys.path:
|
||||
sys.path.insert(0, str(_CI_DIR))
|
||||
|
||||
from test_scope_rules import classify # noqa: E402
|
||||
|
||||
|
||||
def _commit_ref_exists(ref: str) -> bool:
|
||||
return (
|
||||
subprocess.run(
|
||||
["git", "rev-parse", "--verify", "--quiet", f"{ref}^{{commit}}"],
|
||||
check=False,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
).returncode
|
||||
== 0
|
||||
)
|
||||
|
||||
|
||||
def _resolve_base_ref(base: str) -> str:
|
||||
"""Prefer branch refs for unqualified bases so same-named tags do not win."""
|
||||
if "/" in base or base.startswith("refs/"):
|
||||
return base
|
||||
for ref in (
|
||||
f"refs/heads/{base}",
|
||||
f"refs/remotes/origin/{base}",
|
||||
f"refs/remotes/upstream/{base}",
|
||||
):
|
||||
if _commit_ref_exists(ref):
|
||||
return ref
|
||||
return base
|
||||
|
||||
|
||||
def _git_changed_files(base: str) -> list[str]:
|
||||
resolved_base = _resolve_base_ref(base)
|
||||
try:
|
||||
merge_base = subprocess.check_output(
|
||||
["git", "merge-base", "HEAD", resolved_base],
|
||||
text=True,
|
||||
stderr=subprocess.DEVNULL,
|
||||
).strip()
|
||||
except subprocess.CalledProcessError:
|
||||
merge_base = "HEAD~1"
|
||||
result = subprocess.check_output(
|
||||
["git", "diff", "--name-only", merge_base],
|
||||
text=True,
|
||||
)
|
||||
return [f.strip() for f in result.splitlines() if f.strip()]
|
||||
|
||||
|
||||
def _run(cmd: list[str], *, dry_run: bool) -> int:
|
||||
print(f"\n $ {' '.join(cmd)}\n", flush=True)
|
||||
if dry_run:
|
||||
return 0
|
||||
return subprocess.run(cmd, check=False).returncode
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--dry-run", action="store_true", help="Print command without running.")
|
||||
parser.add_argument(
|
||||
"--base",
|
||||
default="main",
|
||||
help="Ref to diff against (default: main). Falls back to HEAD~1 if unavailable.",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
try:
|
||||
changed = _git_changed_files(args.base)
|
||||
except subprocess.CalledProcessError as exc:
|
||||
print(f"error: could not determine changed files: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
if not changed:
|
||||
print("No changed files detected — nothing to test.")
|
||||
return 0
|
||||
|
||||
print(f"Changed files ({len(changed)}):")
|
||||
for path in changed:
|
||||
print(f" {path}")
|
||||
|
||||
escalate, targets, areas = classify(changed)
|
||||
|
||||
if escalate:
|
||||
print("\nEscalating to full unit suite (core/shared code or 3+ areas touched).")
|
||||
return _run(["make", "test-cov"], dry_run=args.dry_run)
|
||||
|
||||
if not targets:
|
||||
print("\nNo test targets matched — running full unit suite as fallback.")
|
||||
return _run(["make", "test-cov"], dry_run=args.dry_run)
|
||||
|
||||
print(f"\nAreas touched: {', '.join(areas)}")
|
||||
print(f"Running scoped tests: {' '.join(targets)}")
|
||||
return _run(
|
||||
[sys.executable, "-m", "pytest", *targets, "-v"],
|
||||
dry_run=args.dry_run,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,608 @@
|
||||
"""Path → pytest target mapping for branch-scoped test runs (CI.md §2).
|
||||
|
||||
This module is the single source of truth for ``make test-scope``. Edit rules
|
||||
here only — do not duplicate the mapping table in CI.md.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
# Distinct app areas in one diff that trigger escalation to ``make test-cov``.
|
||||
ESCALATION_AREA_THRESHOLD = 3
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class PathRule:
|
||||
"""Map changed paths under ``path_prefix`` to pytest targets."""
|
||||
|
||||
path_prefix: str
|
||||
test_targets: tuple[str, ...]
|
||||
always_escalate: bool = False
|
||||
|
||||
|
||||
# Matched in list order — more specific prefixes must appear before parents.
|
||||
RULES: tuple[PathRule, ...] = (
|
||||
# Shared core (always escalate)
|
||||
PathRule("core/domain/", (), always_escalate=True),
|
||||
PathRule("core/", ("tests/core/",)),
|
||||
PathRule("tools/investigation/reporting/", ("tests/delivery/",)),
|
||||
PathRule("tools/investigation/", (), always_escalate=True),
|
||||
PathRule("utils/", (), always_escalate=True),
|
||||
# Specific sub-packages before their parent
|
||||
PathRule("integrations/llm_cli/", ("tests/integrations/llm_cli/",)),
|
||||
PathRule("integrations/opensre/", ("tests/integrations/opensre/",)),
|
||||
PathRule("integrations/hermes/", ("tests/hermes/",)),
|
||||
PathRule(
|
||||
"integrations/alertmanager/",
|
||||
("tests/integrations/alertmanager/", "tests/e2e/alertmanager/"),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/dagster/",
|
||||
("tests/integrations/dagster/", "tests/synthetic/test_dagster_scenario.py"),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/eks/",
|
||||
(
|
||||
"tests/integrations/eks/",
|
||||
"tests/tools/test_eks_deployment_status_tool.py",
|
||||
"tests/tools/test_eks_describe_addon_tool.py",
|
||||
"tests/tools/test_eks_describe_cluster_tool.py",
|
||||
"tests/tools/test_eks_events_tool.py",
|
||||
"tests/tools/test_eks_list_clusters_tool.py",
|
||||
"tests/tools/test_eks_list_deployments_tool.py",
|
||||
"tests/tools/test_eks_list_namespaces_tool.py",
|
||||
"tests/tools/test_eks_list_pods_tool.py",
|
||||
"tests/tools/test_eks_node_health_tool.py",
|
||||
"tests/tools/test_eks_nodegroup_health_tool.py",
|
||||
"tests/tools/test_eks_pod_logs_tool.py",
|
||||
"tests/tools/test_telemetry.py",
|
||||
"tests/benchmarks/cloudopsbench/tests/test_bench_agent.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/elasticsearch/",
|
||||
(
|
||||
"tests/integrations/elasticsearch/",
|
||||
"tests/tools/test_elasticsearch_logs_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/google_docs/",
|
||||
(
|
||||
"tests/integrations/google_docs/",
|
||||
"tests/tools/test_google_docs_create_report_tool.py",
|
||||
"tests/tools/test_telemetry.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/groundcover/",
|
||||
("tests/integrations/groundcover/", "tests/tools/test_groundcover_tools.py"),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/helm/",
|
||||
("tests/integrations/helm/", "tests/tools/test_helm_tools.py"),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/incident_io/",
|
||||
("tests/integrations/incident_io/", "tests/tools/test_incident_io_tool.py"),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/jira/",
|
||||
(
|
||||
"tests/integrations/jira/",
|
||||
"tests/tools/test_jira_add_comment_tool.py",
|
||||
"tests/tools/test_jira_create_issue_tool.py",
|
||||
"tests/tools/test_jira_issue_detail_tool.py",
|
||||
"tests/tools/test_jira_search_issues_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/clickhouse/",
|
||||
(
|
||||
"tests/integrations/clickhouse/",
|
||||
"tests/tools/test_clickhouse_query_activity_tool.py",
|
||||
"tests/tools/test_clickhouse_system_health_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/mariadb/",
|
||||
(
|
||||
"tests/integrations/mariadb/",
|
||||
"tests/tools/test_mariadb_innodb_status_tool.py",
|
||||
"tests/tools/test_mariadb_process_list_tool.py",
|
||||
"tests/tools/test_mariadb_replication_tool.py",
|
||||
"tests/tools/test_mariadb_slow_queries_tool.py",
|
||||
"tests/tools/test_mariadb_status_tool.py",
|
||||
"tests/e2e/mariadb/",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/mongodb_atlas/",
|
||||
(
|
||||
"tests/integrations/mongodb_atlas/",
|
||||
"tests/tools/test_mongodb_atlas_alerts_tool.py",
|
||||
"tests/tools/test_mongodb_atlas_clusters_tool.py",
|
||||
"tests/tools/test_mongodb_atlas_events_tool.py",
|
||||
"tests/tools/test_mongodb_atlas_metrics_tool.py",
|
||||
"tests/tools/test_mongodb_atlas_performance_advisor_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/mongodb/",
|
||||
(
|
||||
"tests/integrations/mongodb/",
|
||||
"tests/tools/test_mongodb_collection_stats_tool.py",
|
||||
"tests/tools/test_mongodb_current_ops_tool.py",
|
||||
"tests/tools/test_mongodb_profiler_tool.py",
|
||||
"tests/tools/test_mongodb_replica_status_tool.py",
|
||||
"tests/tools/test_mongodb_server_status_tool.py",
|
||||
"tests/e2e/mongodb/",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/mysql/",
|
||||
(
|
||||
"tests/integrations/mysql/",
|
||||
"tests/tools/test_mysql_current_processes_tool.py",
|
||||
"tests/tools/test_mysql_replication_status_tool.py",
|
||||
"tests/tools/test_mysql_server_status_tool.py",
|
||||
"tests/tools/test_mysql_slow_queries_tool.py",
|
||||
"tests/tools/test_mysql_table_stats_tool.py",
|
||||
"tests/e2e/mysql/",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/postgresql/",
|
||||
(
|
||||
"tests/integrations/postgresql/",
|
||||
"tests/tools/test_postgresql_current_queries_tool.py",
|
||||
"tests/tools/test_postgresql_locks_tool.py",
|
||||
"tests/tools/test_postgresql_replication_status_tool.py",
|
||||
"tests/tools/test_postgresql_server_status_tool.py",
|
||||
"tests/tools/test_postgresql_slow_queries_tool.py",
|
||||
"tests/tools/test_postgresql_table_stats_tool.py",
|
||||
"tests/e2e/postgresql/",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/redis/",
|
||||
(
|
||||
"tests/integrations/redis/",
|
||||
"tests/tools/test_redis_client_list_tool.py",
|
||||
"tests/tools/test_redis_key_scan_tool.py",
|
||||
"tests/tools/test_redis_latency_doctor_tool.py",
|
||||
"tests/tools/test_redis_list_depth_tool.py",
|
||||
"tests/tools/test_redis_replication_tool.py",
|
||||
"tests/tools/test_redis_server_info_tool.py",
|
||||
"tests/tools/test_redis_slowlog_tool.py",
|
||||
"tests/e2e/redis/",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/snowflake/",
|
||||
(
|
||||
"tests/integrations/snowflake/",
|
||||
"tests/tools/test_snowflake_query_history_tool.py",
|
||||
"tests/tools/test_telemetry.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/azure/",
|
||||
(
|
||||
"tests/tools/test_azure_monitor_logs_tool.py",
|
||||
"tests/tools/test_telemetry.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/azure_sql/",
|
||||
(
|
||||
"tests/integrations/test_azure_sql.py",
|
||||
"tests/tools/test_azure_sql_current_queries_tool.py",
|
||||
"tests/tools/test_azure_sql_resource_stats_tool.py",
|
||||
"tests/tools/test_azure_sql_server_status_tool.py",
|
||||
"tests/tools/test_azure_sql_slow_queries_tool.py",
|
||||
"tests/tools/test_azure_sql_wait_stats_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/betterstack/",
|
||||
(
|
||||
"tests/integrations/test_betterstack.py",
|
||||
"tests/tools/test_betterstack_logs_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/hermes/tools/",
|
||||
(
|
||||
"tests/tools/test_hermes_logs_tool.py",
|
||||
"tests/tools/test_hermes_session_evidence_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/kafka/",
|
||||
(
|
||||
"tests/integrations/test_kafka.py",
|
||||
"tests/tools/test_kafka_consumer_group_tool.py",
|
||||
"tests/tools/test_kafka_topic_health_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/openclaw/",
|
||||
(
|
||||
"tests/tools/test_openclaw_mcp_tool.py",
|
||||
"tests/tools/test_telemetry.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/openobserve/",
|
||||
(
|
||||
"tests/tools/test_openobserve_logs_tool.py",
|
||||
"tests/tools/test_telemetry.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/opensearch/",
|
||||
(
|
||||
"tests/integrations/test_opensearch_catalog.py",
|
||||
"tests/tools/test_opensearch_analytics_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/posthog_mcp/",
|
||||
(
|
||||
"tests/integrations/test_posthog_mcp.py",
|
||||
"tests/tools/test_posthog_mcp_tool.py",
|
||||
"tests/tools/test_telemetry.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/rabbitmq/",
|
||||
(
|
||||
"tests/integrations/test_rabbitmq.py",
|
||||
"tests/tools/test_rabbitmq_broker_overview_tool.py",
|
||||
"tests/tools/test_rabbitmq_connection_stats_tool.py",
|
||||
"tests/tools/test_rabbitmq_consumer_health_tool.py",
|
||||
"tests/tools/test_rabbitmq_node_health_tool.py",
|
||||
"tests/tools/test_rabbitmq_queue_backlog_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/sentry_mcp/",
|
||||
(
|
||||
"tests/integrations/test_sentry_mcp.py",
|
||||
"tests/tools/test_sentry_mcp_tool.py",
|
||||
"tests/tools/test_telemetry.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/sentry/",
|
||||
(
|
||||
"tests/tools/test_sentry_issue_details_tool.py",
|
||||
"tests/tools/test_sentry_issue_events_tool.py",
|
||||
"tests/tools/test_sentry_search_issues_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/supabase/",
|
||||
(
|
||||
"tests/integrations/test_supabase.py",
|
||||
"tests/tools/test_supabase_health_tool.py",
|
||||
"tests/tools/test_supabase_storage_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/bitbucket/",
|
||||
(
|
||||
"tests/integrations/test_bitbucket.py",
|
||||
"tests/tools/test_bitbucket_commits_tool.py",
|
||||
"tests/tools/test_bitbucket_file_contents_tool.py",
|
||||
"tests/tools/test_bitbucket_search_code_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/telegram/tools/",
|
||||
("tests/tools/test_telegram_send_message_tool.py",),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/tracer/tools/",
|
||||
(
|
||||
"tests/tools/test_tracer_airflow_metrics_tool.py",
|
||||
"tests/tools/test_tracer_batch_statistics_tool.py",
|
||||
"tests/tools/test_tracer_error_logs_tool.py",
|
||||
"tests/tools/test_tracer_failed_jobs_tool.py",
|
||||
"tests/tools/test_tracer_failed_run_tool.py",
|
||||
"tests/tools/test_tracer_failed_tools_tool.py",
|
||||
"tests/tools/test_tracer_host_metrics_tool.py",
|
||||
"tests/tools/test_tracer_run_tool.py",
|
||||
"tests/tools/test_tracer_tasks_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/twilio/",
|
||||
(
|
||||
"tests/integrations/test_twilio.py",
|
||||
"tests/tools/test_twilio_notify_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/github/tools/",
|
||||
(
|
||||
"tests/tools/test_github_actions_tool.py",
|
||||
"tests/tools/test_github_commits_tool.py",
|
||||
"tests/tools/test_github_file_contents_tool.py",
|
||||
"tests/tools/test_github_helpers.py",
|
||||
"tests/tools/test_github_issues_tool.py",
|
||||
"tests/tools/test_github_repo_scope.py",
|
||||
"tests/tools/test_github_repository_tool.py",
|
||||
"tests/tools/test_github_repository_tree_tool.py",
|
||||
"tests/tools/test_github_search_code_tool.py",
|
||||
"tests/tools/test_github_workflow_tools.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/gitlab/",
|
||||
(
|
||||
"tests/integrations/test_gitlab.py",
|
||||
"tests/tools/test_gitlab_commits_tool.py",
|
||||
"tests/tools/test_gitlab_file_tool.py",
|
||||
"tests/tools/test_gitlab_mrs_tool.py",
|
||||
"tests/tools/test_gitlab_pipelines_tool.py",
|
||||
"tests/e2e/gitlab/",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/aws/tools/",
|
||||
("tests/tools/test_aws_operation_tool.py",),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/aws_lambda/",
|
||||
(
|
||||
"tests/integrations/aws/test_lambda_client.py",
|
||||
"tests/tools/test_lambda_config_tool.py",
|
||||
"tests/tools/test_lambda_errors_tool.py",
|
||||
"tests/tools/test_lambda_inspect_tool.py",
|
||||
"tests/tools/test_lambda_invocation_logs_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/cloudtrail/",
|
||||
("tests/tools/test_cloudtrail_events.py",),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/cloudwatch/",
|
||||
(
|
||||
"tests/integrations/aws/test_cloudwatch_client.py",
|
||||
"tests/tools/test_cloudwatch_batch_metrics_tool.py",
|
||||
"tests/tools/test_cloudwatch_logs_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/ec2/",
|
||||
("tests/tools/test_ec2_instances_by_tag_tool.py",),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/elb/",
|
||||
("tests/tools/test_elb_target_health_tool.py",),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/rds/",
|
||||
(
|
||||
"tests/integrations/test_rds.py",
|
||||
"tests/tools/test_rds_tools.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/s3/",
|
||||
(
|
||||
"tests/integrations/aws/test_s3_client.py",
|
||||
"tests/tools/test_s3_get_object_tool.py",
|
||||
"tests/tools/test_s3_inspect_tool.py",
|
||||
"tests/tools/test_s3_list_tool.py",
|
||||
"tests/tools/test_s3_marker_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/opsgenie/",
|
||||
(
|
||||
"tests/integrations/opsgenie/",
|
||||
"tests/tools/test_opsgenie_alert_detail_tool.py",
|
||||
"tests/tools/test_opsgenie_alerts_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/pagerduty/",
|
||||
(
|
||||
"tests/integrations/pagerduty/",
|
||||
"tests/tools/test_pagerduty_incident_detail_tool.py",
|
||||
"tests/tools/test_pagerduty_incidents_tool.py",
|
||||
"tests/tools/test_pagerduty_oncall_tool.py",
|
||||
"tests/tools/test_pagerduty_services_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/prefect/",
|
||||
(
|
||||
"tests/integrations/prefect/",
|
||||
"tests/tools/test_prefect_flow_runs_tool.py",
|
||||
"tests/tools/test_prefect_worker_health_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/signoz/",
|
||||
(
|
||||
"tests/integrations/signoz/",
|
||||
"tests/tools/test_signoz_tools.py",
|
||||
"tests/synthetic/test_signoz_scenario.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/splunk/",
|
||||
("tests/integrations/splunk/", "tests/tools/test_splunk_search_tool.py"),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/tempo/",
|
||||
(
|
||||
"tests/integrations/tempo/",
|
||||
"tests/tools/test_tempo_tools.py",
|
||||
"tests/synthetic/test_tempo_scenario.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/temporal/",
|
||||
(
|
||||
"tests/integrations/temporal/",
|
||||
"tests/integrations/test_temporal_catalog.py",
|
||||
"tests/synthetic/test_temporal_scenario.py",
|
||||
"tests/tools/test_temporal_namespace_info_tool.py",
|
||||
"tests/tools/test_temporal_task_queue_tool.py",
|
||||
"tests/tools/test_temporal_workflow_history_tool.py",
|
||||
"tests/tools/test_temporal_workflows_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/vercel/",
|
||||
(
|
||||
"tests/integrations/vercel/",
|
||||
"tests/tools/test_vercel_deployment_status_tool.py",
|
||||
"tests/tools/test_vercel_logs_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/victoria_logs/",
|
||||
(
|
||||
"tests/integrations/victoria_logs/",
|
||||
"tests/tools/test_victoria_logs_tool.py",
|
||||
"tests/e2e/victoria_logs/",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/x_mcp/",
|
||||
(
|
||||
"tests/integrations/test_x_mcp.py",
|
||||
"tests/tools/test_x_mcp_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/argocd/",
|
||||
(
|
||||
"tests/integrations/argocd/",
|
||||
"tests/tools/test_argocd_tools.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/coralogix/",
|
||||
(
|
||||
"tests/integrations/coralogix/",
|
||||
"tests/tools/test_coralogix_logs_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/honeycomb/",
|
||||
(
|
||||
"tests/integrations/honeycomb/",
|
||||
"tests/tools/test_honeycomb_traces_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/jenkins/",
|
||||
("tests/integrations/test_jenkins.py", "tests/synthetic/test_jenkins_scenario.py"),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/datadog/",
|
||||
(
|
||||
"tests/integrations/datadog/",
|
||||
"tests/tools/test_datadog_context_tool.py",
|
||||
"tests/tools/test_datadog_events_tool.py",
|
||||
"tests/tools/test_datadog_logs_tool.py",
|
||||
"tests/tools/test_datadog_metrics_tool.py",
|
||||
"tests/tools/test_datadog_monitors_tool.py",
|
||||
"tests/tools/test_datadog_node_pods_tool.py",
|
||||
),
|
||||
),
|
||||
PathRule(
|
||||
"integrations/grafana/",
|
||||
(
|
||||
"tests/integrations/grafana/",
|
||||
"tests/tools/test_grafana_alert_rules_tool.py",
|
||||
"tests/tools/test_grafana_annotations_tool.py",
|
||||
"tests/tools/test_grafana_logs_tool.py",
|
||||
"tests/tools/test_grafana_metrics_tool.py",
|
||||
"tests/tools/test_grafana_service_names_tool.py",
|
||||
"tests/tools/test_grafana_traces_tool.py",
|
||||
"tests/e2e/grafana_validation/",
|
||||
),
|
||||
),
|
||||
PathRule("integrations/", ("tests/integrations/",)),
|
||||
PathRule("tools/system/fleet_monitoring/", ("tests/agent/", "tests/fleet_monitoring/")),
|
||||
PathRule("surfaces/cli/", ("tests/cli/",)),
|
||||
PathRule("surfaces/interactive_shell/", ("tests/interactive_shell/",)),
|
||||
PathRule("gateway/", ("gateway/tests/",)),
|
||||
PathRule("tools/system/watch_dog/", ("tests/watch_dog/",)),
|
||||
PathRule("tools/", ("tests/tools/",)),
|
||||
PathRule("platform/analytics/", ("tests/analytics/",)),
|
||||
PathRule("platform/guardrails/", ("tests/platform/guardrails/",)),
|
||||
PathRule("platform/masking/", ("tests/masking/",)),
|
||||
PathRule("platform/packaging/", ("tests/packaging/",)),
|
||||
PathRule("platform/sandbox/", ("tests/sandbox/",)),
|
||||
PathRule(
|
||||
"platform/deployment/",
|
||||
("tests/deployment/", "tests/platform/deployment/test_deployment_health.py"),
|
||||
),
|
||||
PathRule("platform/auth/", ("tests/platform/auth/",)),
|
||||
PathRule("gateway/webapp.py", ("gateway/tests/test_webapp.py",)),
|
||||
# Repo-wide config
|
||||
PathRule("pyproject.toml", (), always_escalate=True),
|
||||
PathRule("uv.lock", (), always_escalate=True),
|
||||
PathRule("pytest.ini", (), always_escalate=True),
|
||||
PathRule("Makefile", (), always_escalate=True),
|
||||
PathRule(".github/ci/", ("tests/github_ci/",)),
|
||||
)
|
||||
|
||||
|
||||
def _matches(path: str, prefix: str) -> bool:
|
||||
return path.startswith(prefix) or path == prefix.rstrip("/")
|
||||
|
||||
|
||||
def _area_key(prefix: str) -> str:
|
||||
parts = prefix.split("/")
|
||||
if parts[0] == "deployment" or parts[:2] == ["platform", "deployment"]:
|
||||
return "deployment"
|
||||
return prefix
|
||||
|
||||
|
||||
def classify(changed: list[str]) -> tuple[bool, list[str], list[str]]:
|
||||
"""Return ``(should_escalate, test_targets, matched_areas)``."""
|
||||
escalate = False
|
||||
targets: list[str] = []
|
||||
areas: list[str] = []
|
||||
|
||||
for path in changed:
|
||||
matched = False
|
||||
for rule in RULES:
|
||||
if not _matches(path, rule.path_prefix):
|
||||
continue
|
||||
matched = True
|
||||
if rule.always_escalate:
|
||||
escalate = True
|
||||
else:
|
||||
area = _area_key(rule.path_prefix)
|
||||
if area not in areas:
|
||||
areas.append(area)
|
||||
for target in rule.test_targets:
|
||||
if target not in targets:
|
||||
targets.append(target)
|
||||
break
|
||||
|
||||
if not matched and path.startswith("tests/") and path not in targets:
|
||||
targets.append(path)
|
||||
|
||||
if len(areas) >= ESCALATION_AREA_THRESHOLD:
|
||||
escalate = True
|
||||
|
||||
existing = [t for t in targets if Path(t).exists()]
|
||||
dropped = [t for t in targets if t not in existing]
|
||||
if dropped:
|
||||
print(f" (skipping non-existent targets: {', '.join(dropped)})", flush=True)
|
||||
return escalate, existing, areas
|
||||
@@ -0,0 +1,7 @@
|
||||
name: "CodeQL config"
|
||||
|
||||
paths-ignore:
|
||||
# Vendored third-party pipeline code in test fixtures — not owned by this project.
|
||||
# Covers every tests/e2e/<case>/pipeline_code/ fixture (lambda, flink, prefect,
|
||||
# datadog, kubernetes, …) so new cases are ignored automatically.
|
||||
- tests/e2e/**/pipeline_code/**
|
||||
@@ -0,0 +1,15 @@
|
||||
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
|
||||
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: "uv"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
open-pull-requests-limit: 10
|
||||
|
||||
- package-ecosystem: "npm"
|
||||
directory: "/docs"
|
||||
schedule:
|
||||
interval: "daily"
|
||||
open-pull-requests-limit: 10
|
||||
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build the Discord release announcement JSON payload.
|
||||
|
||||
Reads from environment variables set by the GitHub Actions step:
|
||||
RELEASE_TAG, RELEASE_URL,
|
||||
DISCORD_RELEASES_ROLE_ID, DISCORD_RELEASE_LOGO_EMOJI, DISCORD_RELEASE_LOGO_URL,
|
||||
CHANGELOG_FILE — path to DISCORD_NARRATIVE.md or GENERATED_CHANGELOG.md
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
MAX_CHANGELOG_CHARS = 1400
|
||||
|
||||
tag = os.environ["RELEASE_TAG"]
|
||||
url = os.environ["RELEASE_URL"]
|
||||
role_id = os.environ.get("DISCORD_RELEASES_ROLE_ID", "")
|
||||
logo_emoji = os.environ.get("DISCORD_RELEASE_LOGO_EMOJI", "")
|
||||
logo_url = os.environ.get("DISCORD_RELEASE_LOGO_URL", "")
|
||||
changelog_file = os.environ.get("CHANGELOG_FILE", "")
|
||||
|
||||
if changelog_file and os.path.isfile(changelog_file):
|
||||
with open(changelog_file) as f:
|
||||
changelog = f.read().replace("\r", "").strip()
|
||||
else:
|
||||
changelog = "No changelog available."
|
||||
|
||||
if len(changelog) > MAX_CHANGELOG_CHARS:
|
||||
changelog = changelog[: MAX_CHANGELOG_CHARS - 3] + "..."
|
||||
|
||||
mention = f"<@&{role_id}>\n" if role_id else ""
|
||||
logo_prefix = f"{logo_emoji} " if logo_emoji else ""
|
||||
bt = "`"
|
||||
|
||||
content = (
|
||||
mention + logo_prefix + f"🚀 **opensre {bt}{tag}{bt} is live**\n" + f"🔗 {url}\n\n" + changelog
|
||||
)
|
||||
|
||||
allowed_mentions: dict = {"parse": []}
|
||||
if role_id:
|
||||
allowed_mentions["roles"] = [role_id]
|
||||
|
||||
payload: dict = {"content": content, "allowed_mentions": allowed_mentions}
|
||||
if logo_url:
|
||||
payload["username"] = "OpenSRE"
|
||||
payload["avatar_url"] = logo_url
|
||||
|
||||
json.dump(payload, sys.stdout)
|
||||
@@ -0,0 +1,223 @@
|
||||
"""Assign and notify **new** contributors on `good first issue` threads.
|
||||
|
||||
Here *new* means the commenter has **no merged PRs and no open PRs** in this repo where
|
||||
they are the PR author (GitHub Search API). **One or more merged PRs** means they are not
|
||||
treated as a first-time contributor for this automation (GitHub's ``FIRST_TIME_CONTRIBUTOR``
|
||||
/ ``FIRST_TIMER`` flags are not used).
|
||||
|
||||
Also skips repo insiders (OWNER / MEMBER / COLLABORATOR), bots, closed issues,
|
||||
comments on **pull request** threads (``issue_comment`` fires for PRs too; those
|
||||
use ``issue.pull_request``), and commenters already listed as assignees.
|
||||
|
||||
**One open assignment per eligible new contributor** in this repo: if they already
|
||||
have another **open** issue assigned (Search API), they cannot be auto-assigned here.
|
||||
|
||||
At most **one** auto-assignment per issue: if anyone else is already an assignee,
|
||||
further eligible commenters are skipped (manual pre-assignments count).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
GOOD_FIRST_LABEL = "good first issue"
|
||||
# Do not auto-assign maintainers/collaborators;
|
||||
# eligibility is 0 merged + 0 open PRs as author + not insider.
|
||||
EXCLUDED_COMMENTER_ASSOCIATIONS = frozenset({"OWNER", "MEMBER", "COLLABORATOR"})
|
||||
GITHUB_API = "https://api.github.com"
|
||||
|
||||
|
||||
def _github_api_url(path: str, query: dict[str, str]) -> str:
|
||||
encoded = urllib.parse.urlencode(query)
|
||||
return f"{GITHUB_API}{path}?{encoded}"
|
||||
|
||||
|
||||
def screen_event_without_api(event: dict[str, Any]) -> str | None:
|
||||
"""Return a skip reason before calling the GitHub API, or None if checks should continue."""
|
||||
issue = event.get("issue") or {}
|
||||
comment = event.get("comment") or {}
|
||||
if issue.get("pull_request") is not None:
|
||||
return "comment_on_pull_request"
|
||||
if issue.get("state") != "open":
|
||||
return "issue_not_open"
|
||||
labels = issue.get("labels") or []
|
||||
if not isinstance(labels, list):
|
||||
return "invalid_labels"
|
||||
names = {item.get("name") for item in labels if isinstance(item, dict)}
|
||||
if GOOD_FIRST_LABEL not in names:
|
||||
return "not_good_first_issue"
|
||||
|
||||
c_user = comment.get("user") or {}
|
||||
if c_user.get("type") == "Bot":
|
||||
return "bot_commenter"
|
||||
c_login = c_user.get("login") or ""
|
||||
if not c_login:
|
||||
return "missing_commenter_login"
|
||||
|
||||
c_assoc = comment.get("author_association") or ""
|
||||
if c_assoc in EXCLUDED_COMMENTER_ASSOCIATIONS:
|
||||
return "commenter_repo_insider"
|
||||
|
||||
assignees = issue.get("assignees") or []
|
||||
if isinstance(assignees, list):
|
||||
assigned_logins = {
|
||||
a.get("login") for a in assignees if isinstance(a, dict) and a.get("login")
|
||||
}
|
||||
if c_login in assigned_logins:
|
||||
return "already_assignee"
|
||||
if assigned_logins:
|
||||
return "issue_already_claimed"
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def assign_decision(
|
||||
*,
|
||||
skip_reason_pre_api: str | None,
|
||||
merged_pr_count_for_commenter: int,
|
||||
open_pr_count_for_commenter: int,
|
||||
open_assigned_issue_count_for_commenter: int,
|
||||
) -> tuple[bool, str]:
|
||||
"""Return (should_assign_and_comment, skip_reason_or_empty).
|
||||
|
||||
Eligible "new contributor" means ``merged_pr_count_for_commenter == 0`` and
|
||||
``open_pr_count_for_commenter == 0`` (PR author in this repo, via Search API).
|
||||
They must also have no other open issues assigned in this repo
|
||||
(``open_assigned_issue_count_for_commenter == 0``).
|
||||
"""
|
||||
if skip_reason_pre_api is not None:
|
||||
return False, skip_reason_pre_api
|
||||
if open_pr_count_for_commenter > 0:
|
||||
return False, "has_open_prs"
|
||||
if merged_pr_count_for_commenter > 0:
|
||||
return False, "has_merged_prs"
|
||||
if open_assigned_issue_count_for_commenter > 0:
|
||||
return False, "already_has_open_assigned_issue"
|
||||
return True, ""
|
||||
|
||||
|
||||
def _request_json(url: str, token: str) -> Any:
|
||||
parsed = urllib.parse.urlparse(url)
|
||||
if parsed.scheme != "https" or parsed.netloc != "api.github.com":
|
||||
raise ValueError("GitHub API URL must target https://api.github.com")
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
headers={
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Accept": "application/vnd.github+json",
|
||||
"X-GitHub-Api-Version": "2022-11-28",
|
||||
},
|
||||
method="GET",
|
||||
)
|
||||
with (
|
||||
urllib.request.urlopen(req, timeout=60) as resp
|
||||
): # nosemgrep: python.lang.security.audit.dynamic-urllib-use-detected.dynamic-urllib-use-detected
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
def _search_issue_total_count(query: str, token: str) -> int:
|
||||
url = _github_api_url("/search/issues", {"q": query})
|
||||
try:
|
||||
data = _request_json(url, token)
|
||||
except urllib.error.URLError as exc:
|
||||
print(f"GitHub search failed: {exc}", file=sys.stderr)
|
||||
raise
|
||||
total = data.get("total_count")
|
||||
if not isinstance(total, int):
|
||||
return 0
|
||||
return total
|
||||
|
||||
|
||||
def fetch_merged_pr_count(owner: str, repo: str, login: str, token: str) -> int:
|
||||
q = f"repo:{owner}/{repo} is:pr is:merged author:{login}"
|
||||
return _search_issue_total_count(q, token)
|
||||
|
||||
|
||||
def fetch_open_pr_count(owner: str, repo: str, login: str, token: str) -> int:
|
||||
q = f"repo:{owner}/{repo} is:pr is:open author:{login}"
|
||||
return _search_issue_total_count(q, token)
|
||||
|
||||
|
||||
def fetch_open_assigned_issue_count(owner: str, repo: str, login: str, token: str) -> int:
|
||||
"""Count open issues in this repo where ``login`` is an assignee."""
|
||||
q = f"repo:{owner}/{repo} is:issue is:open assignee:{login}"
|
||||
return _search_issue_total_count(q, token)
|
||||
|
||||
|
||||
def build_assign_notice_body(*, assignee_login: str) -> str:
|
||||
return f"@{assignee_login} You've been **assigned** to this issue. Thanks for picking it up."
|
||||
|
||||
|
||||
def set_github_output(name: str, value: str) -> None:
|
||||
path = os.environ.get("GITHUB_OUTPUT")
|
||||
if not path:
|
||||
return
|
||||
with open(path, "a", encoding="utf-8") as fh:
|
||||
fh.write(f"{name}={value}\n")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
event_path = os.environ.get("GITHUB_EVENT_PATH")
|
||||
repository = os.environ.get("GITHUB_REPOSITORY")
|
||||
token = os.environ.get("GITHUB_TOKEN")
|
||||
if not event_path or not repository or not token:
|
||||
print("Missing GITHUB_EVENT_PATH, GITHUB_REPOSITORY, or GITHUB_TOKEN.", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
raw = Path(event_path).read_text(encoding="utf-8")
|
||||
event = json.loads(raw)
|
||||
|
||||
pre = screen_event_without_api(event)
|
||||
merged_count = 0
|
||||
open_count = 0
|
||||
open_assigned_issue_count = 0
|
||||
|
||||
if pre is None:
|
||||
owner, _, repo = repository.partition("/")
|
||||
if not owner or not repo:
|
||||
print("Invalid GITHUB_REPOSITORY.", file=sys.stderr)
|
||||
return 1
|
||||
comment = event.get("comment") or {}
|
||||
c_login = (comment.get("user") or {}).get("login") or ""
|
||||
try:
|
||||
merged_count = fetch_merged_pr_count(owner, repo, c_login, token)
|
||||
open_count = fetch_open_pr_count(owner, repo, c_login, token)
|
||||
open_assigned_issue_count = fetch_open_assigned_issue_count(owner, repo, c_login, token)
|
||||
except urllib.error.URLError as exc:
|
||||
print(f"GitHub API request failed: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
should, reason = assign_decision(
|
||||
skip_reason_pre_api=pre,
|
||||
merged_pr_count_for_commenter=merged_count,
|
||||
open_pr_count_for_commenter=open_count,
|
||||
open_assigned_issue_count_for_commenter=open_assigned_issue_count,
|
||||
)
|
||||
|
||||
if not should:
|
||||
print(f"Skip: {reason}")
|
||||
set_github_output("should_assign", "false")
|
||||
return 0
|
||||
|
||||
comment_user = (event.get("comment") or {}).get("user") or {}
|
||||
login = comment_user.get("login") if isinstance(comment_user, dict) else ""
|
||||
if not isinstance(login, str) or not login:
|
||||
print("Missing commenter login.", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
body = build_assign_notice_body(assignee_login=login)
|
||||
Path("assign_comment.md").write_text(body, encoding="utf-8")
|
||||
set_github_output("should_assign", "true")
|
||||
print("Wrote assign_comment.md; assignment will be applied in workflow.")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,124 @@
|
||||
"""Write celebrate-merge PR comment body to comment.md (run from Actions after merge)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import random
|
||||
|
||||
discord = os.environ["DISCORD_INVITE_URL"]
|
||||
contributor = os.environ["CONTRIBUTOR_LOGIN"]
|
||||
|
||||
templates: list[str] = [
|
||||
(f"🎉 **MERGED!** @{contributor} just shipped something. The diff gods are pleased. 🙌"),
|
||||
(
|
||||
f"🚀 **Houston, we have a merge.** @{contributor} your PR is in orbit. "
|
||||
"Thanks for launching this one!"
|
||||
),
|
||||
(
|
||||
f"💜 **One more reason the project grows.** Thanks @{contributor} — "
|
||||
"your contribution just landed!"
|
||||
),
|
||||
(
|
||||
f"🎊 **Achievement unlocked: PR Merged.** @{contributor} passed code review, "
|
||||
"survived CI, and shipped. Respect. 🤝"
|
||||
),
|
||||
(
|
||||
f'🔥 **Another one.** @{contributor} said "here\'s a PR" and maintainers said '
|
||||
"\"ship it\". That's how it's done."
|
||||
),
|
||||
(
|
||||
f"🧑💻 **@{contributor} has entered the contributor hall of fame.** "
|
||||
"Merged. Done. Shipped. Go touch grass (then come back with another PR). 🌱"
|
||||
),
|
||||
(
|
||||
f"🎯 **Bullseye.** @{contributor} opened a PR, kept the vibes clean, "
|
||||
"and got it merged. Absolute cinema. 🎬"
|
||||
),
|
||||
(
|
||||
f"⚡ **LGTM → Merged.** @{contributor}, your work is in. "
|
||||
"Every commit counts — thank you for this one."
|
||||
),
|
||||
# new additions
|
||||
(
|
||||
f'😤 **@{contributor} said "I will fix this" and then actually fixed it.** '
|
||||
"Legendary behavior."
|
||||
),
|
||||
(
|
||||
f"🍕 **@{contributor}'s PR:** crispy edges, no unnecessary toppings, delivered on time. "
|
||||
"Understood the assignment. 🔥"
|
||||
),
|
||||
(f"🌊 **Merged.** @{contributor} is now permanently woven into git history. No take-backs. 😄"),
|
||||
(
|
||||
f"🤖 **CI passed. Linter didn't scream. Reviewer typed LGTM.** "
|
||||
f"@{contributor}, every machine in this pipeline just slow-clapped. 🖥️✨"
|
||||
),
|
||||
(f"🧠 **@{contributor} opened a PR.** Maintainers feared them. CI genuflected. It merged. 🚨"),
|
||||
(
|
||||
f"😭 **Clear commit message. Green tests. Kind review.** "
|
||||
f"@{contributor}, stop making the rest of us look bad."
|
||||
),
|
||||
(
|
||||
f"🐸 **Rebase? Handled. Conflicts? Squashed. CI? Vibing.** "
|
||||
f"@{contributor} touched the untouchable and lived. 🫡"
|
||||
),
|
||||
(
|
||||
f"🏆 **@{contributor} did not come to play.** "
|
||||
"PR opened. Review survived. Merged clean. Retire the jersey. 🎽"
|
||||
),
|
||||
(
|
||||
f"🎲 **Researchers are baffled.** @{contributor} opened a PR, got it reviewed without drama, "
|
||||
"and merged clean. This violates known laws of open source. 🔬"
|
||||
),
|
||||
(
|
||||
f"🌮 **@{contributor}'s PR:** showed up unannounced, improved everything, left zero bugs. "
|
||||
"Just like a perfect taco. 🌮"
|
||||
),
|
||||
(
|
||||
f"🐉 **Legend says** enough merged PRs and you ascend. "
|
||||
f"@{contributor} is dangerously close. 🌤️"
|
||||
),
|
||||
(
|
||||
f"🛸 **Aliens watching our repo** just upgraded @{contributor}'s threat level to: "
|
||||
"*do not engage — too competent*. 👽"
|
||||
),
|
||||
(
|
||||
f'🎻 **"The diff was clean, the tests did pass, the reviewer wept."** '
|
||||
f"That poem was about @{contributor}'s PR. 🥹"
|
||||
),
|
||||
(f"🍵 **@{contributor} made tea, opened a PR, and merged before it cooled.** No notes. ☕"),
|
||||
(
|
||||
f"🏄 **Some PRs rot in review for six weeks.** "
|
||||
f'@{contributor}\'s said "not today" and merged like it owned the place. 🌊'
|
||||
),
|
||||
(
|
||||
f"💼 **Interviewer:** describe a time you shipped something impactful.\n\n"
|
||||
f"**@{contributor}:** *points at this PR*\n\n"
|
||||
"**Interviewer:** you're hired. 🤝"
|
||||
),
|
||||
]
|
||||
|
||||
# GIFs are repo-hosted under .github/assets/celebrations/ so GitHub's own CDN serves them.
|
||||
_base = "https://raw.githubusercontent.com/Tracer-Cloud/opensre/main/.github/assets/celebrations"
|
||||
gif_blocks: list[str] = [
|
||||
f"\n\n",
|
||||
f"\n\n",
|
||||
f"\n\n",
|
||||
f"\n\n",
|
||||
f"\n\n",
|
||||
f"\n\n",
|
||||
f"\n\n",
|
||||
f"\n\n",
|
||||
f"\n\n",
|
||||
f"\n\n",
|
||||
]
|
||||
|
||||
head = random.choice(templates) + random.choice(gif_blocks)
|
||||
footer = (
|
||||
"---\n\n"
|
||||
f"👋 **Join us on [Discord - OpenSRE]({discord})** : hang out, contribute, "
|
||||
"or hunt for features and issues. Everyone's welcome."
|
||||
)
|
||||
body = f"{head}\n\n{footer}"
|
||||
|
||||
with open("comment.md", "w", encoding="utf-8") as fh:
|
||||
fh.write(body)
|
||||
@@ -0,0 +1,95 @@
|
||||
#!/usr/bin/env bash
|
||||
# Sync Tracer-Cloud/homebrew-tap Formula/opensre.rb version and archive SHAs.
|
||||
#
|
||||
# Usage (CI):
|
||||
# HOMEBREW_TAP_GITHUB_TOKEN=... VERSION=2026.5.29 ASSET_DIR=release-assets ./sync-homebrew-tap-formula.sh
|
||||
#
|
||||
# Local dry-run (no push):
|
||||
# DRY_RUN=1 VERSION=2026.5.29 ASSET_DIR=/path/to/sha256-files ./sync-homebrew-tap-formula.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
VERSION="${VERSION:-}"
|
||||
ASSET_DIR="${ASSET_DIR:-release-assets}"
|
||||
DRY_RUN="${DRY_RUN:-0}"
|
||||
|
||||
if [ -z "$VERSION" ]; then
|
||||
echo "VERSION is required (e.g. 2026.5.29)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
for platform in linux-x64 linux-arm64 darwin-x64 darwin-arm64; do
|
||||
sha_file="${ASSET_DIR}/opensre_${VERSION}_${platform}.tar.gz.sha256"
|
||||
if [ ! -f "$sha_file" ]; then
|
||||
echo "Missing SHA256 file: $sha_file" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
linux_x64_sha="$(awk '{print $1}' "${ASSET_DIR}/opensre_${VERSION}_linux-x64.tar.gz.sha256")"
|
||||
linux_arm64_sha="$(awk '{print $1}' "${ASSET_DIR}/opensre_${VERSION}_linux-arm64.tar.gz.sha256")"
|
||||
darwin_x64_sha="$(awk '{print $1}' "${ASSET_DIR}/opensre_${VERSION}_darwin-x64.tar.gz.sha256")"
|
||||
darwin_arm64_sha="$(awk '{print $1}' "${ASSET_DIR}/opensre_${VERSION}_darwin-arm64.tar.gz.sha256")"
|
||||
|
||||
tap_dir="$(mktemp -d)/homebrew-tap"
|
||||
if [ -n "${HOMEBREW_TAP_GITHUB_TOKEN:-}" ]; then
|
||||
git clone "https://x-access-token:${HOMEBREW_TAP_GITHUB_TOKEN}@github.com/Tracer-Cloud/homebrew-tap.git" "$tap_dir"
|
||||
else
|
||||
git clone "https://github.com/Tracer-Cloud/homebrew-tap.git" "$tap_dir"
|
||||
fi
|
||||
|
||||
python3 - "$tap_dir/Formula/opensre.rb" "$VERSION" "$darwin_arm64_sha" "$darwin_x64_sha" "$linux_arm64_sha" "$linux_x64_sha" <<'PY'
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
formula_path = Path(sys.argv[1])
|
||||
version = sys.argv[2]
|
||||
darwin_arm64_sha = sys.argv[3]
|
||||
darwin_x64_sha = sys.argv[4]
|
||||
linux_arm64_sha = sys.argv[5]
|
||||
linux_x64_sha = sys.argv[6]
|
||||
|
||||
text = formula_path.read_text()
|
||||
text = re.sub(r'version "[^"]+"', f'version "{version}"', text, count=1)
|
||||
replacements = {
|
||||
r'(opensre_#\{version\}_darwin-arm64\.tar\.gz"\n\s*sha256 ")[^"]+(")': darwin_arm64_sha,
|
||||
r'(opensre_#\{version\}_darwin-x64\.tar\.gz"\n\s*sha256 ")[^"]+(")': darwin_x64_sha,
|
||||
r'(opensre_#\{version\}_linux-arm64\.tar\.gz"\n\s*sha256 ")[^"]+(")': linux_arm64_sha,
|
||||
r'(opensre_#\{version\}_linux-x64\.tar\.gz"\n\s*sha256 ")[^"]+(")': linux_x64_sha,
|
||||
}
|
||||
for pattern, sha in replacements.items():
|
||||
text, count = re.subn(pattern, rf'\g<1>{sha}\g<2>', text, count=1)
|
||||
if count != 1:
|
||||
raise SystemExit(f"Failed to update checksum with pattern: {pattern}")
|
||||
formula_path.write_text(text)
|
||||
PY
|
||||
|
||||
cd "$tap_dir"
|
||||
if git diff --quiet -- Formula/opensre.rb; then
|
||||
echo "Homebrew tap formula already up to date for version ${VERSION}."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "=== Formula diff ==="
|
||||
git diff Formula/opensre.rb
|
||||
echo "===================="
|
||||
|
||||
if [ "$DRY_RUN" = "1" ]; then
|
||||
echo "DRY_RUN=1 — not committing or pushing."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ -z "${HOMEBREW_TAP_GITHUB_TOKEN:-}" ]; then
|
||||
echo "HOMEBREW_TAP_GITHUB_TOKEN is required to push." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git add Formula/opensre.rb
|
||||
git commit -m "chore: update opensre formula to ${VERSION}"
|
||||
git push origin HEAD:main
|
||||
echo "Pushed homebrew-tap update for version ${VERSION}."
|
||||
@@ -0,0 +1,17 @@
|
||||
# GitHub Actions (maintainer reference)
|
||||
|
||||
Internal notes for repository automation under `.github/workflows/`. Not published on the docs site.
|
||||
|
||||
## Workflows
|
||||
|
||||
| Workflow | Purpose |
|
||||
| -------- | ------- |
|
||||
| [`ci.yml`](ci.yml) | PR/push quality gates and sharded pytest |
|
||||
| [`ci-labels-windows.yml`](ci-labels-windows.yml) | Optional Windows CI (`ci:windows` label) |
|
||||
| [`codeql.yml`](codeql.yml) | CodeQL security analysis |
|
||||
| [`greptile-pr-reminder.yml`](greptile-pr-reminder.yml) | Greptile review nudge on PR open |
|
||||
| [`celebrate-merged-pr.yml`](celebrate-merged-pr.yml) | Post-merge celebration comment |
|
||||
| [`good-first-issue-assign.yml`](good-first-issue-assign.yml) | Auto-assign good first issues |
|
||||
| [`release.yml`](release.yml) | Release builds and artifacts |
|
||||
|
||||
See [CI.md](../../CI.md) for local parity commands before push.
|
||||
@@ -0,0 +1,176 @@
|
||||
name: Benchmark image — build + push to ECR (any adapter)
|
||||
|
||||
# Adapter-agnostic image build. The bench image carries the full
|
||||
# ``tests/benchmarks/`` tree, so every registered adapter ships in the
|
||||
# same image. ``benchmark-run.yml`` selects which adapter actually runs
|
||||
# via its ``config`` input. See ``tests/benchmarks/_framework/registry.py``
|
||||
# for the registration contract.
|
||||
|
||||
# Builds tests/benchmarks/cloudopsbench/infra/Dockerfile.bench and pushes the resulting image to the
|
||||
# opensre-bench ECR repository. The bench container is what
|
||||
# `Benchmark run (manual)` invokes on AWS Fargate, so this workflow's
|
||||
# output is the input to that one.
|
||||
#
|
||||
# Triggered automatically on changes to the bench code (or the Dockerfile
|
||||
# itself) and manually via workflow_dispatch.
|
||||
#
|
||||
# After the image is pushed, update the task definition by re-applying the
|
||||
# Terraform with the new tag:
|
||||
#
|
||||
# cd tests/benchmarks/cloudopsbench/infra
|
||||
# terraform apply -var="image_tag=<TAG>"
|
||||
#
|
||||
# (Or wire a follow-up step into this workflow to update task definition
|
||||
# automatically — out of scope for v1.)
|
||||
#
|
||||
# Tag format: short git SHA (`git rev-parse --short HEAD`). Stable, unique
|
||||
# per commit, recognizable in `aws ecr describe-images` output. ECR is
|
||||
# IMMUTABLE — a tag pushed once cannot be overwritten, so re-running the
|
||||
# workflow on the same commit just re-tags (no-op layer push).
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: Image tag to push (default = short git SHA)
|
||||
required: false
|
||||
type: string
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- "tests/benchmarks/cloudopsbench/infra/Dockerfile.bench"
|
||||
- "tests/benchmarks/cloudopsbench/infra/Dockerfile.bench.dockerignore"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- "surfaces/**"
|
||||
- "config/**"
|
||||
- "core/**"
|
||||
- "platform/deployment/**"
|
||||
- "integrations/**"
|
||||
- "platform/**"
|
||||
- "tools/**"
|
||||
- "tests/benchmarks/**"
|
||||
- ".github/workflows/benchmark-image.yml"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write # required for AWS OIDC role assumption
|
||||
|
||||
concurrency:
|
||||
# Allow one in-flight build per ref so two pushes in quick succession
|
||||
# don't race ECR. The newer push cancels the older.
|
||||
group: bench-image-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
AWS_REGION: us-east-1
|
||||
# Account ID is repo-level configuration, not a secret. Set as a GitHub
|
||||
# repository Variable (Settings > Secrets and variables > Actions >
|
||||
# Variables) so it doesn't need to be edited in every workflow file
|
||||
# when the bench moves accounts.
|
||||
ACCOUNT_ID: ${{ vars.AWS_ACCOUNT_ID }}
|
||||
ECR_REPOSITORY: opensre-bench
|
||||
|
||||
jobs:
|
||||
build-and-push:
|
||||
if: github.repository == 'Tracer-Cloud/opensre'
|
||||
name: build + push
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v5
|
||||
with:
|
||||
# Fetch enough history that `git rev-parse --short HEAD` is stable
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Resolve image tag
|
||||
id: tag
|
||||
env:
|
||||
# Route `inputs.tag` through env so the shell treats it as DATA,
|
||||
# not code. Without this, a workflow_dispatch caller could inject
|
||||
# shell commands via a crafted `tag` value containing $() or
|
||||
# backticks. See:
|
||||
# https://securitylab.github.com/research/github-actions-untrusted-input/
|
||||
INPUT_TAG: ${{ inputs.tag }}
|
||||
run: |
|
||||
if [ -n "$INPUT_TAG" ]; then
|
||||
TAG="$INPUT_TAG"
|
||||
else
|
||||
TAG="$(git rev-parse --short HEAD)"
|
||||
fi
|
||||
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
|
||||
echo "Will push: $ECR_REPOSITORY:$TAG"
|
||||
|
||||
- name: Configure AWS credentials (OIDC role assumption)
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
role-to-assume: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/opensre-bench-github-actions
|
||||
role-session-name: github-actions-bench-image-${{ github.run_id }}
|
||||
aws-region: us-east-1
|
||||
|
||||
- name: Login to Amazon ECR
|
||||
id: ecr-login
|
||||
uses: aws-actions/amazon-ecr-login@v2
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
# Buildx adds multi-platform support + better cache control. Even
|
||||
# for single-platform builds, it's the modern default.
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Build and push
|
||||
id: build
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: tests/benchmarks/cloudopsbench/infra/Dockerfile.bench
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
tags: |
|
||||
${{ steps.ecr-login.outputs.registry }}/${{ env.ECR_REPOSITORY }}:${{ steps.tag.outputs.tag }}
|
||||
# Stamp the COMMIT SHA into the image so the runtime can read it
|
||||
# via the OPENSRE_SHA env var. We use github.sha (the full 40-char
|
||||
# commit being built), NOT steps.tag.outputs.tag (which resolves
|
||||
# to the user-supplied inputs.tag like ``hotfix-june`` on
|
||||
# workflow_dispatch and would stamp an unverifiable string as if
|
||||
# it were a SHA). The image tag (for ECR naming) and OPENSRE_SHA
|
||||
# (for provenance) are intentionally decoupled — the tag is for
|
||||
# humans/operators; the SHA is for reproducibility. The runtime
|
||||
# gate also validates SHA shape (7-40 lowercase hex chars), so
|
||||
# even a manually-built image with a bad OPENSRE_SHA fails loudly.
|
||||
build-args: |
|
||||
OPENSRE_SHA=${{ github.sha }}
|
||||
# GitHub Actions cache for Docker layers - speeds up rebuilds
|
||||
# when only the source changes (deps stay in the cached layer).
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
provenance: false # smaller manifest; matches ECR's tolerance for OCI
|
||||
|
||||
- name: Summarise
|
||||
# Route step outputs through env so the shell sees them as DATA,
|
||||
# not code. `steps.tag.outputs.tag` is derived verbatim from the
|
||||
# `inputs.tag` user input — without env scoping, a workflow_dispatch
|
||||
# caller could inject shell commands here even though the
|
||||
# "Resolve image tag" step is hardened. Same risk applies to any
|
||||
# tag-derived chain.
|
||||
env:
|
||||
REGISTRY: ${{ steps.ecr-login.outputs.registry }}
|
||||
TAG: ${{ steps.tag.outputs.tag }}
|
||||
DIGEST: ${{ steps.build.outputs.digest }}
|
||||
run: |
|
||||
echo "## Image pushed" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "| field | value |" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "| --- | --- |" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "| Registry | \`$REGISTRY\` |" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "| Repository | \`$ECR_REPOSITORY\` |" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "| Tag | \`$TAG\` |" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "| Digest | \`$DIGEST\` |" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "To deploy this image:" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "\`\`\`bash" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "cd tests/benchmarks/cloudopsbench/infra" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "terraform apply -var=\"image_tag=$TAG\"" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "\`\`\`" >> "$GITHUB_STEP_SUMMARY"
|
||||
@@ -0,0 +1,135 @@
|
||||
name: Benchmark image — promote tag to task definition (any adapter)
|
||||
|
||||
# Adapter-agnostic image promotion. Rebinds the ECS task definition to
|
||||
# a specific image tag. The image carries every registered adapter; the
|
||||
# config supplied to ``benchmark-run.yml`` picks which one runs.
|
||||
#
|
||||
# Manually-triggered workflow that runs `terraform apply -var=image_tag=<TAG>`
|
||||
# in tests/benchmarks/cloudopsbench/infra/ to register a new ECS task definition revision pointing at
|
||||
# the chosen ECR image. This is the privileged "deploy" step that comes
|
||||
# between an image push (automatic) and a bench run (manual).
|
||||
#
|
||||
# Why not auto-promote on every image push? An image build is a code-change
|
||||
# event. A task-def update is a deploy event. Decoupling them lets you
|
||||
# stage many images and choose deliberately which one production runs.
|
||||
#
|
||||
# Trigger from the GitHub UI:
|
||||
# Actions → "Benchmark image — promote tag to task definition" → Run
|
||||
#
|
||||
# Pre-reqs (one-time):
|
||||
# - tests/benchmarks/cloudopsbench/infra/ Terraform applied at least once. The opensre-bench-github-actions
|
||||
# OIDC role's permissions (ecs:RegisterTaskDefinition, state-bucket read/write,
|
||||
# lock-table read/write, iam:PassRole) are granted by the github_actions_run_bench
|
||||
# inline policy in tests/benchmarks/cloudopsbench/infra/iam_oidc.tf — any apply of that module attaches them.
|
||||
# - Repo secrets seeded into AWS Secrets Manager
|
||||
# - Repo vars set (AWS_ACCOUNT_ID etc., see tests/benchmarks/cloudopsbench/infra/AWS_BENCH_SETUP.md step 4)
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
image_tag:
|
||||
description: 'ECR image tag to promote (e.g. 3792493)'
|
||||
required: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write # required for AWS OIDC role assumption
|
||||
|
||||
concurrency:
|
||||
group: benchmark-promote-image
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
promote:
|
||||
name: terraform apply image_tag=${{ inputs.image_tag }}
|
||||
if: github.repository == 'Tracer-Cloud/opensre'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
|
||||
env:
|
||||
AWS_REGION: us-east-1
|
||||
|
||||
steps:
|
||||
- name: Verify required repo variables
|
||||
env:
|
||||
AWS_ACCOUNT_ID: ${{ vars.AWS_ACCOUNT_ID }}
|
||||
run: |
|
||||
if [ -z "${AWS_ACCOUNT_ID:-}" ]; then
|
||||
echo "::error::Missing repo variable AWS_ACCOUNT_ID. See tests/benchmarks/cloudopsbench/infra/AWS_BENCH_SETUP.md."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Configure AWS credentials (OIDC role assumption)
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
role-to-assume: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/opensre-bench-github-actions
|
||||
role-session-name: bench-promote-${{ github.run_id }}
|
||||
aws-region: us-east-1
|
||||
|
||||
- name: Verify image tag exists in ECR
|
||||
# Fail loudly if the operator typos the tag, before Terraform tries
|
||||
# to register a task definition pointing at a missing image.
|
||||
env:
|
||||
IMAGE_TAG: ${{ inputs.image_tag }}
|
||||
run: |
|
||||
if ! aws ecr describe-images \
|
||||
--repository-name opensre-bench \
|
||||
--image-ids imageTag="$IMAGE_TAG" \
|
||||
>/dev/null 2>&1; then
|
||||
echo "::error::Image tag $IMAGE_TAG not found in ECR repo opensre-bench."
|
||||
echo "::error::Push it first via 'Benchmark image — build + push to ECR'."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- uses: hashicorp/setup-terraform@v3
|
||||
with:
|
||||
terraform_version: 1.7.5
|
||||
|
||||
- name: Terraform init
|
||||
working-directory: tests/benchmarks/cloudopsbench/infra
|
||||
run: terraform init -input=false
|
||||
|
||||
- name: Terraform apply
|
||||
# Plan is captured in the workflow log; review it in the run page.
|
||||
# -auto-approve is intentional — this workflow IS the human approval
|
||||
# (the operator triggered it manually with a specific tag).
|
||||
#
|
||||
# -target=aws_ecs_task_definition.bench scopes apply to the task
|
||||
# definition (and its data-source / role dependencies) only. Without
|
||||
# this, every dispatch tries to reconcile every resource in the
|
||||
# module — IAM, S3, ECR, etc. — against whatever ref the workflow
|
||||
# was dispatched from. Out-of-band local applies cause that
|
||||
# reconciliation to attempt rollbacks the workflow role isn't
|
||||
# permitted to perform (iam:DetachRolePolicy, iam:PutRolePolicy),
|
||||
# failing the run even when the task-def update itself succeeded.
|
||||
working-directory: tests/benchmarks/cloudopsbench/infra
|
||||
env:
|
||||
IMAGE_TAG: ${{ inputs.image_tag }}
|
||||
run: |
|
||||
terraform apply -input=false -auto-approve \
|
||||
-target=aws_ecs_task_definition.bench \
|
||||
-var="image_tag=$IMAGE_TAG"
|
||||
|
||||
- name: Surface the new task definition revision in the job summary
|
||||
working-directory: tests/benchmarks/cloudopsbench/infra
|
||||
env:
|
||||
IMAGE_TAG: ${{ inputs.image_tag }}
|
||||
run: |
|
||||
TASK_DEF_ARN=$(terraform output -raw task_definition_arn)
|
||||
IMAGE_URI=$(aws ecs describe-task-definition \
|
||||
--task-definition "$TASK_DEF_ARN" \
|
||||
--query 'taskDefinition.containerDefinitions[0].image' \
|
||||
--output text)
|
||||
{
|
||||
echo "## Image promoted"
|
||||
echo ""
|
||||
echo "- Promoted tag: \`$IMAGE_TAG\`"
|
||||
echo "- New task definition ARN: \`$TASK_DEF_ARN\`"
|
||||
echo "- Image now in task definition: \`$IMAGE_URI\`"
|
||||
echo ""
|
||||
echo "### Next step"
|
||||
echo ""
|
||||
echo "Trigger **Benchmark — run on Fargate** to launch a run against this image."
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
@@ -0,0 +1,45 @@
|
||||
name: Update Benchmark README Section
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- "docs/benchmarks/results.md"
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
update-benchmark-readme:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Update README benchmark section
|
||||
run: uv run python -m tests.benchmarks.toolcall_model_benchmark.readme_updater
|
||||
|
||||
- name: Commit changes
|
||||
run: |
|
||||
git config user.name "benchmark-readme-action"
|
||||
git config user.email "benchmark-readme-action@noreply.github.com"
|
||||
git add README.md
|
||||
git diff --staged --quiet || git commit -m "docs(benchmark): update README with latest benchmark results"
|
||||
git pull --rebase origin main
|
||||
git push
|
||||
@@ -0,0 +1,191 @@
|
||||
name: Benchmark — run on Fargate (any adapter)
|
||||
|
||||
# Manually-triggered benchmark run on AWS Fargate.
|
||||
#
|
||||
# Adapter-agnostic. This workflow does NOT know or care which benchmark
|
||||
# is running. The ``config`` input names a YAML file; the YAML names an
|
||||
# adapter (``benchmark: <name>``); the framework's registry resolves it
|
||||
# to a registered ``BenchmarkAdapter`` subclass. The same workflow runs
|
||||
# any adapter packaged into the bench image — CloudOpsBench today,
|
||||
# ToolCallBench / future adapters tomorrow, with zero changes
|
||||
# here. See ``tests/benchmarks/_framework/registry.py``.
|
||||
#
|
||||
# Why ECS RunTask instead of a GitHub-hosted ubuntu runner: a full bench
|
||||
# grid runs for hours and writes hundreds of MB of artifacts — Fargate
|
||||
# handles it cleanly, has AWS Secrets Manager wired in via tests/benchmarks/cloudopsbench/infra/,
|
||||
# and writes to the per-run S3 bucket. The workflow's job is just to
|
||||
# launch the task and print where to watch.
|
||||
#
|
||||
# Trigger from the GitHub UI:
|
||||
# Actions → "Benchmark run (manual)" → Run workflow → fill inputs
|
||||
#
|
||||
# Pre-reqs (one-time, see tests/benchmarks/cloudopsbench/infra/README.md):
|
||||
# - tests/benchmarks/cloudopsbench/infra/ Terraform applied
|
||||
# - Bench image pushed to ECR via benchmark-image.yml
|
||||
# - Repo secrets seeded into AWS Secrets Manager via benchmark-seed-secret.yml
|
||||
# - Repo variables set:
|
||||
# AWS_ACCOUNT_ID, BENCH_ECS_CLUSTER, BENCH_TASK_DEFINITION_FAMILY,
|
||||
# BENCH_SUBNET_IDS (comma-separated), BENCH_SECURITY_GROUP_ID
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
config:
|
||||
# No adapter-specific default — the operator must point at a
|
||||
# specific config. Leaving the field empty surfaces the choice
|
||||
# rather than silently dispatching the CloudOpsBench smoke. The
|
||||
# path is relative to the container's repo root. Examples:
|
||||
# tests/benchmarks/cloudopsbench/configs/cloudopsbench_smoke.yml
|
||||
# tests/benchmarks/<adapter>/configs/<config>.yml
|
||||
description: 'Path to YAML config inside the container (e.g. tests/benchmarks/<adapter>/configs/<config>.yml)'
|
||||
required: true
|
||||
dev_mode:
|
||||
description: 'Dev mode (skip integrity gates, no pre-reg needed)'
|
||||
type: boolean
|
||||
default: true
|
||||
|
||||
# Note: there is intentionally no `image_tag` input here. ECS RunTask
|
||||
# container overrides do NOT support overriding the image URI — that lives
|
||||
# on the task definition itself. The image actually pulled is whatever
|
||||
# was registered the last time `terraform apply -var=image_tag=<tag>` ran
|
||||
# in tests/benchmarks/cloudopsbench/infra/. Adding a workflow input here would silently mislead
|
||||
# operators into thinking they control the image per run. To change the
|
||||
# image, push it via `Benchmark image — build + push to ECR`, then re-apply
|
||||
# Terraform with the new tag.
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write # required for AWS OIDC role assumption
|
||||
|
||||
concurrency:
|
||||
group: benchmark-run
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
launch:
|
||||
name: launch ECS task
|
||||
if: github.repository == 'Tracer-Cloud/opensre'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15 # we only LAUNCH the task; we don't wait for it
|
||||
|
||||
env:
|
||||
AWS_REGION: us-east-1
|
||||
|
||||
steps:
|
||||
- name: Verify required repo variables
|
||||
# Fail loudly BEFORE the AWS auth step if any required repo variable
|
||||
# is missing. Without this, an unset var would surface downstream as
|
||||
# an opaque error like "Task Definition can not be blank" from
|
||||
# describe-task-definition. See tests/benchmarks/cloudopsbench/infra/AWS_BENCH_SETUP.md step 4 for how
|
||||
# to set these.
|
||||
env:
|
||||
AWS_ACCOUNT_ID: ${{ vars.AWS_ACCOUNT_ID }}
|
||||
BENCH_ECS_CLUSTER: ${{ vars.BENCH_ECS_CLUSTER }}
|
||||
BENCH_TASK_DEFINITION_FAMILY: ${{ vars.BENCH_TASK_DEFINITION_FAMILY }}
|
||||
BENCH_SUBNET_IDS: ${{ vars.BENCH_SUBNET_IDS }}
|
||||
BENCH_SECURITY_GROUP_ID: ${{ vars.BENCH_SECURITY_GROUP_ID }}
|
||||
run: |
|
||||
missing=()
|
||||
for var in AWS_ACCOUNT_ID BENCH_ECS_CLUSTER BENCH_TASK_DEFINITION_FAMILY \
|
||||
BENCH_SUBNET_IDS BENCH_SECURITY_GROUP_ID; do
|
||||
if [ -z "${!var:-}" ]; then
|
||||
missing+=("$var")
|
||||
fi
|
||||
done
|
||||
if [ "${#missing[@]}" -gt 0 ]; then
|
||||
echo "::error::Missing repo variable(s): ${missing[*]}"
|
||||
echo "::error::Set them under Settings → Secrets and variables → Actions → Variables."
|
||||
echo "::error::Values come from \`cd tests/benchmarks/cloudopsbench/infra && terraform output\` — see tests/benchmarks/cloudopsbench/infra/AWS_BENCH_SETUP.md step 4."
|
||||
exit 1
|
||||
fi
|
||||
echo "All 5 required repo variables are set."
|
||||
|
||||
- name: Configure AWS credentials (OIDC role assumption)
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
role-to-assume: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/opensre-bench-github-actions
|
||||
role-session-name: bench-run-${{ github.run_id }}
|
||||
aws-region: us-east-1
|
||||
|
||||
- name: Launch ECS RunTask
|
||||
# Inputs bound to env vars so the shell does the interpolation —
|
||||
# GitHub Actions template syntax (${{ ... }}) is expanded before
|
||||
# the shell sees it, which would let a crafted input inject shell
|
||||
# metacharacters. Env vars are quoted at use-site.
|
||||
env:
|
||||
BENCH_CONFIG: ${{ inputs.config }}
|
||||
BENCH_DEV_FLAG: ${{ inputs.dev_mode == true && '--dev' || '' }}
|
||||
CLUSTER: ${{ vars.BENCH_ECS_CLUSTER }}
|
||||
TASK_FAMILY: ${{ vars.BENCH_TASK_DEFINITION_FAMILY }}
|
||||
SUBNETS: ${{ vars.BENCH_SUBNET_IDS }}
|
||||
SECURITY_GROUP: ${{ vars.BENCH_SECURITY_GROUP_ID }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
# Surface the image the task definition will actually pull, so
|
||||
# operators see the truth (not a workflow input that ECS ignores).
|
||||
IMAGE_URI=$(aws ecs describe-task-definition \
|
||||
--task-definition "$TASK_FAMILY" \
|
||||
--query 'taskDefinition.containerDefinitions[0].image' \
|
||||
--output text)
|
||||
|
||||
# Overrides JSON: container env vars passed at runtime. The
|
||||
# container's entrypoint reads BENCH_CONFIG + BENCH_DEV_FLAG
|
||||
# and invokes:
|
||||
# uv run python -m tests.benchmarks._framework.cli run \
|
||||
# "$BENCH_CONFIG" $BENCH_DEV_FLAG
|
||||
OVERRIDES=$(jq -nc \
|
||||
--arg config "$BENCH_CONFIG" \
|
||||
--arg dev_flag "$BENCH_DEV_FLAG" \
|
||||
'{
|
||||
containerOverrides: [{
|
||||
name: "bench",
|
||||
environment: [
|
||||
{name: "BENCH_CONFIG", value: $config},
|
||||
{name: "BENCH_DEV_FLAG", value: $dev_flag}
|
||||
]
|
||||
}]
|
||||
}')
|
||||
|
||||
TASK_ARN=$(aws ecs run-task \
|
||||
--cluster "$CLUSTER" \
|
||||
--task-definition "$TASK_FAMILY" \
|
||||
--launch-type FARGATE \
|
||||
--network-configuration "awsvpcConfiguration={subnets=[$SUBNETS],securityGroups=[$SECURITY_GROUP],assignPublicIp=ENABLED}" \
|
||||
--overrides "$OVERRIDES" \
|
||||
--query 'tasks[0].taskArn' \
|
||||
--output text)
|
||||
|
||||
if [ -z "$TASK_ARN" ] || [ "$TASK_ARN" = "None" ]; then
|
||||
echo "::error::run-task returned no taskArn"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
TASK_ID="${TASK_ARN##*/}"
|
||||
echo "task_arn=$TASK_ARN" >> "$GITHUB_OUTPUT"
|
||||
echo "task_id=$TASK_ID" >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Job summary — visible in the workflow run page
|
||||
{
|
||||
echo "## Bench run launched"
|
||||
echo ""
|
||||
echo "- Image (from task definition): \`$IMAGE_URI\`"
|
||||
echo "- Config: \`$BENCH_CONFIG\`"
|
||||
echo "- Dev mode: \`${BENCH_DEV_FLAG:-(off)}\`"
|
||||
echo "- Task ARN: \`$TASK_ARN\`"
|
||||
echo ""
|
||||
echo "### Watch it"
|
||||
echo ""
|
||||
echo "Stream logs locally:"
|
||||
echo ""
|
||||
echo '```bash'
|
||||
echo "aws logs tail /ecs/opensre-bench --follow"
|
||||
echo '```'
|
||||
echo ""
|
||||
echo "Or via AWS Console:"
|
||||
echo "https://us-east-1.console.aws.amazon.com/ecs/v2/clusters/$CLUSTER/tasks/$TASK_ID"
|
||||
echo ""
|
||||
echo "Artifacts land in the bench S3 bucket under \`runs/\` when the task completes."
|
||||
echo ""
|
||||
echo "_To change the running image: push a new tag via \`Benchmark image — build + push to ECR\`, then \`cd tests/benchmarks/cloudopsbench/infra && terraform apply -var=image_tag=<tag>\`._"
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
@@ -0,0 +1,102 @@
|
||||
name: Benchmark secret — seed GitHub repo secret into AWS Secrets Manager
|
||||
|
||||
# Manually-triggered workflow that copies a GitHub repo secret into AWS
|
||||
# Secrets Manager at opensre-bench/llm/<secret>. The bench container reads
|
||||
# its LLM API keys from Secrets Manager at runtime; this workflow is how
|
||||
# you put them there without touching a developer laptop.
|
||||
#
|
||||
# The `secret` dropdown enforces which target is valid — must match one
|
||||
# of the four IAM-granted secret ARNs in tests/benchmarks/cloudopsbench/infra/iam_oidc.tf
|
||||
# (anthropic / openai / deepseek / hf_token).
|
||||
#
|
||||
# Why a workflow instead of a developer running `aws secretsmanager
|
||||
# put-secret-value` locally: keeps keys off developer laptops + out of
|
||||
# shell history, and centralizes rotation — rotate the value in the GH
|
||||
# repo secret and re-run this workflow to propagate.
|
||||
#
|
||||
# Pre-reqs:
|
||||
# - tests/benchmarks/cloudopsbench/infra/ Terraform has been applied (the secret resource exists)
|
||||
# - Corresponding GitHub repo secret is set:
|
||||
# ANTHROPIC_API_KEY / OPENAI_API_KEY / DEEPSEEK_API_KEY / HF_TOKEN
|
||||
#
|
||||
# Order of operations:
|
||||
# 1. terraform-bench.yml (plan) on PR
|
||||
# 2. terraform apply locally
|
||||
# 3. this workflow (workflow_dispatch) — pick the secret to seed
|
||||
# from the dropdown, repeat once per LLM provider key
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
secret:
|
||||
description: Which secret to seed (must match the AWS resource name)
|
||||
required: true
|
||||
type: choice
|
||||
options:
|
||||
- anthropic_api_key
|
||||
- openai_api_key
|
||||
- deepseek_api_key
|
||||
- hf_token
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write # required for AWS OIDC role assumption
|
||||
|
||||
concurrency:
|
||||
group: bench-seed-${{ inputs.secret }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
seed:
|
||||
name: seed ${{ inputs.secret }}
|
||||
if: github.repository == 'Tracer-Cloud/opensre'
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
env:
|
||||
AWS_REGION: us-east-1
|
||||
SECRET_ID: opensre-bench/llm/${{ inputs.secret }}
|
||||
|
||||
steps:
|
||||
- name: Configure AWS credentials (OIDC role assumption)
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
role-to-assume: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/opensre-bench-github-actions
|
||||
role-session-name: github-actions-seed-${{ inputs.secret }}-${{ github.run_id }}
|
||||
aws-region: us-east-1
|
||||
|
||||
- name: Verify secret resource exists
|
||||
run: |
|
||||
if ! aws secretsmanager describe-secret --secret-id "$SECRET_ID" >/dev/null 2>&1; then
|
||||
echo "::error::Secret $SECRET_ID not found. Run \`terraform apply\` in tests/benchmarks/cloudopsbench/infra/ first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Put secret value
|
||||
# All four candidate values are bound at workflow level. The case
|
||||
# statement picks the one matching the chosen target — unused
|
||||
# values stay as masked env vars (GH redacts secret refs in logs).
|
||||
env:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
|
||||
HF_TOKEN: ${{ secrets.HF_TOKEN }}
|
||||
TARGET: ${{ inputs.secret }}
|
||||
run: |
|
||||
case "$TARGET" in
|
||||
anthropic_api_key) V="$ANTHROPIC_API_KEY" ;;
|
||||
openai_api_key) V="$OPENAI_API_KEY" ;;
|
||||
deepseek_api_key) V="$DEEPSEEK_API_KEY" ;;
|
||||
hf_token) V="$HF_TOKEN" ;;
|
||||
*)
|
||||
echo "::error::Unknown target: $TARGET (dropdown should have prevented this)"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
if [ -z "$V" ]; then
|
||||
echo "::error::GitHub repo secret for $TARGET is unset. Configure it under Settings > Secrets and variables > Actions."
|
||||
exit 1
|
||||
fi
|
||||
aws secretsmanager put-secret-value \
|
||||
--secret-id "$SECRET_ID" \
|
||||
--secret-string "$V" >/dev/null
|
||||
echo "Seeded $SECRET_ID (length=${#V})."
|
||||
@@ -0,0 +1,37 @@
|
||||
name: Celebrate merged pull requests
|
||||
|
||||
# Runs after a PR lands (merged into the target branch). Uses pull_request_target so the
|
||||
# token can comment on merged PRs from forks; checkout is base/default only — no PR head.
|
||||
on:
|
||||
pull_request_target:
|
||||
types: [closed]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
celebrate-merge:
|
||||
name: Post-merge celebration comment
|
||||
runs-on: ubuntu-latest
|
||||
if: >-
|
||||
github.event.pull_request.merged == true &&
|
||||
github.event.pull_request.user.type != 'Bot' &&
|
||||
github.event.pull_request.user.login != 'dependabot[bot]'
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Build celebration comment
|
||||
id: celebrate
|
||||
env:
|
||||
DISCORD_INVITE_URL: https://discord.com/invite/opensre
|
||||
CONTRIBUTOR_LOGIN: ${{ github.event.pull_request.user.login }}
|
||||
run: python3 .github/scripts/merged_pr_celebration_message.py
|
||||
|
||||
- name: Comment on merged PR
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: >
|
||||
gh pr comment "${{ github.event.pull_request.number }}"
|
||||
--repo "${{ github.repository }}"
|
||||
--body-file comment.md
|
||||
@@ -0,0 +1,109 @@
|
||||
# Optional Windows CI when PR label `ci:windows` is present.
|
||||
# Separate workflow + concurrency group so Windows label runs are isolated.
|
||||
|
||||
name: CI Labels (Windows)
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
types: [opened, synchronize, reopened, labeled, unlabeled]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
|
||||
# Keep in sync with ci.yml — single source of truth for lint/typecheck/coverage targets.
|
||||
PYTHON_SOURCE_PATHS: "config core integrations platform surfaces tools"
|
||||
|
||||
concurrency:
|
||||
group: ci-labels-windows-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
windows-quality:
|
||||
if: contains(github.event.pull_request.labels.*.name, 'ci:windows')
|
||||
name: windows quality
|
||||
runs-on: windows-latest
|
||||
timeout-minutes: 15
|
||||
continue-on-error: true
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Lint
|
||||
run: uv run python -m ruff check ${{ env.PYTHON_SOURCE_PATHS }} tests/
|
||||
|
||||
- name: Format check
|
||||
run: uv run python -m ruff format --check ${{ env.PYTHON_SOURCE_PATHS }} tests/
|
||||
|
||||
- name: Typecheck
|
||||
run: uv run python -m mypy ${{ env.PYTHON_SOURCE_PATHS }}
|
||||
|
||||
windows-test:
|
||||
if: contains(github.event.pull_request.labels.*.name, 'ci:windows')
|
||||
name: windows test
|
||||
needs: [windows-quality]
|
||||
runs-on: windows-latest
|
||||
timeout-minutes: 25
|
||||
continue-on-error: true
|
||||
|
||||
env:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
JWT_TOKEN: ${{ secrets.JWT_TOKEN }}
|
||||
TRACER_ORG_ID: ${{ secrets.TRACER_ORG_ID }}
|
||||
TRACER_WEB_APP_URL: ${{ secrets.TRACER_WEB_APP_URL }}
|
||||
GRAFANA_READ_TOKEN: ${{ secrets.GRAFANA_READ_TOKEN }}
|
||||
GCLOUD_HOSTED_METRICS_ID: ${{ secrets.GCLOUD_HOSTED_METRICS_ID }}
|
||||
GCLOUD_HOSTED_METRICS_URL: ${{ secrets.GCLOUD_HOSTED_METRICS_URL }}
|
||||
GCLOUD_HOSTED_LOGS_ID: ${{ secrets.GCLOUD_HOSTED_LOGS_ID }}
|
||||
GCLOUD_HOSTED_LOGS_URL: ${{ secrets.GCLOUD_HOSTED_LOGS_URL }}
|
||||
GCLOUD_RW_API_KEY: ${{ secrets.GCLOUD_RW_API_KEY }}
|
||||
GCLOUD_OTLP_ENDPOINT: ${{ secrets.GCLOUD_OTLP_ENDPOINT }}
|
||||
GCLOUD_OTLP_AUTH_HEADER: ${{ secrets.GCLOUD_OTLP_AUTH_HEADER }}
|
||||
PYTHONUTF8: "1"
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Run full tests
|
||||
run: >-
|
||||
uv run python -m pytest -n auto -v
|
||||
--cov=config --cov=core --cov=platform.deployment.aws --cov=platform.deployment --cov=integrations --cov=platform --cov=surfaces --cov=tools
|
||||
--cov-report=term-missing
|
||||
--cov-report=html
|
||||
--ignore=tests/e2e/kubernetes_local_alert_simulation
|
||||
--ignore=tests/synthetic
|
||||
-m "not synthetic and not integration"
|
||||
@@ -0,0 +1,468 @@
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "surfaces/**"
|
||||
- "config/**"
|
||||
- "core/**"
|
||||
- "platform/deployment/**"
|
||||
- "integrations/**"
|
||||
- "platform/**"
|
||||
- "tools/**"
|
||||
- "tests/**"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- "pytest.ini"
|
||||
- "ruff.toml"
|
||||
- "mypy.ini"
|
||||
- "Makefile"
|
||||
- ".importlinter"
|
||||
- ".importlinter.strict"
|
||||
- ".github/ci/**"
|
||||
- ".github/workflows/ci.yml"
|
||||
|
||||
pull_request:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "surfaces/**"
|
||||
- "config/**"
|
||||
- "core/**"
|
||||
- "platform/deployment/**"
|
||||
- "integrations/**"
|
||||
- "platform/**"
|
||||
- "tools/**"
|
||||
- "tests/**"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- "pytest.ini"
|
||||
- "ruff.toml"
|
||||
- "mypy.ini"
|
||||
- "Makefile"
|
||||
- ".importlinter"
|
||||
- ".importlinter.strict"
|
||||
- ".github/ci/**"
|
||||
- ".github/workflows/ci.yml"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
|
||||
PYTHON_SOURCE_PATHS: "config core integrations platform surfaces tools"
|
||||
|
||||
concurrency:
|
||||
group: ci-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
quality:
|
||||
name: quality (${{ matrix.os }})
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
os: [ubuntu-latest]
|
||||
|
||||
runs-on: ${{ matrix.os }}
|
||||
timeout-minutes: 15
|
||||
continue-on-error: ${{ matrix.os == 'windows-latest' }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Lint
|
||||
run: uv run python -m ruff check $PYTHON_SOURCE_PATHS tests/
|
||||
|
||||
- name: Format check
|
||||
run: uv run python -m ruff format --check $PYTHON_SOURCE_PATHS tests/
|
||||
|
||||
- name: Typecheck
|
||||
run: uv run python -m mypy $PYTHON_SOURCE_PATHS
|
||||
|
||||
- name: Import graph
|
||||
run: uv run python .github/ci/check_imports.py --strict
|
||||
|
||||
# Full layered contracts (.importlinter.strict), not the minimal default
|
||||
# .importlinter config used by ``make check-imports``.
|
||||
- name: Import boundaries
|
||||
run: uv run lint-imports --config .importlinter.strict
|
||||
|
||||
test:
|
||||
name: test (${{ matrix.shard }})
|
||||
strategy:
|
||||
fail-fast: false
|
||||
max-parallel: 5
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
shard: tools-runtime
|
||||
# ``tests/core/agent`` is excluded here; it runs in cli-runtime,
|
||||
# which pins openai for its live action-planning oracles.
|
||||
extra_pytest_args: >-
|
||||
--ignore=tests/core/agent
|
||||
pytest_paths: >-
|
||||
tests/tools
|
||||
tests/core
|
||||
tests/platform
|
||||
tests/utils
|
||||
tests/masking
|
||||
tests/watch_dog
|
||||
- os: ubuntu-latest
|
||||
shard: cli-runtime
|
||||
# Live shell action-agent contracts are validated on openai,
|
||||
# matching interactive-shell-live.yml. The
|
||||
# default anthropic tool-call model (haiku) cannot reliably perform
|
||||
# compound action planning, so pin this shard to openai.
|
||||
# ``tests/core/agent`` runs here because it contains the live
|
||||
# action-planning oracles (``test_live_action_planning`` and
|
||||
# ``test_live_turn_execution_oracle``) that need the openai pin —
|
||||
# plus the static ``test_import_boundaries`` guard the surfaces
|
||||
# restructure (#3299) added.
|
||||
llm_provider: openai
|
||||
pytest_paths: >-
|
||||
tests/cli
|
||||
tests/interactive_shell
|
||||
tests/agent
|
||||
tests/core/agent
|
||||
tests/fleet_monitoring
|
||||
tests/analytics
|
||||
tests/tools/investigation
|
||||
tests/delivery
|
||||
tests/sandbox
|
||||
tests/hermes
|
||||
- os: ubuntu-latest
|
||||
shard: integrations-and-misc
|
||||
pytest_paths: >-
|
||||
tests/integrations
|
||||
tests/deployment
|
||||
tests/benchmarks
|
||||
tests/chaos_engineering
|
||||
tests/shared
|
||||
tests/config
|
||||
tests/github_ci
|
||||
tests/packaging
|
||||
tests/scheduler
|
||||
gateway/tests
|
||||
- os: ubuntu-latest
|
||||
shard: e2e-general
|
||||
pytest_paths: >-
|
||||
tests/e2e
|
||||
extra_pytest_args: >-
|
||||
--ignore=tests/e2e/kubernetes
|
||||
--ignore=tests/e2e/openclaw
|
||||
--ignore=tests/e2e/upstream_lambda
|
||||
--ignore=tests/e2e/upstream_prefect_ecs_fargate
|
||||
--ignore=tests/e2e/upstream_apache_flink_ecs
|
||||
- os: ubuntu-latest
|
||||
shard: e2e-provider-and-openclaw
|
||||
pytest_paths: >-
|
||||
tests/e2e/openclaw
|
||||
tests/e2e/upstream_lambda
|
||||
tests/e2e/upstream_prefect_ecs_fargate
|
||||
tests/e2e/upstream_apache_flink_ecs
|
||||
tests/e2e/kubernetes
|
||||
|
||||
runs-on: ${{ matrix.os }}
|
||||
timeout-minutes: 30
|
||||
|
||||
env:
|
||||
# Per-shard LLM provider for live_llm contracts; defaults to anthropic.
|
||||
LLM_PROVIDER: ${{ matrix.llm_provider || 'anthropic' }}
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
JWT_TOKEN: ${{ secrets.JWT_TOKEN }}
|
||||
GRAFANA_READ_TOKEN: ${{ secrets.GRAFANA_READ_TOKEN }}
|
||||
GCLOUD_HOSTED_METRICS_ID: ${{ secrets.GCLOUD_HOSTED_METRICS_ID }}
|
||||
GCLOUD_HOSTED_METRICS_URL: ${{ secrets.GCLOUD_HOSTED_METRICS_URL }}
|
||||
GCLOUD_HOSTED_LOGS_ID: ${{ secrets.GCLOUD_HOSTED_LOGS_ID }}
|
||||
GCLOUD_HOSTED_LOGS_URL: ${{ secrets.GCLOUD_HOSTED_LOGS_URL }}
|
||||
GCLOUD_RW_API_KEY: ${{ secrets.GCLOUD_RW_API_KEY }}
|
||||
GCLOUD_OTLP_ENDPOINT: ${{ secrets.GCLOUD_OTLP_ENDPOINT }}
|
||||
GCLOUD_OTLP_AUTH_HEADER: ${{ secrets.GCLOUD_OTLP_AUTH_HEADER }}
|
||||
PYTHONUTF8: "1"
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Validate pytest marker
|
||||
env:
|
||||
# Fork PRs do not receive repository secrets; skip live_llm contracts
|
||||
# (see interactive-shell-live.yml). Index a JSON string array so the
|
||||
# value is always a marker string — never a bare boolean from
|
||||
# ``a && 'x' || 'y'`` (boolean false → ``-m false`` → xdist
|
||||
# ``N workers [0 items]``, which can still exit 0).
|
||||
PYTEST_MARKER_EXPR: ${{ fromJSON('["not synthetic", "not (synthetic or live_llm)"]')[github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true] }}
|
||||
run: |
|
||||
echo "PYTEST_MARKER_EXPR=${PYTEST_MARKER_EXPR}"
|
||||
case "${PYTEST_MARKER_EXPR}" in
|
||||
"not synthetic"|"not (synthetic or live_llm)") ;;
|
||||
*)
|
||||
echo "Refusing unexpected PYTEST_MARKER_EXPR: '${PYTEST_MARKER_EXPR}'" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
- name: Run tests
|
||||
env:
|
||||
COVERAGE_FILE: .coverage.${{ matrix.shard }}
|
||||
PYTEST_MARKER_EXPR: ${{ fromJSON('["not synthetic", "not (synthetic or live_llm)"]')[github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true] }}
|
||||
# Keep ``run: >`` (folded) so ``matrix.pytest_paths`` stays one argv list.
|
||||
# A ``run: |`` + ``\`` rewrite can drop/split those paths and yield
|
||||
# ``N workers [0 items]`` even when the marker is correct.
|
||||
run: >
|
||||
uv run python -m pytest -n auto -v
|
||||
${{ matrix.pytest_paths }}
|
||||
--cov=config
|
||||
--cov=core
|
||||
--cov=platform.deployment.aws --cov=platform.deployment
|
||||
--cov=integrations
|
||||
--cov=platform
|
||||
--cov=surfaces
|
||||
--cov=tools
|
||||
--cov-report=
|
||||
--ignore=tests/e2e/kubernetes_local_alert_simulation
|
||||
--ignore=tests/synthetic
|
||||
${{ matrix.extra_pytest_args }}
|
||||
-m "${PYTEST_MARKER_EXPR}"
|
||||
|
||||
- name: Upload shard coverage data
|
||||
if: >-
|
||||
always() &&
|
||||
github.event_name == 'push' &&
|
||||
github.ref == 'refs/heads/main'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: coverage-data-${{ matrix.shard }}
|
||||
path: .coverage.${{ matrix.shard }}*
|
||||
if-no-files-found: error
|
||||
include-hidden-files: true
|
||||
retention-days: 7
|
||||
|
||||
coverage-report:
|
||||
name: coverage-report
|
||||
runs-on: ubuntu-latest
|
||||
needs: [test]
|
||||
if: >-
|
||||
always() &&
|
||||
github.event_name == 'push' &&
|
||||
github.ref == 'refs/heads/main'
|
||||
timeout-minutes: 15
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Download shard coverage artifacts
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
pattern: coverage-data-*
|
||||
path: ./
|
||||
merge-multiple: true
|
||||
|
||||
- name: Combine coverage and build report
|
||||
run: |
|
||||
uv run python -m coverage combine .
|
||||
uv run python -m coverage html
|
||||
|
||||
- name: Upload combined coverage
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: coverage-report-ubuntu-latest
|
||||
path: |
|
||||
htmlcov/
|
||||
.coverage
|
||||
retention-days: 7
|
||||
|
||||
test-kubernetes:
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true
|
||||
timeout-minutes: 15
|
||||
|
||||
if: >-
|
||||
github.repository == 'Tracer-Cloud/opensre' &&
|
||||
(github.event_name == 'push' ||
|
||||
(
|
||||
github.event_name == 'pull_request' &&
|
||||
(
|
||||
contains(github.event.pull_request.title, 'k8s') ||
|
||||
contains(github.event.pull_request.title, 'kubernetes')
|
||||
)
|
||||
))
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Run Kubernetes tests
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
AWS_DEFAULT_REGION: us-east-1
|
||||
run: uv run python -m tests.e2e.kubernetes.test_local
|
||||
|
||||
should-run-thorough:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && github.repository == 'Tracer-Cloud/opensre'
|
||||
outputs:
|
||||
thorough: ${{ steps.filter.outputs.thorough }}
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
- uses: dorny/paths-filter@v4
|
||||
id: filter
|
||||
with:
|
||||
filters: |
|
||||
thorough:
|
||||
- ".github/workflows/ci.yml"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- "platform/deployment/**"
|
||||
- "integrations/**"
|
||||
- "tools/investigation/**"
|
||||
- "tools/**"
|
||||
- "tests/e2e/cloudwatch_demo/**"
|
||||
- "tests/e2e/upstream_lambda/**"
|
||||
- "tests/e2e/upstream_prefect_ecs_fargate/**"
|
||||
- "tests/e2e/upstream_apache_flink_ecs/**"
|
||||
|
||||
test-thorough:
|
||||
runs-on: ubuntu-latest
|
||||
needs: [test, should-run-thorough]
|
||||
if: >-
|
||||
github.event_name == 'push' &&
|
||||
github.ref == 'refs/heads/main' &&
|
||||
github.repository == 'Tracer-Cloud/opensre' &&
|
||||
needs.should-run-thorough.outputs.thorough == 'true'
|
||||
timeout-minutes: 30
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- name: cloudwatch-demo
|
||||
command: python3 -m tests.e2e.cloudwatch_demo.test_aws
|
||||
- name: upstream-lambda
|
||||
command: python3 -m tests.e2e.upstream_lambda.test_agent_e2e
|
||||
- name: prefect-ecs-fargate
|
||||
command: python3 -m tests.e2e.upstream_prefect_ecs_fargate.test_agent_e2e
|
||||
- name: flink-ecs
|
||||
command: python3 -m tests.e2e.upstream_apache_flink_ecs.test_agent_e2e
|
||||
|
||||
name: test-thorough (${{ matrix.name }})
|
||||
|
||||
env:
|
||||
# Thorough e2e suites make live LLM calls; run them on openai because the
|
||||
# default anthropic account is credit-exhausted. Requires a valid, funded
|
||||
# OPENAI_API_KEY secret.
|
||||
LLM_PROVIDER: openai
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
JWT_TOKEN: ${{ secrets.JWT_TOKEN }}
|
||||
TRACER_API_URL: ${{ secrets.TRACER_API_URL }}
|
||||
TRACER_INGEST_TOKEN: ${{ secrets.TRACER_INGEST_TOKEN }}
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
AWS_REGION: us-east-1
|
||||
AWS_DEFAULT_REGION: us-east-1
|
||||
GRAFANA_READ_TOKEN: ${{ secrets.GRAFANA_READ_TOKEN }}
|
||||
GCLOUD_HOSTED_METRICS_ID: ${{ secrets.GCLOUD_HOSTED_METRICS_ID }}
|
||||
GCLOUD_HOSTED_METRICS_URL: ${{ secrets.GCLOUD_HOSTED_METRICS_URL }}
|
||||
GCLOUD_HOSTED_LOGS_ID: ${{ secrets.GCLOUD_HOSTED_LOGS_ID }}
|
||||
GCLOUD_HOSTED_LOGS_URL: ${{ secrets.GCLOUD_HOSTED_LOGS_URL }}
|
||||
GCLOUD_RW_API_KEY: ${{ secrets.GCLOUD_RW_API_KEY }}
|
||||
CLOUDWATCH_VERIFY_LOGS: ${{ matrix.name == 'cloudwatch-demo' && '0' || '1' }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Install tracer CLI
|
||||
run: |
|
||||
curl -sSL https://install.tracer.cloud | sh -s user_36fbN6K6FwEgJFHsQv1pUByo8K6
|
||||
|
||||
- name: Initialize tracer (optional)
|
||||
run: |
|
||||
sudo tracer init --token ${{ secrets.JWT_TOKEN }}
|
||||
continue-on-error: true
|
||||
timeout-minutes: 2
|
||||
|
||||
- name: Run test
|
||||
run: uv run ${{ matrix.command }}
|
||||
timeout-minutes: 30
|
||||
@@ -0,0 +1,118 @@
|
||||
name: Closed-loop learning (weekly miss triage)
|
||||
|
||||
# Weekly reminder workflow for the closed-loop learning process documented
|
||||
# in docs/closed-loop-learning.mdx. Runs at 09:00 UTC every Monday and
|
||||
# also supports manual triggering.
|
||||
#
|
||||
# The workflow does not consume the per-user ``~/.opensre/misses.jsonl``
|
||||
# store directly (that lives on engineer machines, not on the runner).
|
||||
# Its purpose is to:
|
||||
# 1) Open or refresh a tracking issue that nudges the on-call engineer
|
||||
# to run ``opensre misses stats --since 7d`` and then
|
||||
# ``opensre misses export --since 7d --top 10 --out tests/benchmarks/production_misses/``.
|
||||
# 2) Verify the ``misses_command`` CLI surface is still wired and exits
|
||||
# cleanly, so the weekly process is not blocked by a CLI regression.
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "0 9 * * 1"
|
||||
workflow_dispatch: {}
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
|
||||
jobs:
|
||||
smoke:
|
||||
name: Verify opensre misses CLI
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Smoke test misses CLI
|
||||
run: |
|
||||
uv run opensre misses --help
|
||||
uv run opensre misses list --json
|
||||
uv run opensre misses stats --json
|
||||
|
||||
remind:
|
||||
name: Open weekly triage reminder
|
||||
needs: smoke
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Open weekly tracking issue (close last week's first)
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
// One issue per week. Previous week's tracker is closed before
|
||||
// opening this week's so triage history is per-week (GitHub's
|
||||
// natural unit) rather than one ever-growing comment thread.
|
||||
// Uses the existing "pending triage" label so we do not
|
||||
// auto-create a new label on first run.
|
||||
const week = new Date().toISOString().slice(0, 10);
|
||||
const titlePrefix = "Weekly closed-loop learning triage";
|
||||
const title = `${titlePrefix} — week of ${week}`;
|
||||
const body = [
|
||||
"Weekly nudge per docs/closed-loop-learning.mdx.",
|
||||
"",
|
||||
"Steps for this week's on-call:",
|
||||
"- [ ] `opensre misses stats --since 7d`",
|
||||
"- [ ] Review recurring `(alert, taxonomy)` pairs",
|
||||
"- [ ] `opensre misses export --since 7d --top 10 --out tests/benchmarks/production_misses/`",
|
||||
"- [ ] Open a PR labelled `benchmark` with the new scenarios",
|
||||
"- [ ] Trigger the benchmark workflow on the PR branch",
|
||||
].join("\n");
|
||||
|
||||
const { data: openIssues } = await github.rest.issues.listForRepo({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
state: "open",
|
||||
labels: "pending triage",
|
||||
per_page: 100,
|
||||
});
|
||||
|
||||
const stale = openIssues.filter(
|
||||
(i) => i.title.startsWith(titlePrefix) && i.title !== title,
|
||||
);
|
||||
for (const issue of stale) {
|
||||
await github.rest.issues.update({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: issue.number,
|
||||
state: "closed",
|
||||
state_reason: "completed",
|
||||
});
|
||||
}
|
||||
|
||||
const existing = openIssues.find((i) => i.title === title);
|
||||
if (existing) {
|
||||
return;
|
||||
}
|
||||
|
||||
await github.rest.issues.create({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
title,
|
||||
body,
|
||||
labels: ["pending triage"],
|
||||
});
|
||||
@@ -0,0 +1,43 @@
|
||||
name: CodeQL
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
# Skip CodeQL for docs-only PRs (Markdown/MDX and the docs site tree) to
|
||||
# save CI minutes. Not a required check, so skipping cannot block merge.
|
||||
# `push` to main and the weekly schedule keep full coverage regardless.
|
||||
paths-ignore:
|
||||
- '**/*.md'
|
||||
- '**/*.mdx'
|
||||
- 'docs/**'
|
||||
schedule:
|
||||
- cron: "0 6 * * 1"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
security-events: write
|
||||
actions: read
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
if: github.repository == 'Tracer-Cloud/opensre'
|
||||
name: Analyze (python)
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@v4
|
||||
with:
|
||||
languages: python
|
||||
build-mode: none
|
||||
config-file: .github/codeql/codeql-config.yml
|
||||
queries: security-and-quality
|
||||
|
||||
- name: Perform CodeQL Analysis
|
||||
uses: github/codeql-action/analyze@v4
|
||||
with:
|
||||
category: /language:python
|
||||
@@ -0,0 +1,148 @@
|
||||
name: Docker Publish
|
||||
|
||||
on:
|
||||
release:
|
||||
types: [published]
|
||||
schedule:
|
||||
- cron: "30 2 * * *"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: "Release tag to build (e.g. v0.1.2026.7.8). Defaults to the latest release."
|
||||
required: false
|
||||
type: string
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
concurrency:
|
||||
group: docker-publish
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
build-and-push:
|
||||
if: github.repository == 'Tracer-Cloud/opensre'
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Log in to GHCR
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Resolve release tag and image tags
|
||||
id: meta
|
||||
env:
|
||||
EVENT_NAME: ${{ github.event_name }}
|
||||
RELEASE_TAG: ${{ github.event.release.tag_name }}
|
||||
RELEASE_PRERELEASE: ${{ github.event.release.prerelease }}
|
||||
DISPATCH_TAG: ${{ inputs.tag }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
image="ghcr.io/${GITHUB_REPOSITORY,,}"
|
||||
latest_tag="$(gh api "repos/${GITHUB_REPOSITORY}/releases/latest" --jq .tag_name)"
|
||||
|
||||
case "$EVENT_NAME" in
|
||||
release)
|
||||
if [ "$RELEASE_PRERELEASE" = "true" ]; then
|
||||
echo "Skipping prerelease $RELEASE_TAG (rolling main builds are not published to GHCR)."
|
||||
echo "build=false" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
tag="$RELEASE_TAG"
|
||||
;;
|
||||
workflow_dispatch)
|
||||
tag="${DISPATCH_TAG:-$latest_tag}"
|
||||
gh release view "$tag" --repo "$GITHUB_REPOSITORY" >/dev/null
|
||||
;;
|
||||
*)
|
||||
tag="$latest_tag"
|
||||
;;
|
||||
esac
|
||||
|
||||
version="${tag#v}"
|
||||
|
||||
# The scheduled run rebuilds the newest release; skip it when that
|
||||
# version is already in the registry (e.g. the release event or a
|
||||
# dispatch already published it).
|
||||
if [ "$EVENT_NAME" = "schedule" ] \
|
||||
&& docker manifest inspect "${image}:${version}" >/dev/null 2>&1; then
|
||||
echo "Image ${image}:${version} already published; nothing to do."
|
||||
echo "build=false" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
tags="${image}:${version}"
|
||||
if [ "$tag" = "$latest_tag" ]; then
|
||||
tags="${tags}"$'\n'"${image}:latest"
|
||||
else
|
||||
echo "Tag $tag is older than latest release $latest_tag; not moving :latest."
|
||||
fi
|
||||
|
||||
{
|
||||
echo "build=true"
|
||||
echo "tag=${tag}"
|
||||
echo "version=${version}"
|
||||
echo "image=${image}"
|
||||
echo "tags<<EOF"
|
||||
echo "$tags"
|
||||
echo "EOF"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
- uses: actions/checkout@v5
|
||||
if: steps.meta.outputs.build == 'true'
|
||||
with:
|
||||
ref: ${{ steps.meta.outputs.tag }}
|
||||
|
||||
- name: Sync release version into pyproject.toml
|
||||
if: steps.meta.outputs.build == 'true'
|
||||
shell: bash
|
||||
run: python3 platform/packaging/sync_release_version.py --tag "${{ steps.meta.outputs.tag }}"
|
||||
|
||||
- name: Set up QEMU
|
||||
if: steps.meta.outputs.build == 'true'
|
||||
uses: docker/setup-qemu-action@v3
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: steps.meta.outputs.build == 'true'
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Build and push image
|
||||
if: steps.meta.outputs.build == 'true'
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: |
|
||||
org.opencontainers.image.title=opensre
|
||||
org.opencontainers.image.description=OpenSRE unified image (MODE=web FastAPI health app, MODE=gateway Telegram gateway)
|
||||
org.opencontainers.image.source=https://github.com/${{ github.repository }}
|
||||
org.opencontainers.image.version=${{ steps.meta.outputs.version }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Smoke test published image
|
||||
if: steps.meta.outputs.build == 'true'
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
image="${{ steps.meta.outputs.image }}:${{ steps.meta.outputs.version }}"
|
||||
docker pull "$image"
|
||||
version_output="$(docker run --rm --entrypoint opensre "$image" --version)"
|
||||
printf '%s\n' "$version_output"
|
||||
case "$version_output" in
|
||||
*"${{ steps.meta.outputs.version }}"*) ;;
|
||||
*)
|
||||
echo "Image version mismatch: expected ${{ steps.meta.outputs.version }}" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
@@ -0,0 +1,186 @@
|
||||
# OpenClaw end-to-end CI.
|
||||
#
|
||||
# Triggers, in order of intent:
|
||||
# 1. PRs that touch OpenClaw-relevant paths (auto, primary signal)
|
||||
# 2. Pushes to main that touch the same paths (post-merge regression catch)
|
||||
# 3. Daily schedule at 06:00 UTC (catches environment drift —
|
||||
# OpenClaw version bumps, MCP SDK
|
||||
# breaks, npm install regressions)
|
||||
# 4. Adding the `ci:openclaw` label to a PR (manual re-run on matching PRs)
|
||||
# 5. Workflow_dispatch from the Actions UI (manual force-run on any ref,
|
||||
# no PR needed)
|
||||
#
|
||||
# The path filter at the trigger level is the primary gate, so non-OpenClaw PRs
|
||||
# pay no CI cost. The label is only useful as a re-trigger on PRs that already
|
||||
# matched paths; for force-running against an unrelated branch, use the manual
|
||||
# dispatch button in the Actions UI.
|
||||
|
||||
name: CI (OpenClaw E2E)
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
types: [opened, synchronize, reopened, labeled, unlabeled]
|
||||
paths:
|
||||
- "integrations/openclaw.py"
|
||||
- "tools/OpenClawMCPTool/**"
|
||||
- "tests/e2e/openclaw/**"
|
||||
- "tests/utils/alert_factory/**"
|
||||
- ".github/workflows/e2e-openclaw.yml"
|
||||
- "docs/openclaw.mdx"
|
||||
- ".tool-versions"
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "integrations/openclaw.py"
|
||||
- "tools/OpenClawMCPTool/**"
|
||||
- "tests/e2e/openclaw/**"
|
||||
- "tests/utils/alert_factory/**"
|
||||
- ".github/workflows/e2e-openclaw.yml"
|
||||
- "docs/openclaw.mdx"
|
||||
- ".tool-versions"
|
||||
schedule:
|
||||
- cron: "0 6 * * *"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
|
||||
|
||||
concurrency:
|
||||
group: ci-openclaw-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
# Quality gates (lint / format-check / mypy) are run by the main
|
||||
# ``ci.yml`` workflow for every PR. We don't duplicate them here —
|
||||
# this workflow's sole job is the OpenClaw e2e suite. If you reach
|
||||
# this workflow, your PR has already cleared the quality gates.
|
||||
openclaw-test:
|
||||
if: github.repository == 'Tracer-Cloud/opensre'
|
||||
name: openclaw test
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 25
|
||||
|
||||
# No ``env:`` block by design. CI runs the boot + use-case
|
||||
# scenarios only — the full-RCA sub-tests call an LLM, cost API
|
||||
# credits, and take ~25s each, so we deliberately don't wire any
|
||||
# LLM secret in. They auto-skip via
|
||||
# ``LLM_CREDENTIAL_SKIP_REASON`` when no key is present in env.
|
||||
# Contributors with ANTHROPIC_API_KEY / OPENAI_API_KEY /
|
||||
# GEMINI_API_KEY set locally still run them via
|
||||
# ``make test-openclaw``. Lang Smith env vars stay scoped to the
|
||||
# local-RCA path for the same reason — no value tracing the
|
||||
# use-case-only CI runs.
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Set up Node (OpenClaw requires >=22.12; version pinned in .tool-versions)
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: ".tool-versions"
|
||||
|
||||
- name: Cache npm downloads
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.npm
|
||||
key: ${{ runner.os }}-openclaw-npm-${{ hashFiles('.tool-versions') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-openclaw-npm-
|
||||
|
||||
- name: Install OpenClaw CLI
|
||||
# ``continue-on-error: true`` so an npm-registry hiccup, a package-name
|
||||
# rename on OpenClaw's side, or a network blip does not kill the whole
|
||||
# workflow. The "Probe OpenClaw availability" step below catches the
|
||||
# failure and downgrades the run to "tests skipped" with a clear
|
||||
# explanation rather than crashing.
|
||||
continue-on-error: true
|
||||
id: install_openclaw
|
||||
run: npm install -g openclaw --prefer-offline --no-audit
|
||||
|
||||
- name: Probe OpenClaw availability
|
||||
# Whether or not ``npm install`` succeeded, check if ``openclaw`` is
|
||||
# actually runnable. Failure modes we surface explicitly:
|
||||
# - CLI not installed (npm step failed silently)
|
||||
# - CLI installed but Node version mismatch (older runner image)
|
||||
# - CLI installed but the bundled JS errored on first run
|
||||
# ``openclaw_runnable`` env var feeds the next step's gating.
|
||||
run: |
|
||||
set +e
|
||||
if ! command -v openclaw >/dev/null 2>&1; then
|
||||
echo "::warning::openclaw CLI is not on PATH (npm install may have failed); the e2e suite will be skipped this run."
|
||||
echo "openclaw_runnable=false" >> "$GITHUB_ENV"
|
||||
exit 0
|
||||
fi
|
||||
version_output="$(openclaw --version 2>&1)"
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "::warning::openclaw is on PATH but errors when run:"
|
||||
echo "$version_output"
|
||||
echo "::warning::Skipping e2e suite this run."
|
||||
echo "openclaw_runnable=false" >> "$GITHUB_ENV"
|
||||
exit 0
|
||||
fi
|
||||
echo "✓ OpenClaw verified: $version_output"
|
||||
echo "openclaw_runnable=true" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Note LLM credential policy
|
||||
# CI deliberately does not wire any LLM key, so the full-RCA
|
||||
# sub-tests skip every run. This step is informational — it
|
||||
# makes the policy obvious in the run log instead of leaving a
|
||||
# silent skip that a reviewer has to dig for. Run RCA locally
|
||||
# with ``make test-openclaw`` if you have a key configured.
|
||||
run: |
|
||||
echo "ℹ︎ CI policy: full-RCA sub-tests are skipped (LLM credentials intentionally not wired)."
|
||||
echo " Boot + use-case sub-tests still run and gate the merge."
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Run OpenClaw E2E suite
|
||||
# Skips with a clear message if the OpenClaw probe failed earlier,
|
||||
# so the workflow finishes green with a documented "couldn't run"
|
||||
# status rather than a confusing red on the install step.
|
||||
run: |
|
||||
if [ "$openclaw_runnable" != "true" ]; then
|
||||
echo "::warning::Skipping pytest invocation — openclaw CLI unavailable in this environment. See earlier warnings."
|
||||
exit 0
|
||||
fi
|
||||
uv run python -m pytest -m e2e -v tests/e2e/openclaw/
|
||||
|
||||
- name: Upload gateway logs on failure
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: openclaw-gateway-logs
|
||||
path: /tmp/openclaw-e2e-logs/
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
- name: Summary
|
||||
# Always runs, regardless of earlier step outcomes. The probe
|
||||
# step above downgrades a missing-CLI / wrong-Node environment
|
||||
# to "tests skipped" rather than failing the workflow, so this
|
||||
# summary is the one place a reviewer sees why fewer tests ran
|
||||
# than expected. Real pytest failures still fail the job.
|
||||
if: always()
|
||||
run: |
|
||||
echo "### OpenClaw E2E run summary"
|
||||
echo " OpenClaw runnable: ${openclaw_runnable:-unknown}"
|
||||
echo " Full-RCA sub-tests: skipped by CI policy (no LLM key wired)"
|
||||
echo " Run RCA locally with: make test-openclaw"
|
||||
@@ -0,0 +1,52 @@
|
||||
# When someone with zero merged PRs in this repo (and not OWNER/MEMBER/COLLABORATOR) comments
|
||||
# on an open issue labeled `good first issue`, assign them and reply — only if the issue has no
|
||||
# assignees yet (first eligible commenter wins; includes when a human pre-assigned someone).
|
||||
# issue_comment also fires for PR review threads; those payloads include issue.pull_request.
|
||||
name: Good first issue — auto-assign
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
|
||||
concurrency:
|
||||
group: good-first-issue-assign-${{ github.repository }}-${{ github.event.issue.number }}
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
pull-requests: read
|
||||
|
||||
jobs:
|
||||
assign:
|
||||
name: Assign new contributor on good first issue
|
||||
runs-on: ubuntu-latest
|
||||
if: >-
|
||||
github.event.issue.state == 'open' &&
|
||||
github.event.comment.user.type != 'Bot' &&
|
||||
!github.event.issue.pull_request
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Decide assign + notice body
|
||||
id: decide
|
||||
env:
|
||||
GITHUB_EVENT_PATH: ${{ github.event_path }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: python3 .github/scripts/good_first_issue_assign.py
|
||||
|
||||
- name: Add assignee and notify
|
||||
if: steps.decide.outputs.should_assign == 'true'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
ISSUE_NUMBER: ${{ github.event.issue.number }}
|
||||
REPO_FULL: ${{ github.repository }}
|
||||
COMMENTER_LOGIN: ${{ github.event.comment.user.login }}
|
||||
run: |
|
||||
gh issue edit "${ISSUE_NUMBER}" \
|
||||
--repo "${REPO_FULL}" \
|
||||
--add-assignee "${COMMENTER_LOGIN}"
|
||||
gh issue comment "${ISSUE_NUMBER}" \
|
||||
--repo "${REPO_FULL}" \
|
||||
--body-file assign_comment.md
|
||||
@@ -0,0 +1,46 @@
|
||||
name: Greptile PR reminder
|
||||
|
||||
# One-time nudge when a PR is opened or reopened. Uses pull_request_target so the token can
|
||||
# comment on PRs from forks. Does not checkout the PR head — only posts a static reminder (see
|
||||
# SECURITY: pull_request_target guidance in GitHub Actions docs).
|
||||
on:
|
||||
pull_request_target:
|
||||
types: [opened, reopened]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
remind:
|
||||
name: Post Greptile review reminder
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event.pull_request.user.type != 'Bot'
|
||||
steps:
|
||||
- name: Comment with Greptile guidance
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
BASE_REF: ${{ github.event.pull_request.base.ref }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
base_ref="${BASE_REF}"
|
||||
contributing_url="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/blob/${base_ref}/CONTRIBUTING.md#greptile-code-review"
|
||||
pr="${PR_NUMBER}"
|
||||
repo="${GITHUB_REPOSITORY}"
|
||||
{
|
||||
echo '### Greptile code review'
|
||||
echo ''
|
||||
echo "This repo uses **Greptile** for automated review. Before merge, aim for **Confidence Score: 5/5** with **zero unresolved** review threads — see [CONTRIBUTING.md](${contributing_url})."
|
||||
echo ''
|
||||
echo '**Run a review** — add a PR comment with:'
|
||||
echo ''
|
||||
echo '```'
|
||||
echo '@greptile review'
|
||||
echo '```'
|
||||
echo ''
|
||||
echo 'Give it **~5-10 minutes** (sometimes longer) for results, then fix feedback and re-trigger until you reach **Confidence Score: 5/5**.'
|
||||
echo ''
|
||||
echo '**Optional:** automate with the [greploop skill](https://skills.sh/greptileai/skills/greploop).'
|
||||
} > comment.md
|
||||
gh pr comment "$pr" --repo "$repo" --body-file comment.md
|
||||
@@ -0,0 +1,50 @@
|
||||
name: Hermes Tests
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/synthetic/hermes_rca/**"
|
||||
- "tests/e2e/hermes/**"
|
||||
- "tests/synthetic/mock_hermes_backend/**"
|
||||
- "tools/HermesSessionEvidenceTool/**"
|
||||
- ".github/workflows/hermes-tests.yml"
|
||||
- "Makefile"
|
||||
- "pytest.ini"
|
||||
schedule:
|
||||
- cron: "17 3 * * *"
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
synthetic:
|
||||
name: Hermes synthetic offline
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: astral-sh/setup-uv@v5
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Run Hermes synthetic tests
|
||||
run: uv run pytest tests/synthetic/hermes_rca -q
|
||||
|
||||
- name: Run Hermes offline suite
|
||||
run: uv run python -m tests.synthetic.hermes_rca.run_suite --offline-only
|
||||
|
||||
e2e:
|
||||
name: Hermes e2e/meta
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: astral-sh/setup-uv@v5
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Run Hermes e2e/meta tests
|
||||
run: uv run pytest tests/e2e/hermes -q
|
||||
@@ -0,0 +1,160 @@
|
||||
name: Interactive Shell Live (PR + post-merge)
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "core/agent/**"
|
||||
- "core/agent_harness/**"
|
||||
- "tests/core/agent/**"
|
||||
- "surfaces/interactive_shell/**"
|
||||
- "core/runtime/**"
|
||||
- "tools/**"
|
||||
- "integrations/**"
|
||||
- "pytest.ini"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- ".github/workflows/interactive-shell-live.yml"
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "core/agent/**"
|
||||
- "core/agent_harness/**"
|
||||
- "tests/core/agent/**"
|
||||
- "surfaces/interactive_shell/**"
|
||||
- "core/runtime/**"
|
||||
- "tools/**"
|
||||
- "integrations/**"
|
||||
- "pytest.ini"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- ".github/workflows/interactive-shell-live.yml"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
shard_indexes:
|
||||
description: "Comma-separated shard indexes to run (0-7)"
|
||||
required: false
|
||||
default: "0,1,2,3,4,5,6,7"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: interactive-shell-live-${{ github.event.pull_request.number || github.sha }}
|
||||
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||
|
||||
jobs:
|
||||
turn-checks:
|
||||
name: turn-checks (no-LLM)
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
|
||||
env:
|
||||
OPENSRE_DISABLE_KEYRING: "1"
|
||||
PYTHONUTF8: "1"
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Deterministic turn + fixture integrity
|
||||
run: >-
|
||||
uv run python -m pytest -n auto -v
|
||||
tests/core/agent/
|
||||
-m "not live_llm"
|
||||
|
||||
turn-live:
|
||||
# Run on push to main, manual dispatch, and same-repo PRs. Fork PRs have no
|
||||
# secrets, so they skip the live job (the no-LLM turn-checks job above
|
||||
# still gates them) instead of failing the credential-validation step.
|
||||
if: >-
|
||||
github.repository == 'Tracer-Cloud/opensre' &&
|
||||
(github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false)
|
||||
name: turn-live shard ${{ matrix.shard_index }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
shard_index: ${{ fromJSON(github.event_name == 'workflow_dispatch' && format('[{0}]', github.event.inputs.shard_indexes) || '[0, 1, 2, 3, 4, 5, 6, 7]') }}
|
||||
|
||||
env:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
LLM_PROVIDER: openai
|
||||
OPENSRE_DISABLE_KEYRING: "1"
|
||||
TURN_SHARD_TOTAL: "8"
|
||||
TURN_SHARD_INDEX: ${{ matrix.shard_index }}
|
||||
PYTHONUTF8: "1"
|
||||
# @live gather scenarios (316, 333–335, 337) require these secrets in CI.
|
||||
# Missing values fail the job in preflight and via skip_or_fail in tests.
|
||||
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
|
||||
SENTRY_ORG_SLUG: tracer-30
|
||||
SENTRY_PROJECT_SLUG: python
|
||||
SENTRY_URL: https://sentry.io
|
||||
DD_API_KEY: ${{ secrets.DD_API_KEY }}
|
||||
DD_APP_KEY: ${{ secrets.DD_APP_KEY }}
|
||||
DD_SITE: datadoghq.com
|
||||
GRAFANA_READ_TOKEN: ${{ secrets.GRAFANA_READ_TOKEN }}
|
||||
GRAFANA_INSTANCE_URL: ${{ secrets.GRAFANA_INSTANCE_URL }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Validate live turn credentials
|
||||
run: |
|
||||
missing=()
|
||||
if [ -z "${OPENAI_API_KEY}" ]; then
|
||||
missing+=(OPENAI_API_KEY)
|
||||
fi
|
||||
if [ -z "${SENTRY_AUTH_TOKEN}" ]; then
|
||||
missing+=(SENTRY_AUTH_TOKEN)
|
||||
fi
|
||||
if [ -z "${DD_API_KEY}" ]; then
|
||||
missing+=(DD_API_KEY)
|
||||
fi
|
||||
if [ -z "${DD_APP_KEY}" ]; then
|
||||
missing+=(DD_APP_KEY)
|
||||
fi
|
||||
if [ ${#missing[@]} -gt 0 ]; then
|
||||
echo "::error::Missing secrets for live turn tests: ${missing[*]}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Run sharded live turn suite
|
||||
run: >-
|
||||
uv run python -m pytest -n auto -v
|
||||
-m live_llm
|
||||
tests/core/agent/test_turn_scenarios.py
|
||||
-k "test_live_turn_execution_oracle or test_live_action_planning"
|
||||
@@ -0,0 +1,678 @@
|
||||
name: Release
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths-ignore:
|
||||
- "docs/**"
|
||||
- "tests/benchmarks/cloudopsbench/infra/**"
|
||||
- "platform/deployment/install-proxy/**"
|
||||
- "tests/e2e/kubernetes/helm/scripts/**"
|
||||
- "tests/**"
|
||||
- "**/*.md"
|
||||
- "**/*.mdx"
|
||||
- ".claude/**"
|
||||
schedule:
|
||||
- cron: "30 0 * * *"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
channel:
|
||||
description: "Release channel to publish."
|
||||
required: false
|
||||
default: release
|
||||
type: choice
|
||||
options:
|
||||
- release
|
||||
- main
|
||||
tag:
|
||||
description: "Optional tag to release (e.g. v0.1.2026.6.26 or v0.1). Ignored for the main channel."
|
||||
required: false
|
||||
type: string
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
actions: read
|
||||
models: read
|
||||
|
||||
concurrency:
|
||||
group: release-${{ github.event_name == 'push' && 'main' || (github.event_name == 'workflow_dispatch' && inputs.channel == 'main' && 'main') || 'stable' }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
prepare:
|
||||
if: github.repository == 'Tracer-Cloud/opensre'
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
channel: ${{ steps.meta.outputs.channel }}
|
||||
tag_name: ${{ steps.meta.outputs.tag_name }}
|
||||
version_name: ${{ steps.meta.outputs.version_name }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Resolve release metadata
|
||||
id: meta
|
||||
env:
|
||||
EVENT_NAME: ${{ github.event_name }}
|
||||
DISPATCH_CHANNEL: ${{ inputs.channel }}
|
||||
DISPATCH_TAG: ${{ inputs.tag }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
channel="release"
|
||||
|
||||
if [ "$EVENT_NAME" = "push" ]; then
|
||||
channel="main"
|
||||
fi
|
||||
|
||||
if [ "$EVENT_NAME" = "workflow_dispatch" ] && [ "$DISPATCH_CHANNEL" = "main" ]; then
|
||||
channel="main"
|
||||
fi
|
||||
|
||||
if [ "$channel" = "main" ]; then
|
||||
if [ "$EVENT_NAME" = "workflow_dispatch" ] && [ -n "$DISPATCH_TAG" ]; then
|
||||
echo "The main channel does not accept a custom tag." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
year="$(date -u +%Y)"
|
||||
month="$(date -u +%-m)"
|
||||
day="$(date -u +%-d)"
|
||||
short_sha="${GITHUB_SHA:0:7}"
|
||||
main_version="0.1.${year}.${month}.${day}+main.${short_sha}"
|
||||
|
||||
echo "channel=main" >> "$GITHUB_OUTPUT"
|
||||
echo "tag_name=main-build" >> "$GITHUB_OUTPUT"
|
||||
echo "version_name=${main_version}" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$EVENT_NAME" = "workflow_dispatch" ] && [ -n "$DISPATCH_TAG" ]; then
|
||||
tag_name="$DISPATCH_TAG"
|
||||
else
|
||||
year="$(date -u +%Y)"
|
||||
month="$(date -u +%-m)"
|
||||
day="$(date -u +%-d)"
|
||||
# Keep the v0.1 prefix so the tag makes clear the product is still
|
||||
# v0.1, while the date stays useful (e.g. v0.1.2026.6.26).
|
||||
tag_name="v0.1.${year}.${month}.${day}"
|
||||
fi
|
||||
|
||||
echo "channel=release" >> "$GITHUB_OUTPUT"
|
||||
echo "tag_name=${tag_name}" >> "$GITHUB_OUTPUT"
|
||||
echo "version_name=${tag_name#v}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
verify:
|
||||
runs-on: ubuntu-latest
|
||||
needs: prepare
|
||||
if: github.event_name != 'push'
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Lint
|
||||
run: make lint
|
||||
|
||||
- name: Type check
|
||||
run: make typecheck
|
||||
|
||||
- name: CLI smoke tests
|
||||
run: make test-cli-smoke
|
||||
|
||||
build-python-dist:
|
||||
if: needs.prepare.outputs.channel == 'release' && needs.verify.result == 'success'
|
||||
runs-on: ubuntu-latest
|
||||
needs: [verify, prepare]
|
||||
env:
|
||||
TAG_NAME: ${{ needs.prepare.outputs.tag_name }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Sync release version
|
||||
shell: bash
|
||||
run: python platform/packaging/sync_release_version.py --tag "$TAG_NAME"
|
||||
|
||||
- name: Build Python distributions
|
||||
run: |
|
||||
uv sync --frozen --extra release-dist
|
||||
uv run python -m build
|
||||
uv run twine check dist/*
|
||||
|
||||
- name: Verify Python distribution version
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
VERSION="${TAG_NAME#v}"
|
||||
test -f "dist/opensre-${VERSION}.tar.gz"
|
||||
test -f "dist/opensre-${VERSION}-py3-none-any.whl"
|
||||
|
||||
- name: Upload Python distributions
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: release-python-dist
|
||||
path: dist/*
|
||||
if-no-files-found: error
|
||||
|
||||
build-binaries:
|
||||
if: always() && (needs.prepare.outputs.channel == 'main' || needs.verify.result == 'success')
|
||||
needs: [verify, prepare]
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- runner: ubuntu-22.04
|
||||
target: linux-x64
|
||||
binary_name: opensre
|
||||
archive_ext: tar.gz
|
||||
pyinstaller_mode: onedir
|
||||
- runner: ubuntu-22.04-arm
|
||||
target: linux-arm64
|
||||
binary_name: opensre
|
||||
archive_ext: tar.gz
|
||||
pyinstaller_mode: onedir
|
||||
- runner: macos-15-intel
|
||||
target: darwin-x64
|
||||
binary_name: opensre
|
||||
archive_ext: tar.gz
|
||||
pyinstaller_mode: onedir
|
||||
- runner: macos-latest
|
||||
target: darwin-arm64
|
||||
binary_name: opensre
|
||||
archive_ext: tar.gz
|
||||
pyinstaller_mode: onedir
|
||||
- runner: windows-latest
|
||||
target: windows-x64
|
||||
binary_name: opensre.exe
|
||||
archive_ext: zip
|
||||
pyinstaller_mode: onefile
|
||||
# windows-arm64 is currently excluded from the default release matrix:
|
||||
# cryptography does not publish win_arm64 wheels, so dependency install
|
||||
# falls back to a source build that requires an OpenSSL toolchain on the
|
||||
# GitHub-hosted runner.
|
||||
runs-on: ${{ matrix.runner }}
|
||||
env:
|
||||
RELEASE_CHANNEL: ${{ needs.prepare.outputs.channel }}
|
||||
TAG_NAME: ${{ needs.prepare.outputs.tag_name }}
|
||||
VERSION_NAME: ${{ needs.prepare.outputs.version_name }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Sync binary version
|
||||
shell: bash
|
||||
run: python platform/packaging/sync_release_version.py --version "$VERSION_NAME"
|
||||
|
||||
- name: Install binary build dependencies
|
||||
shell: bash
|
||||
run: uv sync --frozen --extra release-binary
|
||||
|
||||
- name: Stage stdlib platform module for bundling
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# The first-party ``platform`` package shadows the stdlib ``platform``
|
||||
# module. PyInstaller does not lay out the stdlib as loose ``.py`` files,
|
||||
# so bundle a copy of the genuine module that platform/__init__.py can
|
||||
# load from ``sys._MEIPASS`` at runtime (see _FROZEN_STDLIB_DIR).
|
||||
rm -rf .stdlib_vendor
|
||||
mkdir -p .stdlib_vendor
|
||||
uv run python -c "import os, shutil, sysconfig; src = os.path.join(sysconfig.get_path('stdlib'), 'platform.py'); shutil.copy(src, os.path.join('.stdlib_vendor', 'platform.py')); print('Staged stdlib platform from ' + src)"
|
||||
|
||||
- name: Build binary
|
||||
run: >-
|
||||
uv run pyinstaller surfaces/cli/__main__.py
|
||||
--name opensre
|
||||
--${{ matrix.pyinstaller_mode }}
|
||||
--clean
|
||||
--noconfirm
|
||||
--collect-data surfaces.cli
|
||||
--collect-data config
|
||||
--copy-metadata opensre
|
||||
--collect-data litellm
|
||||
--hidden-import tiktoken_ext
|
||||
--hidden-import tiktoken_ext.openai_public
|
||||
--collect-submodules integrations
|
||||
--collect-submodules surfaces.interactive_shell
|
||||
--collect-submodules tools
|
||||
--add-data "platform:platform"
|
||||
--add-data ".stdlib_vendor:_opensre_stdlib_platform"
|
||||
|
||||
- name: Smoke test binary (Unix)
|
||||
if: runner.os != 'Windows'
|
||||
shell: bash
|
||||
run: |
|
||||
set -uo pipefail
|
||||
# Capture the exit status explicitly so a crashing binary still prints
|
||||
# its stdout/stderr (under `set -e` the failed substitution aborted the
|
||||
# step before the output was ever shown, hiding the real error).
|
||||
# onefile builds place the executable at ./dist/<name>; onedir builds
|
||||
# produce a ./dist/opensre/ directory containing the executable. A bare
|
||||
# `-x` test matches the onedir directory too (dirs carry the search
|
||||
# bit), so require a regular file before treating it as the binary.
|
||||
if [ -f "./dist/${{ matrix.binary_name }}" ] && [ -x "./dist/${{ matrix.binary_name }}" ]; then
|
||||
BINARY_PATH="./dist/${{ matrix.binary_name }}"
|
||||
else
|
||||
BINARY_PATH="./dist/opensre/${{ matrix.binary_name }}"
|
||||
fi
|
||||
set +e
|
||||
VERSION_OUTPUT="$("$BINARY_PATH" --version 2>&1)"
|
||||
VERSION_STATUS=$?
|
||||
set -e
|
||||
printf '%s\n' "$VERSION_OUTPUT"
|
||||
if [ "$VERSION_STATUS" -ne 0 ]; then
|
||||
printf '::error::%s --version exited with status %s\n' "${{ matrix.binary_name }}" "$VERSION_STATUS" >&2
|
||||
exit "$VERSION_STATUS"
|
||||
fi
|
||||
case "$VERSION_OUTPUT" in
|
||||
*"$VERSION_NAME"*) ;;
|
||||
*)
|
||||
printf 'Binary version mismatch: expected %s but saw %s\n' "$VERSION_NAME" "$VERSION_OUTPUT" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
"$BINARY_PATH" -h >/dev/null
|
||||
LITELLM_DATA="./dist/opensre/_internal/litellm/model_prices_and_context_window_backup.json"
|
||||
if [ ! -f "$LITELLM_DATA" ]; then
|
||||
echo "LiteLLM package data missing from Unix onedir bundle: ${LITELLM_DATA}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Check Linux binary glibc compatibility
|
||||
if: runner.os == 'Linux'
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
max_glibc="$(
|
||||
find dist/opensre -type f \( -name opensre -o -name '*.so' -o -name '*.so.*' \) -print0 \
|
||||
| xargs -0 strings 2>/dev/null \
|
||||
| grep -Eo 'GLIBC_[0-9]+\.[0-9]+(\.[0-9]+)?' \
|
||||
| sort -Vu \
|
||||
| tail -n 1 \
|
||||
|| true
|
||||
)"
|
||||
echo "Max required glibc symbol: ${max_glibc:-none}"
|
||||
if [ -n "$max_glibc" ] && [ "$(printf '%s\n' "$max_glibc" GLIBC_2.35 | sort -V | tail -n 1)" != "GLIBC_2.35" ]; then
|
||||
echo "Linux binary requires ${max_glibc}; pin the Linux runner back to Ubuntu 22.04 or lower dependency wheel requirements." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Smoke test binary (Windows)
|
||||
if: runner.os == 'Windows'
|
||||
shell: pwsh
|
||||
run: |
|
||||
$versionOutput = & ".\dist\${{ matrix.binary_name }}" --version 2>&1 | Out-String
|
||||
$versionText = $versionOutput.Trim()
|
||||
Write-Host $versionText
|
||||
$expectedVersion = $env:VERSION_NAME
|
||||
if ($versionText -notmatch [regex]::Escape($expectedVersion)) {
|
||||
throw "Binary version mismatch. Expected '$expectedVersion' but saw '$versionText'."
|
||||
}
|
||||
& ".\dist\${{ matrix.binary_name }}" -h | Out-Null
|
||||
|
||||
- name: Package binary archive (Unix)
|
||||
if: runner.os != 'Windows'
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ "$RELEASE_CHANNEL" = "release" ]; then
|
||||
ASSET_BASENAME="opensre_${TAG_NAME#v}_${{ matrix.target }}"
|
||||
else
|
||||
ASSET_BASENAME="opensre_main_${{ matrix.target }}"
|
||||
fi
|
||||
tar -C dist -czf "${ASSET_BASENAME}.tar.gz" "${{ matrix.binary_name }}"
|
||||
shasum -a 256 "${ASSET_BASENAME}.tar.gz" > "${ASSET_BASENAME}.tar.gz.sha256"
|
||||
|
||||
- name: Package binary archive (Windows)
|
||||
if: runner.os == 'Windows'
|
||||
shell: pwsh
|
||||
run: |
|
||||
if ($env:RELEASE_CHANNEL -eq "release") {
|
||||
$assetBaseName = "opensre_$($env:TAG_NAME.TrimStart('v'))_${{ matrix.target }}"
|
||||
} else {
|
||||
$assetBaseName = "opensre_main_${{ matrix.target }}"
|
||||
}
|
||||
Compress-Archive -Path "dist\${{ matrix.binary_name }}" -DestinationPath "${assetBaseName}.zip"
|
||||
$hash = (Get-FileHash -Algorithm SHA256 "${assetBaseName}.zip").Hash.ToLowerInvariant()
|
||||
Set-Content -Path "${assetBaseName}.zip.sha256" -Value "$hash ${assetBaseName}.zip"
|
||||
|
||||
- name: Upload binary archive
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: ${{ env.RELEASE_CHANNEL == 'main' && 'main-' || '' }}release-${{ matrix.target }}
|
||||
path: |
|
||||
opensre_*_${{ matrix.target }}.${{ matrix.archive_ext }}
|
||||
opensre_*_${{ matrix.target }}.${{ matrix.archive_ext }}.sha256
|
||||
if-no-files-found: error
|
||||
|
||||
publish-release:
|
||||
if: needs.prepare.outputs.channel == 'release'
|
||||
runs-on: ubuntu-latest
|
||||
needs:
|
||||
- prepare
|
||||
- build-python-dist
|
||||
- build-binaries
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
with:
|
||||
ref: ${{ github.event.repository.default_branch }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Download release artifacts
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
pattern: release-*
|
||||
path: release-assets
|
||||
merge-multiple: true
|
||||
|
||||
- name: Resolve release context
|
||||
id: release_ctx
|
||||
env:
|
||||
TAG_NAME: ${{ needs.prepare.outputs.tag_name }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
default_branch="${{ github.event.repository.default_branch }}"
|
||||
git fetch origin "$default_branch" --tags --force
|
||||
target_sha="$(git rev-parse "origin/$default_branch")"
|
||||
|
||||
previous_tag="$(
|
||||
git tag --list 'v0.1.[0-9][0-9][0-9][0-9].[0-9]*.[0-9]*' --sort=-v:refname \
|
||||
| grep -v -x "$TAG_NAME" \
|
||||
| head -n 1 \
|
||||
|| true
|
||||
)"
|
||||
|
||||
range_spec="$target_sha"
|
||||
if [ -n "$previous_tag" ]; then
|
||||
range_spec="${previous_tag}..${target_sha}"
|
||||
fi
|
||||
|
||||
{
|
||||
printf 'target_sha=%s\n' "$target_sha"
|
||||
printf 'previous_tag=%s\n' "$previous_tag"
|
||||
printf 'range_spec=%s\n' "$range_spec"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Create release notes
|
||||
env:
|
||||
TAG_NAME: ${{ needs.prepare.outputs.tag_name }}
|
||||
RANGE_SPEC: ${{ steps.release_ctx.outputs.range_spec }}
|
||||
PREVIOUS_TAG: ${{ steps.release_ctx.outputs.previous_tag }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
{
|
||||
echo "## Changelog"
|
||||
echo
|
||||
if [ -n "$PREVIOUS_TAG" ]; then
|
||||
echo "_Changes since ${PREVIOUS_TAG}_"
|
||||
else
|
||||
echo "_Changes up to ${TAG_NAME}_"
|
||||
fi
|
||||
echo
|
||||
git log "$RANGE_SPEC" --no-merges -n 100 --pretty='- %s (%h) — %an'
|
||||
echo
|
||||
} > GENERATED_CHANGELOG.md
|
||||
|
||||
# Summarize commits into prose via GitHub Models (gpt-4o-mini).
|
||||
# Falls back to raw changelog if the API is unavailable.
|
||||
raw_commits="$(git log "$RANGE_SPEC" --no-merges --pretty='%s' | head -40 || true)"
|
||||
if [ -n "$raw_commits" ]; then
|
||||
api_body="$(jq -n --arg commits "$raw_commits" '{
|
||||
model: "gpt-4o-mini",
|
||||
messages: [
|
||||
{
|
||||
role: "system",
|
||||
content: "You are writing a Discord release announcement for an open-source SRE CLI tool called opensre. Summarize the git commits into 2-4 short, punchy prose sentences — no bullet points, no headers, no markdown. Write in present tense, active voice. Focus on what users will notice: new features, fixes, performance, integrations. Be specific but concise."
|
||||
},
|
||||
{role: "user", content: ("Commits:\n" + $commits)}
|
||||
],
|
||||
max_tokens: 300,
|
||||
temperature: 0.4
|
||||
}')"
|
||||
curl -sS --max-time 20 \
|
||||
-H "Authorization: Bearer $GITHUB_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
"https://models.inference.ai.azure.com/chat/completions" \
|
||||
-d "$api_body" 2>/dev/null \
|
||||
| jq -r '.choices[0].message.content // empty' 2>/dev/null \
|
||||
| tr -d '\r' > DISCORD_NARRATIVE.md || true
|
||||
|
||||
if [ -s DISCORD_NARRATIVE.md ]; then
|
||||
echo "LLM narrative generated ($(wc -c < DISCORD_NARRATIVE.md) bytes)."
|
||||
else
|
||||
echo "LLM summary unavailable; Discord will use raw changelog."
|
||||
rm -f DISCORD_NARRATIVE.md
|
||||
fi
|
||||
fi
|
||||
|
||||
CB='```'
|
||||
{
|
||||
printf '## Install\n\n'
|
||||
printf '### cURL (macOS / Linux)\n\n%sbash\ncurl -fsSL https://install.opensre.com | bash -s -- --release --version %s\n%s\n\n' "$CB" "${TAG_NAME#v}" "$CB"
|
||||
printf '### Homebrew (macOS / Linux)\n\n%sbash\nbrew tap tracer-cloud/tap\nbrew install tracer-cloud/tap/opensre\n%s\n\n' "$CB" "$CB"
|
||||
printf '### PowerShell (Windows)\n\n%spowershell\n$env:OPENSRE_INSTALL_CHANNEL="release"; $env:OPENSRE_VERSION="%s"; irm https://install.opensre.com | iex\n%s\n\n' "$CB" "${TAG_NAME#v}" "$CB"
|
||||
printf '### Python\n\n%sbash\npipx install opensre\n%s\n\n' "$CB" "$CB"
|
||||
} > RELEASE_NOTES.md
|
||||
cat GENERATED_CHANGELOG.md >> RELEASE_NOTES.md
|
||||
|
||||
- name: Create GitHub release
|
||||
id: github_release
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
TAG_NAME: ${{ needs.prepare.outputs.tag_name }}
|
||||
TARGET_SHA: ${{ steps.release_ctx.outputs.target_sha }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
if gh release view "$TAG_NAME" --repo "${{ github.repository }}" >/dev/null 2>&1; then
|
||||
echo "created=false" >> "$GITHUB_OUTPUT"
|
||||
echo "Release $TAG_NAME already exists; nothing to do."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
VERSION="${TAG_NAME#v}"
|
||||
|
||||
# Every release-channel publish is the newest stable build at this
|
||||
# point, so mark it as GitHub's Latest. The title keeps the full
|
||||
# version (e.g. "OpenSRE 0.1.2026.6.26") so the v0.1 line is clear.
|
||||
gh release create "$TAG_NAME" \
|
||||
release-assets/* \
|
||||
--repo "${{ github.repository }}" \
|
||||
--target "$TARGET_SHA" \
|
||||
--title "OpenSRE ${VERSION}" \
|
||||
--notes-file RELEASE_NOTES.md \
|
||||
--latest
|
||||
|
||||
echo "created=true" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Announce on Discord
|
||||
if: steps.github_release.outputs.created == 'true' && !github.event.repository.fork && !github.event.repository.private
|
||||
continue-on-error: true
|
||||
env:
|
||||
DISCORD_WEBHOOK_URL_UPDATE: ${{ secrets.DISCORD_WEBHOOK_URL_UPDATE }}
|
||||
DISCORD_RELEASES_ROLE_ID: ${{ secrets.DISCORD_RELEASES_ROLE_ID }}
|
||||
DISCORD_RELEASE_LOGO_EMOJI: ${{ secrets.DISCORD_RELEASE_LOGO_EMOJI }}
|
||||
DISCORD_RELEASE_LOGO_URL: ${{ secrets.DISCORD_RELEASE_LOGO_URL }}
|
||||
RELEASE_TAG: ${{ needs.prepare.outputs.tag_name }}
|
||||
RELEASE_URL: ${{ github.server_url }}/${{ github.repository }}/releases/tag/${{ needs.prepare.outputs.tag_name }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
if [ -z "${DISCORD_WEBHOOK_URL_UPDATE:-}" ]; then
|
||||
echo "DISCORD_WEBHOOK_URL_UPDATE is not set; skipping Discord announcement."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Prefer LLM-generated narrative; fall back to raw changelog.
|
||||
if [ -f DISCORD_NARRATIVE.md ] && [ -s DISCORD_NARRATIVE.md ]; then
|
||||
changelog_file="DISCORD_NARRATIVE.md"
|
||||
elif [ -f GENERATED_CHANGELOG.md ]; then
|
||||
changelog_file="GENERATED_CHANGELOG.md"
|
||||
else
|
||||
echo "No changelog available." > /tmp/discord_fallback.md
|
||||
changelog_file="/tmp/discord_fallback.md"
|
||||
fi
|
||||
|
||||
# Use an external Python script to build the JSON payload.
|
||||
# This avoids jq issues with multiline LLM output containing control characters.
|
||||
payload="$(CHANGELOG_FILE="$changelog_file" python3 .github/scripts/build-discord-payload.py)"
|
||||
|
||||
curl --fail -sS --max-time 30 -X POST -H "Content-Type: application/json" \
|
||||
-d "$payload" "$DISCORD_WEBHOOK_URL_UPDATE"
|
||||
|
||||
- name: Sync Homebrew tap formula
|
||||
if: steps.github_release.outputs.created == 'true'
|
||||
continue-on-error: true
|
||||
env:
|
||||
HOMEBREW_TAP_GITHUB_TOKEN: ${{ secrets.HOMEBREW_TAP_GITHUB_TOKEN }}
|
||||
VERSION: ${{ needs.prepare.outputs.tag_name }}
|
||||
ASSET_DIR: release-assets
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${HOMEBREW_TAP_GITHUB_TOKEN:-}" ]; then
|
||||
echo "HOMEBREW_TAP_GITHUB_TOKEN is not set; skipping Homebrew tap sync."
|
||||
exit 0
|
||||
fi
|
||||
export VERSION="${VERSION#v}"
|
||||
bash .github/scripts/sync-homebrew-tap-formula.sh
|
||||
|
||||
publish-main-release:
|
||||
if: always() && needs.prepare.outputs.channel == 'main' && needs.build-binaries.result == 'success'
|
||||
runs-on: ubuntu-latest
|
||||
needs:
|
||||
- prepare
|
||||
- build-binaries
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
with:
|
||||
fetch-depth: 0
|
||||
# The rolling `main-build` tag is force-moved to each new main commit
|
||||
# below. Whenever main has advanced past a commit that touches
|
||||
# .github/workflows/**, the default GITHUB_TOKEN (the
|
||||
# github-actions[bot] GitHub App) is refused with "refusing to allow a
|
||||
# GitHub App to create or update workflow ... without `workflows`
|
||||
# permission" — and that scope cannot be granted via the permissions
|
||||
# block. MAIN_BUILD_TAG_TOKEN must be a token that carries workflow
|
||||
# write access (classic PAT with `repo` + `workflow`, a fine-grained
|
||||
# PAT with Contents + Workflows: write, or a GitHub App token). It is
|
||||
# persisted so the tag push below authenticates with it. Falls back to
|
||||
# GITHUB_TOKEN so the rest of the job still runs if the secret is unset
|
||||
# (the tag push will then fail as before until the secret is added).
|
||||
token: ${{ secrets.MAIN_BUILD_TAG_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
persist-credentials: true
|
||||
|
||||
- name: Download main release artifacts
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
pattern: main-release-*
|
||||
path: main-release-assets
|
||||
merge-multiple: true
|
||||
|
||||
- name: Create release notes
|
||||
env:
|
||||
VERSION_NAME: ${{ needs.prepare.outputs.version_name }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
short_sha="$(printf '%s' "$GITHUB_SHA" | cut -c1-7)"
|
||||
built_at="$(date -u +"%Y-%m-%d %H:%M UTC")"
|
||||
|
||||
CB='```'
|
||||
{
|
||||
printf '## Main build\n\nRolling binary build from `main`.\n\n'
|
||||
printf -- '- Version: `%s`\n- Commit: `%s`\n- Built: %s\n\n' "$VERSION_NAME" "$short_sha" "$built_at"
|
||||
printf '### Install\n\nmacOS / Linux:\n\n%sbash\ncurl -fsSL https://install.opensre.com | bash\n%s\n\n' "$CB" "$CB"
|
||||
printf 'Equivalent explicit main channel:\n\n%sbash\ncurl -fsSL https://install.opensre.com | bash -s -- --main\n%s\n\n' "$CB" "$CB"
|
||||
printf 'Windows:\n\n%spowershell\nirm https://install.opensre.com | iex\n%s\n' "$CB" "$CB"
|
||||
} > MAIN_RELEASE_NOTES.md
|
||||
|
||||
- name: Move main build tag to the latest commit
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git tag -f "${{ needs.prepare.outputs.tag_name }}" "$GITHUB_SHA"
|
||||
git push origin "refs/tags/${{ needs.prepare.outputs.tag_name }}" --force
|
||||
|
||||
- name: Publish rolling main release
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
tag_name="${{ needs.prepare.outputs.tag_name }}"
|
||||
|
||||
if gh release view "$tag_name" --repo "${{ github.repository }}" >/dev/null 2>&1; then
|
||||
gh release upload "$tag_name" main-release-assets/* --repo "${{ github.repository }}" --clobber
|
||||
gh release edit "$tag_name" \
|
||||
--repo "${{ github.repository }}" \
|
||||
--title "Main" \
|
||||
--notes-file MAIN_RELEASE_NOTES.md
|
||||
exit 0
|
||||
fi
|
||||
|
||||
gh release create "$tag_name" \
|
||||
main-release-assets/* \
|
||||
--repo "${{ github.repository }}" \
|
||||
--target "$GITHUB_SHA" \
|
||||
--title "Main" \
|
||||
--notes-file MAIN_RELEASE_NOTES.md \
|
||||
--prerelease
|
||||
@@ -0,0 +1,84 @@
|
||||
name: Synthetic Deterministic Tests
|
||||
|
||||
# Regression guardrail: run every offline (no-LLM) synthetic test on every
|
||||
# PR and merge so pipeline refactors can't silently break investigations.
|
||||
#
|
||||
# Tests that need a live LLM are excluded via the marker filter below.
|
||||
# Those suites live in interactive-shell-live.yml and hermes-tests.yml instead.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "surfaces/**"
|
||||
- "config/**"
|
||||
- "core/**"
|
||||
- "integrations/**"
|
||||
- "platform/**"
|
||||
- "tools/**"
|
||||
- "tests/synthetic/**"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- "pytest.ini"
|
||||
- ".github/workflows/synthetic-deterministic.yml"
|
||||
pull_request:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "surfaces/**"
|
||||
- "config/**"
|
||||
- "core/**"
|
||||
- "integrations/**"
|
||||
- "platform/**"
|
||||
- "tools/**"
|
||||
- "tests/synthetic/**"
|
||||
- "pyproject.toml"
|
||||
- "uv.lock"
|
||||
- "pytest.ini"
|
||||
- ".github/workflows/synthetic-deterministic.yml"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: synthetic-det-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
offline:
|
||||
name: Synthetic offline (deterministic)
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Run deterministic synthetic tests
|
||||
# Excludes markers that require a live LLM:
|
||||
# synthetic — full RCA scenario suites (EKS, RDS, Hermes, Dagster …)
|
||||
# axis2 — adversarial scenario suites (also call the LLM)
|
||||
# live_llm — explicit live-credential gate
|
||||
# e2e — requires live infrastructure
|
||||
run: |
|
||||
uv run pytest tests/synthetic \
|
||||
-m "not synthetic and not live_llm and not e2e and not axis2" \
|
||||
-q
|
||||
|
||||
- name: Run hermes_rca offline scenario suite
|
||||
run: uv run python -m tests.synthetic.hermes_rca.run_suite --offline-only
|
||||
@@ -0,0 +1,282 @@
|
||||
name: terraform-bench (plan)
|
||||
|
||||
# Runs terraform fmt + init + validate + plan against tests/benchmarks/cloudopsbench/infra/ on every PR
|
||||
# that touches the bench infra. Posts the plan output as a sticky PR comment so
|
||||
# reviewers can see what would change before approving.
|
||||
#
|
||||
# Never applies — apply is developer-local in v1 (see tests/benchmarks/cloudopsbench/infra/README.md).
|
||||
# When apply-from-CI is desirable later, add a separate workflow that runs on
|
||||
# merge-to-main with a different (higher-privileged) AWS role.
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/benchmarks/cloudopsbench/infra/**"
|
||||
- ".github/workflows/terraform-bench.yml"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
id-token: write # required for AWS OIDC role assumption
|
||||
|
||||
concurrency:
|
||||
group: terraform-bench-${{ github.workflow }}-${{ github.head_ref || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
plan:
|
||||
name: terraform plan
|
||||
if: github.repository == 'Tracer-Cloud/opensre' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository)
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: tests/benchmarks/cloudopsbench/infra
|
||||
|
||||
env:
|
||||
AWS_REGION: us-east-1
|
||||
# tfplan + plan.txt artifacts are not uploaded; plan output goes into the
|
||||
# PR comment. If you want a downloadable plan binary, add an
|
||||
# actions/upload-artifact step after `terraform show`.
|
||||
TF_IN_AUTOMATION: "true"
|
||||
TF_INPUT: "0"
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v5
|
||||
|
||||
- name: Setup Terraform
|
||||
uses: hashicorp/setup-terraform@v3
|
||||
with:
|
||||
terraform_version: "1.7.5"
|
||||
# Wrapper MUST stay enabled (default) — it's what populates
|
||||
# `steps.<id>.outputs.stdout` for the `terraform show` step that
|
||||
# feeds the sticky PR comment. Disabling it makes the comment
|
||||
# render with a blank plan body. See greptile review on initial
|
||||
# PR for details.
|
||||
|
||||
- name: Configure AWS credentials (OIDC role assumption)
|
||||
# No long-lived AWS keys. The role is provisioned by tests/benchmarks/cloudopsbench/infra/
|
||||
# Terraform; ARN is hardcoded here because Terraform outputs aren't
|
||||
# available before the workflow can run. Account ID is the
|
||||
# tracer-cloud account.
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
role-to-assume: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/opensre-bench-terraform-plan
|
||||
role-session-name: github-actions-terraform-plan-${{ github.run_id }}
|
||||
aws-region: us-east-1
|
||||
|
||||
- name: terraform fmt
|
||||
id: fmt
|
||||
run: terraform fmt -check -recursive -no-color
|
||||
continue-on-error: true
|
||||
|
||||
- name: terraform init
|
||||
id: init
|
||||
run: terraform init -no-color
|
||||
|
||||
- name: terraform validate
|
||||
id: validate
|
||||
run: terraform validate -no-color
|
||||
|
||||
- name: terraform plan
|
||||
id: plan
|
||||
run: terraform plan -no-color -input=false -lock=false -out=tfplan
|
||||
continue-on-error: true
|
||||
|
||||
- name: terraform show
|
||||
id: show
|
||||
if: steps.plan.outcome == 'success'
|
||||
run: terraform show -no-color tfplan
|
||||
|
||||
- name: Post plan to PR
|
||||
if: github.event_name == 'pull_request' && always()
|
||||
uses: actions/github-script@v7
|
||||
env:
|
||||
PLAN_STDOUT: ${{ steps.show.outputs.stdout }}
|
||||
FMT_OUTCOME: ${{ steps.fmt.outcome }}
|
||||
INIT_OUTCOME: ${{ steps.init.outcome }}
|
||||
VALIDATE_OUTCOME: ${{ steps.validate.outcome }}
|
||||
PLAN_OUTCOME: ${{ steps.plan.outcome }}
|
||||
WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
with:
|
||||
script: |
|
||||
const fs = require('fs');
|
||||
const MARKER = '<!-- terraform-bench-plan-comment -->';
|
||||
|
||||
const fmt = process.env.FMT_OUTCOME;
|
||||
const init = process.env.INIT_OUTCOME;
|
||||
const validate = process.env.VALIDATE_OUTCOME;
|
||||
const plan = process.env.PLAN_OUTCOME;
|
||||
const url = process.env.WORKFLOW_URL;
|
||||
|
||||
const icon = (outcome) => outcome === 'success' ? '✅' :
|
||||
outcome === 'failure' ? '❌' :
|
||||
outcome === 'skipped' ? '⏭️' : '⚠️';
|
||||
|
||||
let planBody = process.env.PLAN_STDOUT || '';
|
||||
// GitHub PR comments cap at 65536 chars. Reserve ~1000 for surrounding text.
|
||||
const MAX = 60000;
|
||||
let truncated = false;
|
||||
if (planBody.length > MAX) {
|
||||
planBody = planBody.slice(-MAX);
|
||||
truncated = true;
|
||||
}
|
||||
|
||||
const body = [
|
||||
MARKER,
|
||||
'### terraform-bench plan',
|
||||
'',
|
||||
'| step | outcome |',
|
||||
'| --- | --- |',
|
||||
`| fmt | ${icon(fmt)} \`${fmt}\` |`,
|
||||
`| init | ${icon(init)} \`${init}\` |`,
|
||||
`| validate | ${icon(validate)} \`${validate}\` |`,
|
||||
`| plan | ${icon(plan)} \`${plan}\` |`,
|
||||
'',
|
||||
fmt !== 'success' ? '> Run `terraform fmt -recursive` in `tests/benchmarks/cloudopsbench/infra/` to fix formatting.' : '',
|
||||
'',
|
||||
plan === 'success'
|
||||
? `<details><summary>Plan output${truncated ? ' (truncated — see <a href="' + url + '">workflow logs</a> for full output)' : ''}</summary>\n\n\`\`\`hcl\n${planBody}\n\`\`\`\n\n</details>`
|
||||
: `Plan did not run successfully — see [workflow logs](${url}) for details.`,
|
||||
'',
|
||||
`_Updated by [\`terraform-bench.yml\`](${url})._`,
|
||||
].join('\n');
|
||||
|
||||
// Find an existing comment to update, else create a new one.
|
||||
const { data: comments } = await github.rest.issues.listComments({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.issue.number,
|
||||
});
|
||||
const existing = comments.find(c => c.body && c.body.includes(MARKER));
|
||||
|
||||
if (existing) {
|
||||
await github.rest.issues.updateComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
comment_id: existing.id,
|
||||
body,
|
||||
});
|
||||
} else {
|
||||
await github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.issue.number,
|
||||
body,
|
||||
});
|
||||
}
|
||||
|
||||
- name: Fail if any step errored
|
||||
if: |
|
||||
steps.fmt.outcome == 'failure' ||
|
||||
steps.init.outcome == 'failure' ||
|
||||
steps.validate.outcome == 'failure' ||
|
||||
steps.plan.outcome == 'failure'
|
||||
run: |
|
||||
echo "::error::One or more terraform steps failed. See PR comment + workflow logs."
|
||||
exit 1
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# Security + lint scanners — run in parallel with plan. #
|
||||
# #
|
||||
# Suppression policy (intentional skips) is defined per-tool: #
|
||||
# - checkov: tests/benchmarks/cloudopsbench/infra/.checkov.yaml #
|
||||
# - tfsec: tests/benchmarks/cloudopsbench/infra/.tfsec/config.yml #
|
||||
# - tflint: tests/benchmarks/cloudopsbench/infra/.tflint.hcl #
|
||||
# Each suppression includes a justification at the source. Day-one #
|
||||
# findings should be REAL findings — change configs, not the workflow, #
|
||||
# if a rule needs to be skipped. #
|
||||
# #
|
||||
# checkov + tfsec hard-fail (security blocks PRs). tflint soft-fails #
|
||||
# because its rules cover code quality / deprecations, not security — #
|
||||
# we want visibility without blocking. #
|
||||
# ---------------------------------------------------------------------- #
|
||||
|
||||
tflint:
|
||||
name: tflint
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: tests/benchmarks/cloudopsbench/infra
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v5
|
||||
|
||||
- name: Setup tflint
|
||||
uses: terraform-linters/setup-tflint@v4
|
||||
with:
|
||||
tflint_version: latest
|
||||
|
||||
- name: tflint --init (download plugins)
|
||||
run: tflint --init
|
||||
|
||||
- name: tflint
|
||||
id: tflint
|
||||
run: tflint --recursive --format compact
|
||||
continue-on-error: true
|
||||
|
||||
- name: Surface tflint findings (non-blocking)
|
||||
if: steps.tflint.outcome == 'failure'
|
||||
run: |
|
||||
echo "::warning::tflint reported findings. Code-quality only — does not block PR. Review in workflow logs."
|
||||
|
||||
tfsec:
|
||||
name: tfsec
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v5
|
||||
|
||||
# tfsec via Docker, bypassing aquasecurity/tfsec-action.
|
||||
#
|
||||
# Why direct Docker run: the action shells out to api.github.com to
|
||||
# resolve the latest release binary, and the aquasecurity org has an
|
||||
# IP allow list that blocks GitHub Actions runner IPs — making the
|
||||
# action effectively unusable from CI. The aquasec/tfsec image on
|
||||
# Docker Hub has no such restriction.
|
||||
#
|
||||
# Migration path (v2): tfsec is in maintenance mode; rule set lives on
|
||||
# in Trivy. Replace with `aquasec/trivy:latest config /src` when
|
||||
# ready — same .tfsec/config.yml is recognized.
|
||||
#
|
||||
# Suppressions: tests/benchmarks/cloudopsbench/infra/.tfsec/config.yml. tfsec exits 1 on any
|
||||
# finding by default, which fails the step — exactly the hard-fail
|
||||
# behavior we want. Add a justified exclude to the config rather
|
||||
# than disabling the step.
|
||||
- name: tfsec scan (via Docker)
|
||||
run: |
|
||||
docker run --rm \
|
||||
-v "${{ github.workspace }}/tests/benchmarks/cloudopsbench/infra:/src" \
|
||||
aquasec/tfsec:latest \
|
||||
/src \
|
||||
--config-file=/src/.tfsec/config.yml \
|
||||
--no-color
|
||||
|
||||
checkov:
|
||||
name: checkov
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v5
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
# Suppressions live in tests/benchmarks/cloudopsbench/infra/.checkov.yaml. soft_fail is
|
||||
# FALSE — real findings block the PR. Add a justified skip-check
|
||||
# to the config rather than disabling here.
|
||||
- name: checkov scan
|
||||
uses: bridgecrewio/checkov-action@v12
|
||||
with:
|
||||
directory: tests/benchmarks/cloudopsbench/infra
|
||||
framework: terraform
|
||||
config_file: tests/benchmarks/cloudopsbench/infra/.checkov.yaml
|
||||
quiet: true
|
||||
soft_fail: false
|
||||
# Re-enable SARIF upload when CodeQL alerts are wanted:
|
||||
# output_format: cli,sarif
|
||||
# output_file_path: console,checkov-results.sarif
|
||||
@@ -0,0 +1,76 @@
|
||||
name: Tracer Demo – Prefect ECS
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
tracer-demo-prefect-ecs:
|
||||
if: github.repository == 'Tracer-Cloud/opensre'
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
JWT_TOKEN: ${{ secrets.JWT_TOKEN }}
|
||||
TRACER_ORG_ID: ${{ secrets.TRACER_ORG_ID }}
|
||||
TRACER_WEB_APP_URL: ${{ secrets.TRACER_WEB_APP_URL }}
|
||||
TRACER_API_URL: ${{ secrets.TRACER_API_URL }}
|
||||
TRACER_INGEST_TOKEN: ${{ secrets.TRACER_INGEST_TOKEN }}
|
||||
|
||||
AWS_REGION: us-east-1
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install Python dependencies
|
||||
run: uv sync --frozen --extra dev
|
||||
|
||||
- name: Configure AWS Credentials
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
aws-region: us-east-1
|
||||
|
||||
- name: Install tracer CLI
|
||||
run: |
|
||||
curl -sSL https://install.tracer.cloud | CLI_BRANCH=dev sh -s user_36fbN6K6FwEgJFHsQv1pUByo8K6
|
||||
|
||||
- name: Initialize tracer
|
||||
run: |
|
||||
sudo tracer init --token ${{ secrets.JWT_TOKEN }}
|
||||
|
||||
- name: Export Tracer run id
|
||||
shell: bash
|
||||
run: |
|
||||
RUN_ID=$(tracer info --json | python3 -c "import sys,json; print(json.load(sys.stdin)['run']['id'])")
|
||||
PIPELINE_NAME=$(tracer info --json | python3 -c "import sys,json; print(json.load(sys.stdin)['pipeline']['name'])")
|
||||
ORG_SLUG=$(tracer info --json | python3 -c "import sys,json; print(json.load(sys.stdin)['pipeline']['organization'])")
|
||||
|
||||
echo "TRACER_RUN_ID=$RUN_ID" >> $GITHUB_ENV
|
||||
echo "TRACER_PIPELINE_NAME=$PIPELINE_NAME" >> $GITHUB_ENV
|
||||
echo "TRACER_ORG_SLUG=$ORG_SLUG" >> $GITHUB_ENV
|
||||
|
||||
# Optional but helpful: stable trace id for ingest rows
|
||||
echo "TRACER_TRACE_ID=trace_$RUN_ID" >> $GITHUB_ENV
|
||||
|
||||
echo "Exported TRACER_RUN_ID=$RUN_ID"
|
||||
|
||||
- name: Run Prefect ECS Demo
|
||||
continue-on-error: true
|
||||
run: |
|
||||
uv run python -m tests.e2e.upstream_prefect_ecs_fargate.test_agent_e2e
|
||||
@@ -0,0 +1,85 @@
|
||||
name: Trigger K8s Alert
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
verify:
|
||||
description: "Verify logs in Datadog + post to Slack"
|
||||
type: boolean
|
||||
default: true
|
||||
schedule:
|
||||
- cron: "0 9 * * 1" # every Monday 9am UTC
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
trigger-alert:
|
||||
if: github.repository == 'Tracer-Cloud/opensre'
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
AWS_REGION: us-east-1
|
||||
DD_API_KEY: ${{ secrets.DD_API_KEY }}
|
||||
DD_APP_KEY: ${{ secrets.DD_APP_KEY }}
|
||||
DD_SITE: datadoghq.com
|
||||
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
SLACK_DEVS_ALERTS_CHANNEL_ID: C09S7GDG60J
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
|
||||
- name: Configure AWS credentials
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
aws-region: ${{ env.AWS_REGION }}
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install project
|
||||
run: uv sync --frozen
|
||||
|
||||
- name: Trigger pipeline failure via centralized config (~10s)
|
||||
id: trigger
|
||||
run: |
|
||||
echo "trigger_epoch=$(date +%s)" >> "$GITHUB_OUTPUT"
|
||||
set +e
|
||||
uv run python -m tests.e2e.kubernetes.trigger_alert | tee /tmp/trigger.log
|
||||
trigger_exit=${PIPESTATUS[0]}
|
||||
set -e
|
||||
echo "trigger_completed_epoch=$(date +%s)" >> "$GITHUB_OUTPUT"
|
||||
if grep -q '"status": "accepted_timeout"' /tmp/trigger.log; then
|
||||
echo "trigger_accepted_timeout=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "trigger_accepted_timeout=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
exit "$trigger_exit"
|
||||
|
||||
- name: Report trigger result
|
||||
run: |
|
||||
echo "Trigger command completed successfully"
|
||||
|
||||
- name: Verify Datadog + Slack (~5 min)
|
||||
if: ${{ inputs.verify == true || github.event_name == 'schedule' }}
|
||||
run: |
|
||||
POST_TRIGGER_WAIT=""
|
||||
if [ "${{ steps.trigger.outputs.trigger_accepted_timeout }}" = "true" ]; then
|
||||
POST_TRIGGER_WAIT="--post-trigger-wait 90"
|
||||
fi
|
||||
uv run python -m tests.e2e.kubernetes.trigger_alert \
|
||||
--verify-only \
|
||||
--since-epoch ${{ steps.trigger.outputs.trigger_completed_epoch }} \
|
||||
--dd-max-wait 300 \
|
||||
$POST_TRIGGER_WAIT
|
||||