chore: import upstream snapshot with attribution
CI (OpenClaw E2E) / openclaw test (push) Has been cancelled
CI / coverage-report (push) Has been cancelled
CI / test-kubernetes (push) Has been cancelled
CI / should-run-thorough (push) Has been cancelled
CI / test-thorough (cloudwatch-demo) (push) Has been cancelled
CI / test-thorough (flink-ecs) (push) Has been cancelled
CI / test-thorough (upstream-lambda) (push) Has been cancelled
CI / test-thorough (prefect-ecs-fargate) (push) Has been cancelled
Release / build-binaries (zip, opensre.exe, onefile, windows-latest, windows-x64) (push) Has been cancelled
Benchmark image — build + push to ECR (any adapter) / build + push (push) Has been cancelled
CI / quality (ubuntu-latest) (push) Has been cancelled
CI / test (tools-runtime) (push) Has been cancelled
CI / test (e2e-general) (push) Has been cancelled
CI / test (cli-runtime) (push) Has been cancelled
CI / test (e2e-provider-and-openclaw) (push) Has been cancelled
CI / test (integrations-and-misc) (push) Has been cancelled
Release / verify (push) Has been cancelled
Release / build-python-dist (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-15-intel, darwin-x64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-latest, darwin-arm64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04, linux-x64) (push) Has been cancelled
Release / publish-release (push) Has been cancelled
Release / publish-main-release (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-checks (no-LLM) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-live shard ${{ matrix.shard_index }} (push) Has been cancelled
Release / prepare (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04-arm, linux-arm64) (push) Has been cancelled
Synthetic Deterministic Tests / Synthetic offline (deterministic) (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:10:45 +08:00
commit 4b6817381b
3933 changed files with 525247 additions and 0 deletions
+92
View File
@@ -0,0 +1,92 @@
name: Bug Report
description: Report something that isn't working
title: "[BUG] "
labels:
- bug
body:
- type: markdown
attributes:
value: |
> **Security notice:** Never paste real API keys, tokens, or passwords in this form. Replace them with placeholders like `glsa_***`, `sk-***`, or `your-token-here` before submitting. Tokens posted here are publicly visible and cannot be fully deleted from GitHub's history.
Thanks for the report. Please include **observed** behavior and repro steps — guesses make triage slower.
- type: textarea
id: summary
attributes:
label: Summary
description: One-line description of the bug. Be specific (e.g. agent fails when `.env` contains `=` in values). Avoid vague phrases like "something doesn't work."
placeholder: "Example: Agent fails to parse config when .env vars contain special characters"
validations:
required: true
- type: textarea
id: expected_actual
attributes:
label: Expected vs actual behavior
description: What should happen, and what happens instead? Include exact errors, exit codes, or unexpected output where relevant.
placeholder: |
**Expected:** …
**Actual:** …
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Steps to reproduce
description: Minimal steps that consistently trigger the bug (commands, config snippets — redact secrets).
placeholder: |
1.
2.
3.
validations:
required: true
- type: dropdown
id: reproducible
attributes:
label: Can you reproduce it consistently?
options:
- "Yes"
- "No"
- "Sometimes"
validations:
required: true
- type: dropdown
id: frequency
attributes:
label: How often does it occur?
options:
- Every time
- Intermittent
- Under specific conditions
validations:
required: true
- type: dropdown
id: os
attributes:
label: Operating system
description: Pick the closest match.
options:
- macOS
- Linux
- Windows
- Other
validations:
required: true
- type: textarea
id: logs
attributes:
label: Logs and error output
description: Full error message and relevant logs. Redact API keys, tokens, and passwords.
placeholder: |
```
[paste error / logs]
```
validations:
required: false
- type: textarea
id: context
attributes:
label: Additional context
description: Screenshots, related issues (`#123`), what you were trying to do when this happened.
validations:
required: false
+8
View File
@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: "Questions or Support"
url: https://github.com/Tracer-Cloud/open-sre-agent/discussions
about: "Ask questions or get help in Discussions"
- name: "Security Issue"
url: https://github.com/Tracer-Cloud/open-sre-agent/security/policy
about: "Report a security vulnerability safely"
@@ -0,0 +1,22 @@
name: Feature Request
description: Suggest a new feature or capability
title: "[FEATURE] "
labels:
- enhancement
- pending triage
body:
- type: textarea
id: problem
attributes:
label: Problem statement
description: Why is this needed? User pain, limitation, or gap today (not the solution yet).
placeholder: "Example: Users can't export agent logs to external systems without custom scripts."
validations:
required: true
- type: textarea
id: solution
attributes:
label: Proposed solution
description: How should it work? Commands, APIs, UX, inputs/outputs, example snippets.
validations:
required: true
+20
View File
@@ -0,0 +1,20 @@
name: Improvement
description: Suggest a refactor, optimization, or quality improvement
title: "[IMPROVEMENT] "
labels:
- enhancement
body:
- type: textarea
id: current
attributes:
label: Current state
description: How does it work now? Link to code (`file:line`), snippets, or behavior/architecture.
validations:
required: true
- type: textarea
id: desired
attributes:
label: Desired state
description: What should change, why it is better (metrics if possible), and any architectural notes.
validations:
required: true
+51
View File
@@ -0,0 +1,51 @@
Fixes #
<!-- Add issue number above -->
#### Describe the changes you have made in this PR -
### Demo/Screenshot for feature changes and bug fixes -
<!-- Include at least one proof of the change: UI screenshot, terminal screenshot/log snippet, short video/GIF, or equivalent demo output. -->
<!-- Do not add code diff here -->
---
## Code Understanding and AI Usage
**Did you use AI assistance (ChatGPT, Claude, Copilot, etc.) to write any part of this code?**
- [ ] No, I wrote all the code myself
- [ ] Yes, I used AI assistance (continue below)
**If you used AI assistance:**
- [ ] I have reviewed every single line of the AI-generated code
- [ ] I can explain the purpose and logic of each function/component I added
- [ ] I have tested edge cases and understand how the code handles them
- [ ] I have modified the AI output to follow this project's coding standards and conventions
**Explain your implementation approach:**
<!--
Describe in your own words:
- What problem does your code solve?
- What alternative approaches did you consider?
- Why did you choose this specific implementation?
- What are the key functions/components and what do they do?
This helps reviewers understand your thought process and ensures you understand the code.
-->
---
## Checklist before requesting a review
- [ ] I have added proper PR title and linked to the issue
- [ ] I have performed a self-review of my code
- [ ] **I can explain the purpose of every function, class, and logic block I added**
- [ ] I understand why my changes work and have tested them thoroughly
- [ ] I have considered potential edge cases and how my code handles them
- [ ] If it is a core feature, I have added thorough tests
- [ ] My code follows the project's style guidelines and conventions
---
Note: Please check **Allow edits from maintainers** if you would like us to assist in the PR.
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.9 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 817 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 476 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.5 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 374 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 438 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 890 B

+237
View File
@@ -0,0 +1,237 @@
"""Enforce forbidden *direct* import edges between top-level packages.
Unlike import-linter (which flags transitive chains), this checker looks at
**module-top-level** and **nested** (function/class-body) ``import`` /
``from … import`` statements. Module-top-level uses the same AST walk as
``check_import_cycles``; nested imports close the loophole where lazy
``surfaces.*`` imports bypass the module-level graph.
Used by ``make check-imports`` (and ``check_imports``) alongside
import-linter's config contract.
"""
from __future__ import annotations
import sys
from dataclasses import dataclass
from pathlib import Path
_CI_DIR = Path(__file__).resolve().parent
if str(_CI_DIR) not in sys.path:
sys.path.insert(0, str(_CI_DIR))
from check_import_cycles import ( # noqa: E402
_build_graph,
_nested_imports,
discover_first_party_roots,
module_from_path,
)
_REPO_ROOT = Path(__file__).resolve().parents[2]
# ``source_prefix -> forbidden destination roots`` for direct imports only.
# Enforces the layering contract documented in ``surfaces/__init__.py``:
# "Nothing first-party may import from surfaces/". Adds an explicit bound
# on ``platform``, ``core``, ``gateway`` so the surfaces ban
# is CI-enforced, not just doc-described.
_FORBIDDEN_DIRECT: dict[str, frozenset[str]] = {
"platform": frozenset({"surfaces"}),
"core": frozenset({"surfaces"}),
"gateway": frozenset({"surfaces"}),
"integrations": frozenset({"tools", "surfaces"}),
"tools": frozenset({"surfaces"}),
}
# Known direct violations being burned down — remove entries as fixes land.
# Format: ``"source.module -> dest.module"`` (exact modules from the graph).
_BASELINE_IGNORES: frozenset[str] = frozenset(
{
# Gateway hosts the interactive_shell runtime — pre-existing reuse
# to be burned down by extracting shared runtime primitives out of
# ``surfaces/interactive_shell/`` and into a layer below ``surfaces``.
"gateway.storage.session.resolver -> surfaces.interactive_shell.runtime.context",
# tools/interactive_shell action tools reach UP into surfaces/interactive_shell
# for runtime / command_registry / UI primitives. Clears when the action
# tools themselves are refactored to be UI-agnostic (e.g. return
# "approval-required" sentinels instead of calling execution_confirm
# directly) so the surface owns its own confirmation UX.
"tools.interactive_shell.actions.cli_command -> surfaces.interactive_shell.runtime.subprocess_runner",
# These shell action tools type against the shell ``Session`` (investigation_launch
# reads session.terminal.background_mode_enabled). Clears when they are made
# session-core-agnostic so they no longer import the shell session type.
"tools.interactive_shell.actions.investigation -> surfaces.interactive_shell.session",
"tools.interactive_shell.actions.sample_alert -> surfaces.interactive_shell.session",
"tools.interactive_shell.shared.investigation_launch -> surfaces.interactive_shell.session",
"tools.interactive_shell.actions.llm_provider -> surfaces.interactive_shell.command_registry",
"tools.interactive_shell.actions.llm_provider -> surfaces.interactive_shell.ui.execution_confirm",
"tools.interactive_shell.actions.slash -> surfaces.interactive_shell.command_registry",
"tools.interactive_shell.actions.slash -> surfaces.interactive_shell.command_registry.slash_catalog",
"tools.interactive_shell.actions.slash -> surfaces.interactive_shell.ui",
"tools.interactive_shell.actions.slash -> surfaces.interactive_shell.ui.execution_confirm",
"tools.interactive_shell.actions.slash -> surfaces.interactive_shell.utils.telemetry.turn_outcome",
"tools.interactive_shell.actions.task_cancel -> surfaces.interactive_shell.command_registry",
"tools.interactive_shell.actions.task_cancel -> surfaces.interactive_shell.runtime",
"tools.interactive_shell.actions.task_cancel -> surfaces.interactive_shell.ui.execution_confirm",
"tools.interactive_shell.implementation.claude_code_executor -> surfaces.interactive_shell.runtime",
"tools.interactive_shell.implementation.claude_code_executor -> surfaces.interactive_shell.runtime.subprocess_runner.task_streaming",
"tools.interactive_shell.implementation.claude_code_executor -> surfaces.interactive_shell.ui",
"tools.interactive_shell.implementation.claude_code_executor -> surfaces.interactive_shell.ui.execution_confirm",
"tools.interactive_shell.implementation.claude_code_executor -> surfaces.interactive_shell.utils.error_handling.exception_reporting",
"tools.interactive_shell.shell.runner -> surfaces.interactive_shell.runtime",
"tools.interactive_shell.shell.runner -> surfaces.interactive_shell.runtime.subprocess_runner.task_streaming",
"tools.interactive_shell.shell.runner -> surfaces.interactive_shell.ui",
"tools.interactive_shell.shell.runner -> surfaces.interactive_shell.ui.execution_confirm",
"tools.interactive_shell.shell.runner -> surfaces.interactive_shell.utils.error_handling.exception_reporting",
"tools.interactive_shell.synthetic.runner -> surfaces.interactive_shell.runtime",
"tools.interactive_shell.synthetic.runner -> surfaces.interactive_shell.runtime.subprocess_runner.task_streaming",
"tools.interactive_shell.synthetic.runner -> surfaces.interactive_shell.ui",
"tools.interactive_shell.synthetic.runner -> surfaces.interactive_shell.ui.execution_confirm",
"tools.interactive_shell.synthetic.runner -> surfaces.interactive_shell.utils.error_handling.exception_reporting",
}
)
# Function/class-body lazy imports that bypass the module-level graph.
# Format matches ``_BASELINE_IGNORES``; burn down by moving shared code
# below ``surfaces/`` or making tools UI-agnostic.
_NESTED_BASELINE_IGNORES: frozenset[str] = frozenset(
{
"tools.interactive_shell.actions.investigation -> surfaces.interactive_shell.runtime.background.runner",
"tools.interactive_shell.actions.investigation -> surfaces.interactive_shell.runtime.investigation_adapter",
"tools.interactive_shell.actions.sample_alert -> surfaces.interactive_shell.runtime.background.runner",
"tools.interactive_shell.actions.sample_alert -> surfaces.interactive_shell.runtime.investigation_adapter",
"tools.interactive_shell.actions.llm_provider -> surfaces.cli.wizard.config",
# Nested only: runner.py also has a module-level ui import (see _BASELINE_IGNORES).
"tools.interactive_shell.shell.runner -> surfaces.interactive_shell.ui",
}
)
@dataclass(frozen=True)
class DirectViolation:
source: str
target: str
@property
def edge(self) -> str:
return f"{self.source} -> {self.target}"
@dataclass(frozen=True)
class NestedViolation:
source: str
target: str
lineno: int
@property
def edge(self) -> str:
return f"{self.source} -> {self.target}"
def _source_root(module: str) -> str:
return module.split(".", 1)[0]
def find_direct_violations(
graph: dict[str, set[str]],
*,
forbidden: dict[str, frozenset[str]] | None = None,
baseline_ignores: frozenset[str] | None = None,
) -> list[DirectViolation]:
rules = forbidden or _FORBIDDEN_DIRECT
ignores = baseline_ignores if baseline_ignores is not None else _BASELINE_IGNORES
violations: list[DirectViolation] = []
for source_module, targets in sorted(graph.items()):
source_root = _source_root(source_module)
forbidden_roots = rules.get(source_root)
if not forbidden_roots:
continue
for target_module in sorted(targets):
target_root = _source_root(target_module)
if target_root not in forbidden_roots:
continue
edge = DirectViolation(source_module, target_module)
if edge.edge in ignores:
continue
violations.append(edge)
return violations
def find_nested_direct_violations(
root: Path,
first_party_roots: tuple[str, ...],
*,
forbidden: dict[str, frozenset[str]] | None = None,
baseline_ignores: frozenset[str] | None = None,
) -> list[NestedViolation]:
rules = forbidden or _FORBIDDEN_DIRECT
ignores = baseline_ignores if baseline_ignores is not None else _NESTED_BASELINE_IGNORES
roots = frozenset(first_party_roots)
violations: list[NestedViolation] = []
for pkg in first_party_roots:
if pkg not in rules:
continue
pkg_path = root / pkg
if not pkg_path.exists():
continue
for py in pkg_path.rglob("*.py"):
if "__pycache__" in py.parts:
continue
source_module = module_from_path(root, py)
source_root = _source_root(source_module)
forbidden_roots = rules.get(source_root)
if not forbidden_roots:
continue
source = py.read_text(encoding="utf-8")
for target_module, lineno in _nested_imports(source, first_party_roots=roots):
target_root = _source_root(target_module)
if target_root not in forbidden_roots:
continue
violation = NestedViolation(source_module, target_module, lineno)
if violation.edge in ignores:
continue
violations.append(violation)
return sorted(violations, key=lambda item: (item.source, item.lineno, item.target))
def main(argv: list[str] | None = None) -> int:
del argv
root = _REPO_ROOT
first_party_roots = discover_first_party_roots(root)
graph = _build_graph(root, first_party_roots)
module_violations = find_direct_violations(graph)
nested_violations = find_nested_direct_violations(root, first_party_roots)
if not module_violations and not nested_violations:
print(
"No forbidden direct import edges found "
f"(module baseline: {len(_BASELINE_IGNORES)}, "
f"nested baseline: {len(_NESTED_BASELINE_IGNORES)})."
)
return 0
if module_violations:
print(f"FAIL: {len(module_violations)} forbidden module-level direct import edge(s):")
for violation in module_violations:
print(f" {violation.edge}")
if nested_violations:
if module_violations:
print()
print(f"FAIL: {len(nested_violations)} forbidden nested direct import edge(s):")
for violation in nested_violations:
print(f" {violation.edge} (line {violation.lineno})")
print(
"\nFix by moving shared code to a lower layer (platform/common, core/contracts) "
"or add a temporary baseline entry in .github/ci/check_direct_imports.py "
"with a linked issue — do not use function-level lazy imports to hide "
"new direct edges."
)
return 1
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+342
View File
@@ -0,0 +1,342 @@
"""Detect first-party module-load import cycles via Tarjan's SCC.
Walks every first-party Python module in the repo, builds an import
graph from **top-level** ``import`` / ``from ... import`` statements
only (function-level lazy imports are intentional runtime breaks and
not counted), then reports any strongly-connected component of size
> 1, plus any single-module self-loop.
Used by ``make check-imports`` (via ``check_imports``) locally and by CI.
Exit codes:
0 — zero cycles found
1 — at least one cycle found (output lists every SCC + its edges)
How to break a cycle
====================
Two patterns work. Pick by what consumers do with the name.
1. ``import pkg.sub as sub`` (preferred when consumers monkeypatch)
Switch ::
from pkg.sub import name # binds ``name`` at import time
...
name(args) # uses the bound reference
to ::
import pkg.sub as sub # imports the submodule directly
...
sub.name(args) # attribute lookup at call time
Both break the static cycle (no edge from the consumer back to the
``pkg`` package). The second form keeps attribute-lookup semantics,
which matters whenever consumers monkeypatch ``pkg.sub.name`` in
tests — the patched attribute IS looked up at each call. The first
form binds the name at import time and ignores later patching.
2. Port / Protocol (when the cycle crosses architectural layers)
When the cycle is ``layerA <-> layerB`` (e.g. analytics ↔ sentry,
integrations ↔ services), neither side should depend on the other
directly. Extract a third module — a ``Protocol`` or a small
abstract dataclass — that both depend on, and inject the concrete
implementation at startup. See the verifier plugin-registry
refactor (``integrations/verification/registry.py``) for the
canonical example.
Avoid ``from pkg import sub`` (where ``sub`` is a submodule of ``pkg``)
inside that ``pkg`` itself or any of its children — that's the form
this script flags. It triggers ``pkg``'s ``__init__`` even when you
just want the submodule, and re-export patterns in ``__init__`` close
the loop.
Function-local imports are NOT flagged. They're a legitimate Python
pattern for startup-cost deferral (heavy modules in click subcommand
bodies), optional dependencies, and conditional / platform-specific
code paths. Use sparingly — keep top-level the default — and comment
the *why* so future readers don't mistake them for cycle workarounds.
"""
from __future__ import annotations
import ast
import sys
from collections import defaultdict
from collections.abc import Iterable
from functools import lru_cache
from pathlib import Path
# Directories at the repo root that are never first-party import roots.
_SKIP_ROOT_DIRS = frozenset(
{
".git",
".github",
".pytest_cache",
".ruff_cache",
".venv",
"__pycache__",
"docs",
"opensre.egg-info",
"packaging",
"tests",
"venv",
}
)
@lru_cache(maxsize=1)
def discover_first_party_roots(repo_root: Path | None = None) -> tuple[str, ...]:
"""Return top-level package names that contain importable Python code."""
root = repo_root or Path(__file__).resolve().parents[2]
names: list[str] = []
for child in sorted(root.iterdir()):
if not child.is_dir() or child.name.startswith("."):
continue
if child.name in _SKIP_ROOT_DIRS:
continue
if not any(child.rglob("*.py")):
continue
names.append(child.name)
return tuple(names)
def _top_level_imports(source: str, *, first_party_roots: frozenset[str]) -> set[str]:
"""Return first-party module paths imported at the module top level.
Function-bodies, class-bodies, conditional / try-except wrappers all
count as top-level if they are direct module statements — the only
imports skipped are those nested **inside a function or class body**.
A lazy ``from X import Y`` inside a function does not deadlock at
module load, so it should not be flagged as a cycle.
"""
try:
tree = ast.parse(source)
except SyntaxError:
return set()
names: set[str] = set()
def _add(module_path: str) -> None:
top = module_path.split(".", 1)[0]
if top in first_party_roots:
names.add(module_path)
def _walk_top(body: Iterable[ast.stmt]) -> None:
for node in body:
if isinstance(node, ast.Import):
for alias in node.names:
_add(alias.name)
elif isinstance(node, ast.ImportFrom):
if node.level or not node.module:
continue
_add(node.module)
elif isinstance(node, ast.If):
_walk_top(node.body)
_walk_top(node.orelse)
elif isinstance(node, ast.Try | ast.TryStar):
# ``ast.TryStar`` (Python 3.11+, ``try/except*``) shares
# the same handler/orelse/finalbody shape as ``ast.Try``.
_walk_top(node.body)
for handler in node.handlers:
_walk_top(handler.body)
_walk_top(node.orelse)
_walk_top(node.finalbody)
elif isinstance(node, ast.With | ast.AsyncWith):
_walk_top(node.body)
_walk_top(tree.body)
return names
def _nested_imports(source: str, *, first_party_roots: frozenset[str]) -> list[tuple[str, int]]:
"""Return ``(target_module, lineno)`` for imports inside function/class bodies.
Complements ``_top_level_imports``: catches lazy imports that bypass the
module-level direct-edge checker while still being layering violations.
"""
try:
tree = ast.parse(source)
except SyntaxError:
return []
results: list[tuple[str, int]] = []
def _add(module_path: str, lineno: int) -> None:
top = module_path.split(".", 1)[0]
if top in first_party_roots:
results.append((module_path, lineno))
def _walk_nested(body: Iterable[ast.stmt], *, nested: bool) -> None:
for node in body:
if isinstance(node, ast.Import):
if nested:
for alias in node.names:
_add(alias.name, node.lineno)
elif isinstance(node, ast.ImportFrom):
if nested and not node.level and node.module:
_add(node.module, node.lineno)
elif isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef | ast.ClassDef):
_walk_nested(node.body, nested=True)
elif isinstance(node, ast.If):
_walk_nested(node.body, nested=nested)
_walk_nested(node.orelse, nested=nested)
elif isinstance(node, ast.Try | ast.TryStar):
_walk_nested(node.body, nested=nested)
for handler in node.handlers:
_walk_nested(handler.body, nested=nested)
_walk_nested(node.orelse, nested=nested)
_walk_nested(node.finalbody, nested=nested)
elif isinstance(node, ast.With | ast.AsyncWith):
_walk_nested(node.body, nested=nested)
elif isinstance(node, ast.For | ast.AsyncFor | ast.While):
_walk_nested(node.body, nested=nested)
_walk_nested(node.orelse, nested=nested)
elif isinstance(node, ast.Match):
for case in node.cases:
_walk_nested(case.body, nested=nested)
_walk_nested(tree.body, nested=False)
return results
def module_from_path(root: Path, py: Path) -> str:
"""Resolve a repo-relative ``.py`` path to its dotted module name."""
module = ".".join(py.with_suffix("").relative_to(root).parts)
return module.removesuffix(".__init__")
def _build_graph(root: Path, first_party_roots: tuple[str, ...]) -> dict[str, set[str]]:
"""Build the first-party module-level import graph rooted at ``root``."""
roots = frozenset(first_party_roots)
graph: dict[str, set[str]] = defaultdict(set)
for pkg in first_party_roots:
pkg_path = root / pkg
if not pkg_path.exists():
continue
for py in pkg_path.rglob("*.py"):
if "__pycache__" in py.parts:
continue
module = module_from_path(root, py)
source = py.read_text(encoding="utf-8")
graph[module].update(_top_level_imports(source, first_party_roots=roots))
return graph
def _tarjan_sccs(graph: dict[str, set[str]]) -> list[list[str]]:
"""Return every strongly-connected component of size > 1, plus any
single-module self-loop."""
index: dict[str, int] = {}
lowlink: dict[str, int] = {}
on_stack: dict[str, bool] = {}
stack: list[str] = []
sccs: list[list[str]] = []
counter = [0]
def strongconnect(v: str) -> None:
index[v] = counter[0]
lowlink[v] = counter[0]
counter[0] += 1
stack.append(v)
on_stack[v] = True
for w in graph.get(v, ()):
if w not in index:
strongconnect(w)
lowlink[v] = min(lowlink[v], lowlink[w])
elif on_stack.get(w):
lowlink[v] = min(lowlink[v], index[w])
if lowlink[v] == index[v]:
component: list[str] = []
while True:
w = stack.pop()
on_stack[w] = False
component.append(w)
if w == v:
break
# ``while True`` above guarantees ``component`` is non-empty here.
if len(component) > 1 or component[0] in graph.get(component[0], ()):
sccs.append(component)
sys.setrecursionlimit(10000)
for vertex in list(graph.keys()):
if vertex not in index:
strongconnect(vertex)
return sccs
def _format_scc(scc: list[str], graph: dict[str, set[str]]) -> str:
"""Format an SCC for human-readable output: members + edges within."""
members = sorted(scc)
in_scc = set(scc)
is_self_loop = len(scc) == 1
edges: list[str] = []
for module in members:
for target in sorted(graph.get(module, ())):
if target not in in_scc:
continue
# Single-module self-loops have only the self-edge; show it
# explicitly so the developer knows which import closes the
# loop. Multi-module SCCs hide self-edges to keep the diff
# focused on the cross-module edges that close the cycle.
if target == module and not is_self_loop:
continue
edges.append(f" {module} -> {target}")
lines = [f" Modules ({len(scc)}):"]
lines.extend(f" - {m}" for m in members)
if is_self_loop and not edges:
lines.append(" Self-import detected (module imports itself at top level).")
elif edges:
lines.append(" Edges within SCC:")
lines.extend(edges)
return "\n".join(lines)
def main(argv: list[str] | None = None) -> int:
del argv
root = Path(__file__).resolve().parents[2]
first_party_roots = discover_first_party_roots(root)
graph = _build_graph(root, first_party_roots)
sccs = _tarjan_sccs(graph)
if not sccs:
print(
f"No import cycles found across {len(graph)} first-party modules "
f"({len(first_party_roots)} roots)."
)
return 0
print(f"FAIL: {len(sccs)} import cycle(s) found across {len(graph)} first-party modules.")
for i, scc in enumerate(sorted(sccs, key=lambda s: -len(s)), 1):
print(f"\n## SCC #{i} ({len(scc)} module{'s' if len(scc) > 1 else ''}):")
print(_format_scc(scc, graph))
print(
"\nTo break a cycle, prefer:\n"
" import pkg.sub as sub\n"
" ...\n"
" sub.name(args)\n"
"over:\n"
" from pkg.sub import name\n"
" ...\n"
" name(args)\n"
"\n"
"Both fix the static cycle; only the first keeps attribute-lookup\n"
"semantics so tests that monkeypatch ``pkg.sub.name`` still work.\n"
"\n"
"For cross-layer cycles, introduce a Protocol/port both sides\n"
"depend on (see ``integrations/verification/registry.py`` for the\n"
"canonical example).\n"
"\n"
"Full pattern reference: docstring at the top of this script."
)
return 1
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+99
View File
@@ -0,0 +1,99 @@
"""Run all import-graph quality checks in one command.
Orchestrates, in order:
1. **Cycles** — module-load SCCs (``check_import_cycles``)
2. **Layers** — ``config`` independence via import-linter (``.importlinter``)
3. **Direct edges** — forbidden top-level imports (``check_direct_imports``)
Used by ``make check-imports``, ``make check``, and CI.
Exit codes:
0 — all checks passed
1 — one or more checks failed (each section prints its own detail)
"""
from __future__ import annotations
import subprocess
import sys
from collections.abc import Callable, Sequence
from dataclasses import dataclass
from pathlib import Path
_CI_DIR = Path(__file__).resolve().parent
if str(_CI_DIR) not in sys.path:
sys.path.insert(0, str(_CI_DIR))
from check_direct_imports import main as check_direct_imports # noqa: E402
from check_import_cycles import main as check_import_cycles # noqa: E402
_REPO_ROOT = Path(__file__).resolve().parents[2]
@dataclass(frozen=True)
class ImportCheck:
name: str
run: Callable[[], int]
def _run_importlinter(*, config: Path | None = None) -> int:
lint_imports = Path(sys.executable).with_name("lint-imports")
if not lint_imports.is_file():
print(
"lint-imports not found — install dev deps (import-linter package).",
file=sys.stderr,
)
return 1
command = [str(lint_imports)]
if config is not None:
command.extend(["--config", str(config)])
completed = subprocess.run(command, cwd=_REPO_ROOT, check=False)
return int(completed.returncode)
def import_checks(*, strict_layers: bool = False) -> Sequence[ImportCheck]:
layer_config = _REPO_ROOT / ".importlinter.strict" if strict_layers else None
return (
ImportCheck("Import cycles (Tarjan SCC)", check_import_cycles),
ImportCheck(
"Import layers (import-linter)",
lambda: _run_importlinter(config=layer_config),
),
ImportCheck("Forbidden direct import edges (module + nested)", check_direct_imports),
)
def main(argv: list[str] | None = None) -> int:
args = list(argv or [])
strict_layers = False
if args == ["--strict"]:
strict_layers = True
args = []
if args:
print(f"Unknown arguments: {' '.join(args)}", file=sys.stderr)
print("Usage: check_imports.py [--strict]", file=sys.stderr)
return 2
checks = import_checks(strict_layers=strict_layers)
failures: list[str] = []
for index, check in enumerate(checks, start=1):
print(f"=== [{index}/{len(checks)}] {check.name} ===")
exit_code = check.run()
if exit_code != 0:
failures.append(check.name)
print()
if not failures:
print(f"All {len(checks)} import checks passed.")
return 0
print(f"FAIL: {len(failures)} import check(s) failed:")
for name in failures:
print(f" - {name}")
return 1
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+194
View File
@@ -0,0 +1,194 @@
"""Run the live-LLM turn scenario suite sharded across local processes.
This mirrors the CI ``turn-live`` job
(``.github/workflows/interactive-shell-live.yml``): each shard sets
``TURN_SHARD_TOTAL`` / ``TURN_SHARD_INDEX`` and runs the live pytest selection
``-m live_llm -k "test_live_turn_execution_oracle or test_live_action_planning"``
against ``tests/core/agent/test_turn_scenarios.py``.
The suite is IO-bound (it waits on real LLM API calls), so running all shards
concurrently finishes in roughly one shard's wall time instead of the serial
total. Each shard runs as its own pytest process; per-shard output is streamed
to a log file and the exit codes are aggregated into a final summary.
Usage:
uv run python .github/ci/run_live_turn_shards.py # all 8 shards
uv run python .github/ci/run_live_turn_shards.py --shards 4
uv run python .github/ci/run_live_turn_shards.py --indexes 0,3
uv run python .github/ci/run_live_turn_shards.py -- -x # extra pytest args
"""
from __future__ import annotations
import argparse
import os
import subprocess
import sys
import time
from dataclasses import dataclass
from pathlib import Path
from typing import IO
_TARGET = "tests/core/agent/test_turn_scenarios.py"
_K_EXPR = "test_live_turn_execution_oracle or test_live_action_planning"
_LOG_DIR = Path(".turn-shard-logs")
@dataclass(frozen=True)
class ShardResult:
index: int
exit_code: int
duration_s: float
log_path: Path
def _parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--shards",
type=int,
default=int(os.getenv("TURN_SHARD_TOTAL", "8")),
help="Total number of shards to split the suite into (default: 8).",
)
parser.add_argument(
"--indexes",
type=str,
default="",
help="Comma-separated subset of shard indexes to run (default: all).",
)
parser.add_argument(
"--workers-per-shard",
type=str,
default="auto",
help="pytest-xdist worker count per shard, passed to -n (default: auto).",
)
parser.add_argument(
"--provider",
type=str,
default=os.getenv("LLM_PROVIDER", "openai"),
help="LLM provider for the live run (default: $LLM_PROVIDER or openai).",
)
parser.add_argument(
"pytest_args",
nargs="*",
help="Extra args forwarded to each pytest shard (after a -- separator).",
)
return parser.parse_args(argv)
def _resolve_indexes(shards: int, indexes: str) -> list[int]:
if shards < 1:
raise SystemExit("--shards must be >= 1")
if not indexes.strip():
return list(range(shards))
selected = sorted({int(part) for part in indexes.split(",") if part.strip()})
out_of_range = [i for i in selected if i < 0 or i >= shards]
if out_of_range:
raise SystemExit(f"shard indexes out of range for --shards {shards}: {out_of_range}")
return selected
def _build_command(workers: str, pytest_args: list[str]) -> list[str]:
return [
sys.executable,
"-m",
"pytest",
"-n",
workers,
"-v",
"-m",
"live_llm",
_TARGET,
"-k",
_K_EXPR,
*pytest_args,
]
def _shard_env(*, shard_total: int, shard_index: int, provider: str) -> dict[str, str]:
env = os.environ.copy()
env["TURN_SHARD_TOTAL"] = str(shard_total)
env["TURN_SHARD_INDEX"] = str(shard_index)
env["LLM_PROVIDER"] = provider
env.setdefault("OPENSRE_DISABLE_KEYRING", "1")
env.setdefault("PYTHONUTF8", "1")
return env
def _run_shards(
*, shard_total: int, indexes: list[int], workers: str, provider: str, pytest_args: list[str]
) -> list[ShardResult]:
_LOG_DIR.mkdir(parents=True, exist_ok=True)
command = _build_command(workers, pytest_args)
print(f"Launching {len(indexes)} shard(s) of {shard_total} (provider={provider}):")
print(f" {' '.join(command)}\n")
started: dict[int, tuple[subprocess.Popen[bytes], float, Path, IO[bytes]]] = {}
for shard_index in indexes:
log_path = _LOG_DIR / f"shard-{shard_index}.log"
log_handle: IO[bytes] = log_path.open("wb")
process = subprocess.Popen(
command,
env=_shard_env(shard_total=shard_total, shard_index=shard_index, provider=provider),
stdout=log_handle,
stderr=subprocess.STDOUT,
)
started[shard_index] = (process, time.monotonic(), log_path, log_handle)
print(f" shard {shard_index} -> pid {process.pid}, log {log_path}")
print("\nWaiting for shards to finish...\n")
results: list[ShardResult] = []
pending = set(started)
while pending:
for shard_index in sorted(pending):
process, start_time, log_path, log_handle = started[shard_index]
code = process.poll()
if code is None:
continue
log_handle.close()
duration = time.monotonic() - start_time
status = "PASS" if code == 0 else f"FAIL (exit {code})"
print(f" shard {shard_index} {status} in {duration:.0f}s")
results.append(
ShardResult(
index=shard_index,
exit_code=code,
duration_s=duration,
log_path=log_path,
)
)
pending.discard(shard_index)
if pending:
time.sleep(2.0)
return sorted(results, key=lambda r: r.index)
def _print_summary(results: list[ShardResult]) -> int:
failures = [r for r in results if r.exit_code != 0]
print("\n==================== live turn shard summary ====================")
for result in results:
status = "PASS" if result.exit_code == 0 else f"FAIL (exit {result.exit_code})"
print(f" shard {result.index}: {status} [{result.duration_s:.0f}s] {result.log_path}")
if failures:
print(f"\n{len(failures)} shard(s) failed. Inspect the logs above.")
return 1
print("\nAll shards passed.")
return 0
def main(argv: list[str] | None = None) -> int:
args = _parse_args(sys.argv[1:] if argv is None else argv)
indexes = _resolve_indexes(args.shards, args.indexes)
results = _run_shards(
shard_total=args.shards,
indexes=indexes,
workers=args.workers_per_shard,
provider=args.provider,
pytest_args=list(args.pytest_args),
)
return _print_summary(results)
if __name__ == "__main__":
raise SystemExit(main())
+120
View File
@@ -0,0 +1,120 @@
#!/usr/bin/env python3
"""Run pytest targets relevant to files changed on this branch.
Usage
-----
make test-scope
make test-scope ARGS=--dry-run
python .github/ci/run_test_scope.py [--dry-run] [--base <ref>]
Exit codes mirror pytest: 0 = all pass, non-zero = failure or config error.
"""
from __future__ import annotations
import argparse
import subprocess
import sys
from pathlib import Path
_CI_DIR = Path(__file__).resolve().parent
if str(_CI_DIR) not in sys.path:
sys.path.insert(0, str(_CI_DIR))
from test_scope_rules import classify # noqa: E402
def _commit_ref_exists(ref: str) -> bool:
return (
subprocess.run(
["git", "rev-parse", "--verify", "--quiet", f"{ref}^{{commit}}"],
check=False,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
).returncode
== 0
)
def _resolve_base_ref(base: str) -> str:
"""Prefer branch refs for unqualified bases so same-named tags do not win."""
if "/" in base or base.startswith("refs/"):
return base
for ref in (
f"refs/heads/{base}",
f"refs/remotes/origin/{base}",
f"refs/remotes/upstream/{base}",
):
if _commit_ref_exists(ref):
return ref
return base
def _git_changed_files(base: str) -> list[str]:
resolved_base = _resolve_base_ref(base)
try:
merge_base = subprocess.check_output(
["git", "merge-base", "HEAD", resolved_base],
text=True,
stderr=subprocess.DEVNULL,
).strip()
except subprocess.CalledProcessError:
merge_base = "HEAD~1"
result = subprocess.check_output(
["git", "diff", "--name-only", merge_base],
text=True,
)
return [f.strip() for f in result.splitlines() if f.strip()]
def _run(cmd: list[str], *, dry_run: bool) -> int:
print(f"\n $ {' '.join(cmd)}\n", flush=True)
if dry_run:
return 0
return subprocess.run(cmd, check=False).returncode
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--dry-run", action="store_true", help="Print command without running.")
parser.add_argument(
"--base",
default="main",
help="Ref to diff against (default: main). Falls back to HEAD~1 if unavailable.",
)
args = parser.parse_args(argv)
try:
changed = _git_changed_files(args.base)
except subprocess.CalledProcessError as exc:
print(f"error: could not determine changed files: {exc}", file=sys.stderr)
return 1
if not changed:
print("No changed files detected — nothing to test.")
return 0
print(f"Changed files ({len(changed)}):")
for path in changed:
print(f" {path}")
escalate, targets, areas = classify(changed)
if escalate:
print("\nEscalating to full unit suite (core/shared code or 3+ areas touched).")
return _run(["make", "test-cov"], dry_run=args.dry_run)
if not targets:
print("\nNo test targets matched — running full unit suite as fallback.")
return _run(["make", "test-cov"], dry_run=args.dry_run)
print(f"\nAreas touched: {', '.join(areas)}")
print(f"Running scoped tests: {' '.join(targets)}")
return _run(
[sys.executable, "-m", "pytest", *targets, "-v"],
dry_run=args.dry_run,
)
if __name__ == "__main__":
sys.exit(main())
+608
View File
@@ -0,0 +1,608 @@
"""Path → pytest target mapping for branch-scoped test runs (CI.md §2).
This module is the single source of truth for ``make test-scope``. Edit rules
here only — do not duplicate the mapping table in CI.md.
"""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
# Distinct app areas in one diff that trigger escalation to ``make test-cov``.
ESCALATION_AREA_THRESHOLD = 3
@dataclass(frozen=True, slots=True)
class PathRule:
"""Map changed paths under ``path_prefix`` to pytest targets."""
path_prefix: str
test_targets: tuple[str, ...]
always_escalate: bool = False
# Matched in list order — more specific prefixes must appear before parents.
RULES: tuple[PathRule, ...] = (
# Shared core (always escalate)
PathRule("core/domain/", (), always_escalate=True),
PathRule("core/", ("tests/core/",)),
PathRule("tools/investigation/reporting/", ("tests/delivery/",)),
PathRule("tools/investigation/", (), always_escalate=True),
PathRule("utils/", (), always_escalate=True),
# Specific sub-packages before their parent
PathRule("integrations/llm_cli/", ("tests/integrations/llm_cli/",)),
PathRule("integrations/opensre/", ("tests/integrations/opensre/",)),
PathRule("integrations/hermes/", ("tests/hermes/",)),
PathRule(
"integrations/alertmanager/",
("tests/integrations/alertmanager/", "tests/e2e/alertmanager/"),
),
PathRule(
"integrations/dagster/",
("tests/integrations/dagster/", "tests/synthetic/test_dagster_scenario.py"),
),
PathRule(
"integrations/eks/",
(
"tests/integrations/eks/",
"tests/tools/test_eks_deployment_status_tool.py",
"tests/tools/test_eks_describe_addon_tool.py",
"tests/tools/test_eks_describe_cluster_tool.py",
"tests/tools/test_eks_events_tool.py",
"tests/tools/test_eks_list_clusters_tool.py",
"tests/tools/test_eks_list_deployments_tool.py",
"tests/tools/test_eks_list_namespaces_tool.py",
"tests/tools/test_eks_list_pods_tool.py",
"tests/tools/test_eks_node_health_tool.py",
"tests/tools/test_eks_nodegroup_health_tool.py",
"tests/tools/test_eks_pod_logs_tool.py",
"tests/tools/test_telemetry.py",
"tests/benchmarks/cloudopsbench/tests/test_bench_agent.py",
),
),
PathRule(
"integrations/elasticsearch/",
(
"tests/integrations/elasticsearch/",
"tests/tools/test_elasticsearch_logs_tool.py",
),
),
PathRule(
"integrations/google_docs/",
(
"tests/integrations/google_docs/",
"tests/tools/test_google_docs_create_report_tool.py",
"tests/tools/test_telemetry.py",
),
),
PathRule(
"integrations/groundcover/",
("tests/integrations/groundcover/", "tests/tools/test_groundcover_tools.py"),
),
PathRule(
"integrations/helm/",
("tests/integrations/helm/", "tests/tools/test_helm_tools.py"),
),
PathRule(
"integrations/incident_io/",
("tests/integrations/incident_io/", "tests/tools/test_incident_io_tool.py"),
),
PathRule(
"integrations/jira/",
(
"tests/integrations/jira/",
"tests/tools/test_jira_add_comment_tool.py",
"tests/tools/test_jira_create_issue_tool.py",
"tests/tools/test_jira_issue_detail_tool.py",
"tests/tools/test_jira_search_issues_tool.py",
),
),
PathRule(
"integrations/clickhouse/",
(
"tests/integrations/clickhouse/",
"tests/tools/test_clickhouse_query_activity_tool.py",
"tests/tools/test_clickhouse_system_health_tool.py",
),
),
PathRule(
"integrations/mariadb/",
(
"tests/integrations/mariadb/",
"tests/tools/test_mariadb_innodb_status_tool.py",
"tests/tools/test_mariadb_process_list_tool.py",
"tests/tools/test_mariadb_replication_tool.py",
"tests/tools/test_mariadb_slow_queries_tool.py",
"tests/tools/test_mariadb_status_tool.py",
"tests/e2e/mariadb/",
),
),
PathRule(
"integrations/mongodb_atlas/",
(
"tests/integrations/mongodb_atlas/",
"tests/tools/test_mongodb_atlas_alerts_tool.py",
"tests/tools/test_mongodb_atlas_clusters_tool.py",
"tests/tools/test_mongodb_atlas_events_tool.py",
"tests/tools/test_mongodb_atlas_metrics_tool.py",
"tests/tools/test_mongodb_atlas_performance_advisor_tool.py",
),
),
PathRule(
"integrations/mongodb/",
(
"tests/integrations/mongodb/",
"tests/tools/test_mongodb_collection_stats_tool.py",
"tests/tools/test_mongodb_current_ops_tool.py",
"tests/tools/test_mongodb_profiler_tool.py",
"tests/tools/test_mongodb_replica_status_tool.py",
"tests/tools/test_mongodb_server_status_tool.py",
"tests/e2e/mongodb/",
),
),
PathRule(
"integrations/mysql/",
(
"tests/integrations/mysql/",
"tests/tools/test_mysql_current_processes_tool.py",
"tests/tools/test_mysql_replication_status_tool.py",
"tests/tools/test_mysql_server_status_tool.py",
"tests/tools/test_mysql_slow_queries_tool.py",
"tests/tools/test_mysql_table_stats_tool.py",
"tests/e2e/mysql/",
),
),
PathRule(
"integrations/postgresql/",
(
"tests/integrations/postgresql/",
"tests/tools/test_postgresql_current_queries_tool.py",
"tests/tools/test_postgresql_locks_tool.py",
"tests/tools/test_postgresql_replication_status_tool.py",
"tests/tools/test_postgresql_server_status_tool.py",
"tests/tools/test_postgresql_slow_queries_tool.py",
"tests/tools/test_postgresql_table_stats_tool.py",
"tests/e2e/postgresql/",
),
),
PathRule(
"integrations/redis/",
(
"tests/integrations/redis/",
"tests/tools/test_redis_client_list_tool.py",
"tests/tools/test_redis_key_scan_tool.py",
"tests/tools/test_redis_latency_doctor_tool.py",
"tests/tools/test_redis_list_depth_tool.py",
"tests/tools/test_redis_replication_tool.py",
"tests/tools/test_redis_server_info_tool.py",
"tests/tools/test_redis_slowlog_tool.py",
"tests/e2e/redis/",
),
),
PathRule(
"integrations/snowflake/",
(
"tests/integrations/snowflake/",
"tests/tools/test_snowflake_query_history_tool.py",
"tests/tools/test_telemetry.py",
),
),
PathRule(
"integrations/azure/",
(
"tests/tools/test_azure_monitor_logs_tool.py",
"tests/tools/test_telemetry.py",
),
),
PathRule(
"integrations/azure_sql/",
(
"tests/integrations/test_azure_sql.py",
"tests/tools/test_azure_sql_current_queries_tool.py",
"tests/tools/test_azure_sql_resource_stats_tool.py",
"tests/tools/test_azure_sql_server_status_tool.py",
"tests/tools/test_azure_sql_slow_queries_tool.py",
"tests/tools/test_azure_sql_wait_stats_tool.py",
),
),
PathRule(
"integrations/betterstack/",
(
"tests/integrations/test_betterstack.py",
"tests/tools/test_betterstack_logs_tool.py",
),
),
PathRule(
"integrations/hermes/tools/",
(
"tests/tools/test_hermes_logs_tool.py",
"tests/tools/test_hermes_session_evidence_tool.py",
),
),
PathRule(
"integrations/kafka/",
(
"tests/integrations/test_kafka.py",
"tests/tools/test_kafka_consumer_group_tool.py",
"tests/tools/test_kafka_topic_health_tool.py",
),
),
PathRule(
"integrations/openclaw/",
(
"tests/tools/test_openclaw_mcp_tool.py",
"tests/tools/test_telemetry.py",
),
),
PathRule(
"integrations/openobserve/",
(
"tests/tools/test_openobserve_logs_tool.py",
"tests/tools/test_telemetry.py",
),
),
PathRule(
"integrations/opensearch/",
(
"tests/integrations/test_opensearch_catalog.py",
"tests/tools/test_opensearch_analytics_tool.py",
),
),
PathRule(
"integrations/posthog_mcp/",
(
"tests/integrations/test_posthog_mcp.py",
"tests/tools/test_posthog_mcp_tool.py",
"tests/tools/test_telemetry.py",
),
),
PathRule(
"integrations/rabbitmq/",
(
"tests/integrations/test_rabbitmq.py",
"tests/tools/test_rabbitmq_broker_overview_tool.py",
"tests/tools/test_rabbitmq_connection_stats_tool.py",
"tests/tools/test_rabbitmq_consumer_health_tool.py",
"tests/tools/test_rabbitmq_node_health_tool.py",
"tests/tools/test_rabbitmq_queue_backlog_tool.py",
),
),
PathRule(
"integrations/sentry_mcp/",
(
"tests/integrations/test_sentry_mcp.py",
"tests/tools/test_sentry_mcp_tool.py",
"tests/tools/test_telemetry.py",
),
),
PathRule(
"integrations/sentry/",
(
"tests/tools/test_sentry_issue_details_tool.py",
"tests/tools/test_sentry_issue_events_tool.py",
"tests/tools/test_sentry_search_issues_tool.py",
),
),
PathRule(
"integrations/supabase/",
(
"tests/integrations/test_supabase.py",
"tests/tools/test_supabase_health_tool.py",
"tests/tools/test_supabase_storage_tool.py",
),
),
PathRule(
"integrations/bitbucket/",
(
"tests/integrations/test_bitbucket.py",
"tests/tools/test_bitbucket_commits_tool.py",
"tests/tools/test_bitbucket_file_contents_tool.py",
"tests/tools/test_bitbucket_search_code_tool.py",
),
),
PathRule(
"integrations/telegram/tools/",
("tests/tools/test_telegram_send_message_tool.py",),
),
PathRule(
"integrations/tracer/tools/",
(
"tests/tools/test_tracer_airflow_metrics_tool.py",
"tests/tools/test_tracer_batch_statistics_tool.py",
"tests/tools/test_tracer_error_logs_tool.py",
"tests/tools/test_tracer_failed_jobs_tool.py",
"tests/tools/test_tracer_failed_run_tool.py",
"tests/tools/test_tracer_failed_tools_tool.py",
"tests/tools/test_tracer_host_metrics_tool.py",
"tests/tools/test_tracer_run_tool.py",
"tests/tools/test_tracer_tasks_tool.py",
),
),
PathRule(
"integrations/twilio/",
(
"tests/integrations/test_twilio.py",
"tests/tools/test_twilio_notify_tool.py",
),
),
PathRule(
"integrations/github/tools/",
(
"tests/tools/test_github_actions_tool.py",
"tests/tools/test_github_commits_tool.py",
"tests/tools/test_github_file_contents_tool.py",
"tests/tools/test_github_helpers.py",
"tests/tools/test_github_issues_tool.py",
"tests/tools/test_github_repo_scope.py",
"tests/tools/test_github_repository_tool.py",
"tests/tools/test_github_repository_tree_tool.py",
"tests/tools/test_github_search_code_tool.py",
"tests/tools/test_github_workflow_tools.py",
),
),
PathRule(
"integrations/gitlab/",
(
"tests/integrations/test_gitlab.py",
"tests/tools/test_gitlab_commits_tool.py",
"tests/tools/test_gitlab_file_tool.py",
"tests/tools/test_gitlab_mrs_tool.py",
"tests/tools/test_gitlab_pipelines_tool.py",
"tests/e2e/gitlab/",
),
),
PathRule(
"integrations/aws/tools/",
("tests/tools/test_aws_operation_tool.py",),
),
PathRule(
"integrations/aws_lambda/",
(
"tests/integrations/aws/test_lambda_client.py",
"tests/tools/test_lambda_config_tool.py",
"tests/tools/test_lambda_errors_tool.py",
"tests/tools/test_lambda_inspect_tool.py",
"tests/tools/test_lambda_invocation_logs_tool.py",
),
),
PathRule(
"integrations/cloudtrail/",
("tests/tools/test_cloudtrail_events.py",),
),
PathRule(
"integrations/cloudwatch/",
(
"tests/integrations/aws/test_cloudwatch_client.py",
"tests/tools/test_cloudwatch_batch_metrics_tool.py",
"tests/tools/test_cloudwatch_logs_tool.py",
),
),
PathRule(
"integrations/ec2/",
("tests/tools/test_ec2_instances_by_tag_tool.py",),
),
PathRule(
"integrations/elb/",
("tests/tools/test_elb_target_health_tool.py",),
),
PathRule(
"integrations/rds/",
(
"tests/integrations/test_rds.py",
"tests/tools/test_rds_tools.py",
),
),
PathRule(
"integrations/s3/",
(
"tests/integrations/aws/test_s3_client.py",
"tests/tools/test_s3_get_object_tool.py",
"tests/tools/test_s3_inspect_tool.py",
"tests/tools/test_s3_list_tool.py",
"tests/tools/test_s3_marker_tool.py",
),
),
PathRule(
"integrations/opsgenie/",
(
"tests/integrations/opsgenie/",
"tests/tools/test_opsgenie_alert_detail_tool.py",
"tests/tools/test_opsgenie_alerts_tool.py",
),
),
PathRule(
"integrations/pagerduty/",
(
"tests/integrations/pagerduty/",
"tests/tools/test_pagerduty_incident_detail_tool.py",
"tests/tools/test_pagerduty_incidents_tool.py",
"tests/tools/test_pagerduty_oncall_tool.py",
"tests/tools/test_pagerduty_services_tool.py",
),
),
PathRule(
"integrations/prefect/",
(
"tests/integrations/prefect/",
"tests/tools/test_prefect_flow_runs_tool.py",
"tests/tools/test_prefect_worker_health_tool.py",
),
),
PathRule(
"integrations/signoz/",
(
"tests/integrations/signoz/",
"tests/tools/test_signoz_tools.py",
"tests/synthetic/test_signoz_scenario.py",
),
),
PathRule(
"integrations/splunk/",
("tests/integrations/splunk/", "tests/tools/test_splunk_search_tool.py"),
),
PathRule(
"integrations/tempo/",
(
"tests/integrations/tempo/",
"tests/tools/test_tempo_tools.py",
"tests/synthetic/test_tempo_scenario.py",
),
),
PathRule(
"integrations/temporal/",
(
"tests/integrations/temporal/",
"tests/integrations/test_temporal_catalog.py",
"tests/synthetic/test_temporal_scenario.py",
"tests/tools/test_temporal_namespace_info_tool.py",
"tests/tools/test_temporal_task_queue_tool.py",
"tests/tools/test_temporal_workflow_history_tool.py",
"tests/tools/test_temporal_workflows_tool.py",
),
),
PathRule(
"integrations/vercel/",
(
"tests/integrations/vercel/",
"tests/tools/test_vercel_deployment_status_tool.py",
"tests/tools/test_vercel_logs_tool.py",
),
),
PathRule(
"integrations/victoria_logs/",
(
"tests/integrations/victoria_logs/",
"tests/tools/test_victoria_logs_tool.py",
"tests/e2e/victoria_logs/",
),
),
PathRule(
"integrations/x_mcp/",
(
"tests/integrations/test_x_mcp.py",
"tests/tools/test_x_mcp_tool.py",
),
),
PathRule(
"integrations/argocd/",
(
"tests/integrations/argocd/",
"tests/tools/test_argocd_tools.py",
),
),
PathRule(
"integrations/coralogix/",
(
"tests/integrations/coralogix/",
"tests/tools/test_coralogix_logs_tool.py",
),
),
PathRule(
"integrations/honeycomb/",
(
"tests/integrations/honeycomb/",
"tests/tools/test_honeycomb_traces_tool.py",
),
),
PathRule(
"integrations/jenkins/",
("tests/integrations/test_jenkins.py", "tests/synthetic/test_jenkins_scenario.py"),
),
PathRule(
"integrations/datadog/",
(
"tests/integrations/datadog/",
"tests/tools/test_datadog_context_tool.py",
"tests/tools/test_datadog_events_tool.py",
"tests/tools/test_datadog_logs_tool.py",
"tests/tools/test_datadog_metrics_tool.py",
"tests/tools/test_datadog_monitors_tool.py",
"tests/tools/test_datadog_node_pods_tool.py",
),
),
PathRule(
"integrations/grafana/",
(
"tests/integrations/grafana/",
"tests/tools/test_grafana_alert_rules_tool.py",
"tests/tools/test_grafana_annotations_tool.py",
"tests/tools/test_grafana_logs_tool.py",
"tests/tools/test_grafana_metrics_tool.py",
"tests/tools/test_grafana_service_names_tool.py",
"tests/tools/test_grafana_traces_tool.py",
"tests/e2e/grafana_validation/",
),
),
PathRule("integrations/", ("tests/integrations/",)),
PathRule("tools/system/fleet_monitoring/", ("tests/agent/", "tests/fleet_monitoring/")),
PathRule("surfaces/cli/", ("tests/cli/",)),
PathRule("surfaces/interactive_shell/", ("tests/interactive_shell/",)),
PathRule("gateway/", ("gateway/tests/",)),
PathRule("tools/system/watch_dog/", ("tests/watch_dog/",)),
PathRule("tools/", ("tests/tools/",)),
PathRule("platform/analytics/", ("tests/analytics/",)),
PathRule("platform/guardrails/", ("tests/platform/guardrails/",)),
PathRule("platform/masking/", ("tests/masking/",)),
PathRule("platform/packaging/", ("tests/packaging/",)),
PathRule("platform/sandbox/", ("tests/sandbox/",)),
PathRule(
"platform/deployment/",
("tests/deployment/", "tests/platform/deployment/test_deployment_health.py"),
),
PathRule("platform/auth/", ("tests/platform/auth/",)),
PathRule("gateway/webapp.py", ("gateway/tests/test_webapp.py",)),
# Repo-wide config
PathRule("pyproject.toml", (), always_escalate=True),
PathRule("uv.lock", (), always_escalate=True),
PathRule("pytest.ini", (), always_escalate=True),
PathRule("Makefile", (), always_escalate=True),
PathRule(".github/ci/", ("tests/github_ci/",)),
)
def _matches(path: str, prefix: str) -> bool:
return path.startswith(prefix) or path == prefix.rstrip("/")
def _area_key(prefix: str) -> str:
parts = prefix.split("/")
if parts[0] == "deployment" or parts[:2] == ["platform", "deployment"]:
return "deployment"
return prefix
def classify(changed: list[str]) -> tuple[bool, list[str], list[str]]:
"""Return ``(should_escalate, test_targets, matched_areas)``."""
escalate = False
targets: list[str] = []
areas: list[str] = []
for path in changed:
matched = False
for rule in RULES:
if not _matches(path, rule.path_prefix):
continue
matched = True
if rule.always_escalate:
escalate = True
else:
area = _area_key(rule.path_prefix)
if area not in areas:
areas.append(area)
for target in rule.test_targets:
if target not in targets:
targets.append(target)
break
if not matched and path.startswith("tests/") and path not in targets:
targets.append(path)
if len(areas) >= ESCALATION_AREA_THRESHOLD:
escalate = True
existing = [t for t in targets if Path(t).exists()]
dropped = [t for t in targets if t not in existing]
if dropped:
print(f" (skipping non-existent targets: {', '.join(dropped)})", flush=True)
return escalate, existing, areas
+7
View File
@@ -0,0 +1,7 @@
name: "CodeQL config"
paths-ignore:
# Vendored third-party pipeline code in test fixtures — not owned by this project.
# Covers every tests/e2e/<case>/pipeline_code/ fixture (lambda, flink, prefect,
# datadog, kubernetes, …) so new cases are ignored automatically.
- tests/e2e/**/pipeline_code/**
+15
View File
@@ -0,0 +1,15 @@
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
version: 2
updates:
- package-ecosystem: "uv"
directory: "/"
schedule:
interval: "weekly"
open-pull-requests-limit: 10
- package-ecosystem: "npm"
directory: "/docs"
schedule:
interval: "daily"
open-pull-requests-limit: 10
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env python3
"""Build the Discord release announcement JSON payload.
Reads from environment variables set by the GitHub Actions step:
RELEASE_TAG, RELEASE_URL,
DISCORD_RELEASES_ROLE_ID, DISCORD_RELEASE_LOGO_EMOJI, DISCORD_RELEASE_LOGO_URL,
CHANGELOG_FILE — path to DISCORD_NARRATIVE.md or GENERATED_CHANGELOG.md
"""
from __future__ import annotations
import json
import os
import sys
MAX_CHANGELOG_CHARS = 1400
tag = os.environ["RELEASE_TAG"]
url = os.environ["RELEASE_URL"]
role_id = os.environ.get("DISCORD_RELEASES_ROLE_ID", "")
logo_emoji = os.environ.get("DISCORD_RELEASE_LOGO_EMOJI", "")
logo_url = os.environ.get("DISCORD_RELEASE_LOGO_URL", "")
changelog_file = os.environ.get("CHANGELOG_FILE", "")
if changelog_file and os.path.isfile(changelog_file):
with open(changelog_file) as f:
changelog = f.read().replace("\r", "").strip()
else:
changelog = "No changelog available."
if len(changelog) > MAX_CHANGELOG_CHARS:
changelog = changelog[: MAX_CHANGELOG_CHARS - 3] + "..."
mention = f"<@&{role_id}>\n" if role_id else ""
logo_prefix = f"{logo_emoji} " if logo_emoji else ""
bt = "`"
content = (
mention + logo_prefix + f"🚀 **opensre {bt}{tag}{bt} is live**\n" + f"🔗 {url}\n\n" + changelog
)
allowed_mentions: dict = {"parse": []}
if role_id:
allowed_mentions["roles"] = [role_id]
payload: dict = {"content": content, "allowed_mentions": allowed_mentions}
if logo_url:
payload["username"] = "OpenSRE"
payload["avatar_url"] = logo_url
json.dump(payload, sys.stdout)
+223
View File
@@ -0,0 +1,223 @@
"""Assign and notify **new** contributors on `good first issue` threads.
Here *new* means the commenter has **no merged PRs and no open PRs** in this repo where
they are the PR author (GitHub Search API). **One or more merged PRs** means they are not
treated as a first-time contributor for this automation (GitHub's ``FIRST_TIME_CONTRIBUTOR``
/ ``FIRST_TIMER`` flags are not used).
Also skips repo insiders (OWNER / MEMBER / COLLABORATOR), bots, closed issues,
comments on **pull request** threads (``issue_comment`` fires for PRs too; those
use ``issue.pull_request``), and commenters already listed as assignees.
**One open assignment per eligible new contributor** in this repo: if they already
have another **open** issue assigned (Search API), they cannot be auto-assigned here.
At most **one** auto-assignment per issue: if anyone else is already an assignee,
further eligible commenters are skipped (manual pre-assignments count).
"""
from __future__ import annotations
import json
import os
import sys
import urllib.error
import urllib.parse
import urllib.request
from pathlib import Path
from typing import Any
GOOD_FIRST_LABEL = "good first issue"
# Do not auto-assign maintainers/collaborators;
# eligibility is 0 merged + 0 open PRs as author + not insider.
EXCLUDED_COMMENTER_ASSOCIATIONS = frozenset({"OWNER", "MEMBER", "COLLABORATOR"})
GITHUB_API = "https://api.github.com"
def _github_api_url(path: str, query: dict[str, str]) -> str:
encoded = urllib.parse.urlencode(query)
return f"{GITHUB_API}{path}?{encoded}"
def screen_event_without_api(event: dict[str, Any]) -> str | None:
"""Return a skip reason before calling the GitHub API, or None if checks should continue."""
issue = event.get("issue") or {}
comment = event.get("comment") or {}
if issue.get("pull_request") is not None:
return "comment_on_pull_request"
if issue.get("state") != "open":
return "issue_not_open"
labels = issue.get("labels") or []
if not isinstance(labels, list):
return "invalid_labels"
names = {item.get("name") for item in labels if isinstance(item, dict)}
if GOOD_FIRST_LABEL not in names:
return "not_good_first_issue"
c_user = comment.get("user") or {}
if c_user.get("type") == "Bot":
return "bot_commenter"
c_login = c_user.get("login") or ""
if not c_login:
return "missing_commenter_login"
c_assoc = comment.get("author_association") or ""
if c_assoc in EXCLUDED_COMMENTER_ASSOCIATIONS:
return "commenter_repo_insider"
assignees = issue.get("assignees") or []
if isinstance(assignees, list):
assigned_logins = {
a.get("login") for a in assignees if isinstance(a, dict) and a.get("login")
}
if c_login in assigned_logins:
return "already_assignee"
if assigned_logins:
return "issue_already_claimed"
return None
def assign_decision(
*,
skip_reason_pre_api: str | None,
merged_pr_count_for_commenter: int,
open_pr_count_for_commenter: int,
open_assigned_issue_count_for_commenter: int,
) -> tuple[bool, str]:
"""Return (should_assign_and_comment, skip_reason_or_empty).
Eligible "new contributor" means ``merged_pr_count_for_commenter == 0`` and
``open_pr_count_for_commenter == 0`` (PR author in this repo, via Search API).
They must also have no other open issues assigned in this repo
(``open_assigned_issue_count_for_commenter == 0``).
"""
if skip_reason_pre_api is not None:
return False, skip_reason_pre_api
if open_pr_count_for_commenter > 0:
return False, "has_open_prs"
if merged_pr_count_for_commenter > 0:
return False, "has_merged_prs"
if open_assigned_issue_count_for_commenter > 0:
return False, "already_has_open_assigned_issue"
return True, ""
def _request_json(url: str, token: str) -> Any:
parsed = urllib.parse.urlparse(url)
if parsed.scheme != "https" or parsed.netloc != "api.github.com":
raise ValueError("GitHub API URL must target https://api.github.com")
req = urllib.request.Request(
url,
headers={
"Authorization": f"Bearer {token}",
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
},
method="GET",
)
with (
urllib.request.urlopen(req, timeout=60) as resp
): # nosemgrep: python.lang.security.audit.dynamic-urllib-use-detected.dynamic-urllib-use-detected
return json.loads(resp.read().decode("utf-8"))
def _search_issue_total_count(query: str, token: str) -> int:
url = _github_api_url("/search/issues", {"q": query})
try:
data = _request_json(url, token)
except urllib.error.URLError as exc:
print(f"GitHub search failed: {exc}", file=sys.stderr)
raise
total = data.get("total_count")
if not isinstance(total, int):
return 0
return total
def fetch_merged_pr_count(owner: str, repo: str, login: str, token: str) -> int:
q = f"repo:{owner}/{repo} is:pr is:merged author:{login}"
return _search_issue_total_count(q, token)
def fetch_open_pr_count(owner: str, repo: str, login: str, token: str) -> int:
q = f"repo:{owner}/{repo} is:pr is:open author:{login}"
return _search_issue_total_count(q, token)
def fetch_open_assigned_issue_count(owner: str, repo: str, login: str, token: str) -> int:
"""Count open issues in this repo where ``login`` is an assignee."""
q = f"repo:{owner}/{repo} is:issue is:open assignee:{login}"
return _search_issue_total_count(q, token)
def build_assign_notice_body(*, assignee_login: str) -> str:
return f"@{assignee_login} You've been **assigned** to this issue. Thanks for picking it up."
def set_github_output(name: str, value: str) -> None:
path = os.environ.get("GITHUB_OUTPUT")
if not path:
return
with open(path, "a", encoding="utf-8") as fh:
fh.write(f"{name}={value}\n")
def main() -> int:
event_path = os.environ.get("GITHUB_EVENT_PATH")
repository = os.environ.get("GITHUB_REPOSITORY")
token = os.environ.get("GITHUB_TOKEN")
if not event_path or not repository or not token:
print("Missing GITHUB_EVENT_PATH, GITHUB_REPOSITORY, or GITHUB_TOKEN.", file=sys.stderr)
return 1
raw = Path(event_path).read_text(encoding="utf-8")
event = json.loads(raw)
pre = screen_event_without_api(event)
merged_count = 0
open_count = 0
open_assigned_issue_count = 0
if pre is None:
owner, _, repo = repository.partition("/")
if not owner or not repo:
print("Invalid GITHUB_REPOSITORY.", file=sys.stderr)
return 1
comment = event.get("comment") or {}
c_login = (comment.get("user") or {}).get("login") or ""
try:
merged_count = fetch_merged_pr_count(owner, repo, c_login, token)
open_count = fetch_open_pr_count(owner, repo, c_login, token)
open_assigned_issue_count = fetch_open_assigned_issue_count(owner, repo, c_login, token)
except urllib.error.URLError as exc:
print(f"GitHub API request failed: {exc}", file=sys.stderr)
return 1
should, reason = assign_decision(
skip_reason_pre_api=pre,
merged_pr_count_for_commenter=merged_count,
open_pr_count_for_commenter=open_count,
open_assigned_issue_count_for_commenter=open_assigned_issue_count,
)
if not should:
print(f"Skip: {reason}")
set_github_output("should_assign", "false")
return 0
comment_user = (event.get("comment") or {}).get("user") or {}
login = comment_user.get("login") if isinstance(comment_user, dict) else ""
if not isinstance(login, str) or not login:
print("Missing commenter login.", file=sys.stderr)
return 1
body = build_assign_notice_body(assignee_login=login)
Path("assign_comment.md").write_text(body, encoding="utf-8")
set_github_output("should_assign", "true")
print("Wrote assign_comment.md; assignment will be applied in workflow.")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,124 @@
"""Write celebrate-merge PR comment body to comment.md (run from Actions after merge)."""
from __future__ import annotations
import os
import random
discord = os.environ["DISCORD_INVITE_URL"]
contributor = os.environ["CONTRIBUTOR_LOGIN"]
templates: list[str] = [
(f"🎉 **MERGED!** @{contributor} just shipped something. The diff gods are pleased. 🙌"),
(
f"🚀 **Houston, we have a merge.** @{contributor} your PR is in orbit. "
"Thanks for launching this one!"
),
(
f"💜 **One more reason the project grows.** Thanks @{contributor}"
"your contribution just landed!"
),
(
f"🎊 **Achievement unlocked: PR Merged.** @{contributor} passed code review, "
"survived CI, and shipped. Respect. 🤝"
),
(
f'🔥 **Another one.** @{contributor} said "here\'s a PR" and maintainers said '
"\"ship it\". That's how it's done."
),
(
f"🧑‍💻 **@{contributor} has entered the contributor hall of fame.** "
"Merged. Done. Shipped. Go touch grass (then come back with another PR). 🌱"
),
(
f"🎯 **Bullseye.** @{contributor} opened a PR, kept the vibes clean, "
"and got it merged. Absolute cinema. 🎬"
),
(
f"⚡ **LGTM → Merged.** @{contributor}, your work is in. "
"Every commit counts — thank you for this one."
),
# new additions
(
f'😤 **@{contributor} said "I will fix this" and then actually fixed it.** '
"Legendary behavior."
),
(
f"🍕 **@{contributor}'s PR:** crispy edges, no unnecessary toppings, delivered on time. "
"Understood the assignment. 🔥"
),
(f"🌊 **Merged.** @{contributor} is now permanently woven into git history. No take-backs. 😄"),
(
f"🤖 **CI passed. Linter didn't scream. Reviewer typed LGTM.** "
f"@{contributor}, every machine in this pipeline just slow-clapped. 🖥️✨"
),
(f"🧠 **@{contributor} opened a PR.** Maintainers feared them. CI genuflected. It merged. 🚨"),
(
f"😭 **Clear commit message. Green tests. Kind review.** "
f"@{contributor}, stop making the rest of us look bad."
),
(
f"🐸 **Rebase? Handled. Conflicts? Squashed. CI? Vibing.** "
f"@{contributor} touched the untouchable and lived. 🫡"
),
(
f"🏆 **@{contributor} did not come to play.** "
"PR opened. Review survived. Merged clean. Retire the jersey. 🎽"
),
(
f"🎲 **Researchers are baffled.** @{contributor} opened a PR, got it reviewed without drama, "
"and merged clean. This violates known laws of open source. 🔬"
),
(
f"🌮 **@{contributor}'s PR:** showed up unannounced, improved everything, left zero bugs. "
"Just like a perfect taco. 🌮"
),
(
f"🐉 **Legend says** enough merged PRs and you ascend. "
f"@{contributor} is dangerously close. 🌤️"
),
(
f"🛸 **Aliens watching our repo** just upgraded @{contributor}'s threat level to: "
"*do not engage — too competent*. 👽"
),
(
f'🎻 **"The diff was clean, the tests did pass, the reviewer wept."** '
f"That poem was about @{contributor}'s PR. 🥹"
),
(f"🍵 **@{contributor} made tea, opened a PR, and merged before it cooled.** No notes. ☕"),
(
f"🏄 **Some PRs rot in review for six weeks.** "
f'@{contributor}\'s said "not today" and merged like it owned the place. 🌊'
),
(
f"💼 **Interviewer:** describe a time you shipped something impactful.\n\n"
f"**@{contributor}:** *points at this PR*\n\n"
"**Interviewer:** you're hired. 🤝"
),
]
# GIFs are repo-hosted under .github/assets/celebrations/ so GitHub's own CDN serves them.
_base = "https://raw.githubusercontent.com/Tracer-Cloud/opensre/main/.github/assets/celebrations"
gif_blocks: list[str] = [
f"\n\n![]({_base}/party.gif)",
f"\n\n![]({_base}/celebrate.gif)",
f"\n\n![]({_base}/ship.gif)",
f"\n\n![]({_base}/shipped.gif)",
f"\n\n![]({_base}/fireworks.gif)",
f"\n\n![]({_base}/woohoo.gif)",
f"\n\n![]({_base}/office-celebrate.gif)",
f"\n\n![]({_base}/merge-celebrate-1.gif)",
f"\n\n![]({_base}/merge-celebrate-2.gif)",
f"\n\n![]({_base}/merge-celebrate-3.gif)",
]
head = random.choice(templates) + random.choice(gif_blocks)
footer = (
"---\n\n"
f"👋 **Join us on [Discord - OpenSRE]({discord})** : hang out, contribute, "
"or hunt for features and issues. Everyone's welcome."
)
body = f"{head}\n\n{footer}"
with open("comment.md", "w", encoding="utf-8") as fh:
fh.write(body)
+95
View File
@@ -0,0 +1,95 @@
#!/usr/bin/env bash
# Sync Tracer-Cloud/homebrew-tap Formula/opensre.rb version and archive SHAs.
#
# Usage (CI):
# HOMEBREW_TAP_GITHUB_TOKEN=... VERSION=2026.5.29 ASSET_DIR=release-assets ./sync-homebrew-tap-formula.sh
#
# Local dry-run (no push):
# DRY_RUN=1 VERSION=2026.5.29 ASSET_DIR=/path/to/sha256-files ./sync-homebrew-tap-formula.sh
set -euo pipefail
VERSION="${VERSION:-}"
ASSET_DIR="${ASSET_DIR:-release-assets}"
DRY_RUN="${DRY_RUN:-0}"
if [ -z "$VERSION" ]; then
echo "VERSION is required (e.g. 2026.5.29)" >&2
exit 1
fi
for platform in linux-x64 linux-arm64 darwin-x64 darwin-arm64; do
sha_file="${ASSET_DIR}/opensre_${VERSION}_${platform}.tar.gz.sha256"
if [ ! -f "$sha_file" ]; then
echo "Missing SHA256 file: $sha_file" >&2
exit 1
fi
done
linux_x64_sha="$(awk '{print $1}' "${ASSET_DIR}/opensre_${VERSION}_linux-x64.tar.gz.sha256")"
linux_arm64_sha="$(awk '{print $1}' "${ASSET_DIR}/opensre_${VERSION}_linux-arm64.tar.gz.sha256")"
darwin_x64_sha="$(awk '{print $1}' "${ASSET_DIR}/opensre_${VERSION}_darwin-x64.tar.gz.sha256")"
darwin_arm64_sha="$(awk '{print $1}' "${ASSET_DIR}/opensre_${VERSION}_darwin-arm64.tar.gz.sha256")"
tap_dir="$(mktemp -d)/homebrew-tap"
if [ -n "${HOMEBREW_TAP_GITHUB_TOKEN:-}" ]; then
git clone "https://x-access-token:${HOMEBREW_TAP_GITHUB_TOKEN}@github.com/Tracer-Cloud/homebrew-tap.git" "$tap_dir"
else
git clone "https://github.com/Tracer-Cloud/homebrew-tap.git" "$tap_dir"
fi
python3 - "$tap_dir/Formula/opensre.rb" "$VERSION" "$darwin_arm64_sha" "$darwin_x64_sha" "$linux_arm64_sha" "$linux_x64_sha" <<'PY'
from __future__ import annotations
import re
import sys
from pathlib import Path
formula_path = Path(sys.argv[1])
version = sys.argv[2]
darwin_arm64_sha = sys.argv[3]
darwin_x64_sha = sys.argv[4]
linux_arm64_sha = sys.argv[5]
linux_x64_sha = sys.argv[6]
text = formula_path.read_text()
text = re.sub(r'version "[^"]+"', f'version "{version}"', text, count=1)
replacements = {
r'(opensre_#\{version\}_darwin-arm64\.tar\.gz"\n\s*sha256 ")[^"]+(")': darwin_arm64_sha,
r'(opensre_#\{version\}_darwin-x64\.tar\.gz"\n\s*sha256 ")[^"]+(")': darwin_x64_sha,
r'(opensre_#\{version\}_linux-arm64\.tar\.gz"\n\s*sha256 ")[^"]+(")': linux_arm64_sha,
r'(opensre_#\{version\}_linux-x64\.tar\.gz"\n\s*sha256 ")[^"]+(")': linux_x64_sha,
}
for pattern, sha in replacements.items():
text, count = re.subn(pattern, rf'\g<1>{sha}\g<2>', text, count=1)
if count != 1:
raise SystemExit(f"Failed to update checksum with pattern: {pattern}")
formula_path.write_text(text)
PY
cd "$tap_dir"
if git diff --quiet -- Formula/opensre.rb; then
echo "Homebrew tap formula already up to date for version ${VERSION}."
exit 0
fi
echo "=== Formula diff ==="
git diff Formula/opensre.rb
echo "===================="
if [ "$DRY_RUN" = "1" ]; then
echo "DRY_RUN=1 — not committing or pushing."
exit 0
fi
if [ -z "${HOMEBREW_TAP_GITHUB_TOKEN:-}" ]; then
echo "HOMEBREW_TAP_GITHUB_TOKEN is required to push." >&2
exit 1
fi
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add Formula/opensre.rb
git commit -m "chore: update opensre formula to ${VERSION}"
git push origin HEAD:main
echo "Pushed homebrew-tap update for version ${VERSION}."
+17
View File
@@ -0,0 +1,17 @@
# GitHub Actions (maintainer reference)
Internal notes for repository automation under `.github/workflows/`. Not published on the docs site.
## Workflows
| Workflow | Purpose |
| -------- | ------- |
| [`ci.yml`](ci.yml) | PR/push quality gates and sharded pytest |
| [`ci-labels-windows.yml`](ci-labels-windows.yml) | Optional Windows CI (`ci:windows` label) |
| [`codeql.yml`](codeql.yml) | CodeQL security analysis |
| [`greptile-pr-reminder.yml`](greptile-pr-reminder.yml) | Greptile review nudge on PR open |
| [`celebrate-merged-pr.yml`](celebrate-merged-pr.yml) | Post-merge celebration comment |
| [`good-first-issue-assign.yml`](good-first-issue-assign.yml) | Auto-assign good first issues |
| [`release.yml`](release.yml) | Release builds and artifacts |
See [CI.md](../../CI.md) for local parity commands before push.
+176
View File
@@ -0,0 +1,176 @@
name: Benchmark image — build + push to ECR (any adapter)
# Adapter-agnostic image build. The bench image carries the full
# ``tests/benchmarks/`` tree, so every registered adapter ships in the
# same image. ``benchmark-run.yml`` selects which adapter actually runs
# via its ``config`` input. See ``tests/benchmarks/_framework/registry.py``
# for the registration contract.
# Builds tests/benchmarks/cloudopsbench/infra/Dockerfile.bench and pushes the resulting image to the
# opensre-bench ECR repository. The bench container is what
# `Benchmark run (manual)` invokes on AWS Fargate, so this workflow's
# output is the input to that one.
#
# Triggered automatically on changes to the bench code (or the Dockerfile
# itself) and manually via workflow_dispatch.
#
# After the image is pushed, update the task definition by re-applying the
# Terraform with the new tag:
#
# cd tests/benchmarks/cloudopsbench/infra
# terraform apply -var="image_tag=<TAG>"
#
# (Or wire a follow-up step into this workflow to update task definition
# automatically — out of scope for v1.)
#
# Tag format: short git SHA (`git rev-parse --short HEAD`). Stable, unique
# per commit, recognizable in `aws ecr describe-images` output. ECR is
# IMMUTABLE — a tag pushed once cannot be overwritten, so re-running the
# workflow on the same commit just re-tags (no-op layer push).
on:
workflow_dispatch:
inputs:
tag:
description: Image tag to push (default = short git SHA)
required: false
type: string
push:
branches:
- main
paths:
- "tests/benchmarks/cloudopsbench/infra/Dockerfile.bench"
- "tests/benchmarks/cloudopsbench/infra/Dockerfile.bench.dockerignore"
- "pyproject.toml"
- "uv.lock"
- "surfaces/**"
- "config/**"
- "core/**"
- "platform/deployment/**"
- "integrations/**"
- "platform/**"
- "tools/**"
- "tests/benchmarks/**"
- ".github/workflows/benchmark-image.yml"
permissions:
contents: read
id-token: write # required for AWS OIDC role assumption
concurrency:
# Allow one in-flight build per ref so two pushes in quick succession
# don't race ECR. The newer push cancels the older.
group: bench-image-${{ github.ref }}
cancel-in-progress: true
env:
AWS_REGION: us-east-1
# Account ID is repo-level configuration, not a secret. Set as a GitHub
# repository Variable (Settings > Secrets and variables > Actions >
# Variables) so it doesn't need to be edited in every workflow file
# when the bench moves accounts.
ACCOUNT_ID: ${{ vars.AWS_ACCOUNT_ID }}
ECR_REPOSITORY: opensre-bench
jobs:
build-and-push:
if: github.repository == 'Tracer-Cloud/opensre'
name: build + push
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
with:
# Fetch enough history that `git rev-parse --short HEAD` is stable
fetch-depth: 1
- name: Resolve image tag
id: tag
env:
# Route `inputs.tag` through env so the shell treats it as DATA,
# not code. Without this, a workflow_dispatch caller could inject
# shell commands via a crafted `tag` value containing $() or
# backticks. See:
# https://securitylab.github.com/research/github-actions-untrusted-input/
INPUT_TAG: ${{ inputs.tag }}
run: |
if [ -n "$INPUT_TAG" ]; then
TAG="$INPUT_TAG"
else
TAG="$(git rev-parse --short HEAD)"
fi
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
echo "Will push: $ECR_REPOSITORY:$TAG"
- name: Configure AWS credentials (OIDC role assumption)
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/opensre-bench-github-actions
role-session-name: github-actions-bench-image-${{ github.run_id }}
aws-region: us-east-1
- name: Login to Amazon ECR
id: ecr-login
uses: aws-actions/amazon-ecr-login@v2
- name: Set up Docker Buildx
# Buildx adds multi-platform support + better cache control. Even
# for single-platform builds, it's the modern default.
uses: docker/setup-buildx-action@v3
- name: Build and push
id: build
uses: docker/build-push-action@v6
with:
context: .
file: tests/benchmarks/cloudopsbench/infra/Dockerfile.bench
platforms: linux/amd64
push: true
tags: |
${{ steps.ecr-login.outputs.registry }}/${{ env.ECR_REPOSITORY }}:${{ steps.tag.outputs.tag }}
# Stamp the COMMIT SHA into the image so the runtime can read it
# via the OPENSRE_SHA env var. We use github.sha (the full 40-char
# commit being built), NOT steps.tag.outputs.tag (which resolves
# to the user-supplied inputs.tag like ``hotfix-june`` on
# workflow_dispatch and would stamp an unverifiable string as if
# it were a SHA). The image tag (for ECR naming) and OPENSRE_SHA
# (for provenance) are intentionally decoupled — the tag is for
# humans/operators; the SHA is for reproducibility. The runtime
# gate also validates SHA shape (7-40 lowercase hex chars), so
# even a manually-built image with a bad OPENSRE_SHA fails loudly.
build-args: |
OPENSRE_SHA=${{ github.sha }}
# GitHub Actions cache for Docker layers - speeds up rebuilds
# when only the source changes (deps stay in the cached layer).
cache-from: type=gha
cache-to: type=gha,mode=max
provenance: false # smaller manifest; matches ECR's tolerance for OCI
- name: Summarise
# Route step outputs through env so the shell sees them as DATA,
# not code. `steps.tag.outputs.tag` is derived verbatim from the
# `inputs.tag` user input — without env scoping, a workflow_dispatch
# caller could inject shell commands here even though the
# "Resolve image tag" step is hardened. Same risk applies to any
# tag-derived chain.
env:
REGISTRY: ${{ steps.ecr-login.outputs.registry }}
TAG: ${{ steps.tag.outputs.tag }}
DIGEST: ${{ steps.build.outputs.digest }}
run: |
echo "## Image pushed" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "| field | value |" >> "$GITHUB_STEP_SUMMARY"
echo "| --- | --- |" >> "$GITHUB_STEP_SUMMARY"
echo "| Registry | \`$REGISTRY\` |" >> "$GITHUB_STEP_SUMMARY"
echo "| Repository | \`$ECR_REPOSITORY\` |" >> "$GITHUB_STEP_SUMMARY"
echo "| Tag | \`$TAG\` |" >> "$GITHUB_STEP_SUMMARY"
echo "| Digest | \`$DIGEST\` |" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "To deploy this image:" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "\`\`\`bash" >> "$GITHUB_STEP_SUMMARY"
echo "cd tests/benchmarks/cloudopsbench/infra" >> "$GITHUB_STEP_SUMMARY"
echo "terraform apply -var=\"image_tag=$TAG\"" >> "$GITHUB_STEP_SUMMARY"
echo "\`\`\`" >> "$GITHUB_STEP_SUMMARY"
@@ -0,0 +1,135 @@
name: Benchmark image — promote tag to task definition (any adapter)
# Adapter-agnostic image promotion. Rebinds the ECS task definition to
# a specific image tag. The image carries every registered adapter; the
# config supplied to ``benchmark-run.yml`` picks which one runs.
#
# Manually-triggered workflow that runs `terraform apply -var=image_tag=<TAG>`
# in tests/benchmarks/cloudopsbench/infra/ to register a new ECS task definition revision pointing at
# the chosen ECR image. This is the privileged "deploy" step that comes
# between an image push (automatic) and a bench run (manual).
#
# Why not auto-promote on every image push? An image build is a code-change
# event. A task-def update is a deploy event. Decoupling them lets you
# stage many images and choose deliberately which one production runs.
#
# Trigger from the GitHub UI:
# Actions → "Benchmark image — promote tag to task definition" → Run
#
# Pre-reqs (one-time):
# - tests/benchmarks/cloudopsbench/infra/ Terraform applied at least once. The opensre-bench-github-actions
# OIDC role's permissions (ecs:RegisterTaskDefinition, state-bucket read/write,
# lock-table read/write, iam:PassRole) are granted by the github_actions_run_bench
# inline policy in tests/benchmarks/cloudopsbench/infra/iam_oidc.tf — any apply of that module attaches them.
# - Repo secrets seeded into AWS Secrets Manager
# - Repo vars set (AWS_ACCOUNT_ID etc., see tests/benchmarks/cloudopsbench/infra/AWS_BENCH_SETUP.md step 4)
on:
workflow_dispatch:
inputs:
image_tag:
description: 'ECR image tag to promote (e.g. 3792493)'
required: true
permissions:
contents: read
id-token: write # required for AWS OIDC role assumption
concurrency:
group: benchmark-promote-image
cancel-in-progress: false
jobs:
promote:
name: terraform apply image_tag=${{ inputs.image_tag }}
if: github.repository == 'Tracer-Cloud/opensre'
runs-on: ubuntu-latest
timeout-minutes: 10
env:
AWS_REGION: us-east-1
steps:
- name: Verify required repo variables
env:
AWS_ACCOUNT_ID: ${{ vars.AWS_ACCOUNT_ID }}
run: |
if [ -z "${AWS_ACCOUNT_ID:-}" ]; then
echo "::error::Missing repo variable AWS_ACCOUNT_ID. See tests/benchmarks/cloudopsbench/infra/AWS_BENCH_SETUP.md."
exit 1
fi
- uses: actions/checkout@v5
- name: Configure AWS credentials (OIDC role assumption)
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/opensre-bench-github-actions
role-session-name: bench-promote-${{ github.run_id }}
aws-region: us-east-1
- name: Verify image tag exists in ECR
# Fail loudly if the operator typos the tag, before Terraform tries
# to register a task definition pointing at a missing image.
env:
IMAGE_TAG: ${{ inputs.image_tag }}
run: |
if ! aws ecr describe-images \
--repository-name opensre-bench \
--image-ids imageTag="$IMAGE_TAG" \
>/dev/null 2>&1; then
echo "::error::Image tag $IMAGE_TAG not found in ECR repo opensre-bench."
echo "::error::Push it first via 'Benchmark image — build + push to ECR'."
exit 1
fi
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.7.5
- name: Terraform init
working-directory: tests/benchmarks/cloudopsbench/infra
run: terraform init -input=false
- name: Terraform apply
# Plan is captured in the workflow log; review it in the run page.
# -auto-approve is intentional — this workflow IS the human approval
# (the operator triggered it manually with a specific tag).
#
# -target=aws_ecs_task_definition.bench scopes apply to the task
# definition (and its data-source / role dependencies) only. Without
# this, every dispatch tries to reconcile every resource in the
# module — IAM, S3, ECR, etc. — against whatever ref the workflow
# was dispatched from. Out-of-band local applies cause that
# reconciliation to attempt rollbacks the workflow role isn't
# permitted to perform (iam:DetachRolePolicy, iam:PutRolePolicy),
# failing the run even when the task-def update itself succeeded.
working-directory: tests/benchmarks/cloudopsbench/infra
env:
IMAGE_TAG: ${{ inputs.image_tag }}
run: |
terraform apply -input=false -auto-approve \
-target=aws_ecs_task_definition.bench \
-var="image_tag=$IMAGE_TAG"
- name: Surface the new task definition revision in the job summary
working-directory: tests/benchmarks/cloudopsbench/infra
env:
IMAGE_TAG: ${{ inputs.image_tag }}
run: |
TASK_DEF_ARN=$(terraform output -raw task_definition_arn)
IMAGE_URI=$(aws ecs describe-task-definition \
--task-definition "$TASK_DEF_ARN" \
--query 'taskDefinition.containerDefinitions[0].image' \
--output text)
{
echo "## Image promoted"
echo ""
echo "- Promoted tag: \`$IMAGE_TAG\`"
echo "- New task definition ARN: \`$TASK_DEF_ARN\`"
echo "- Image now in task definition: \`$IMAGE_URI\`"
echo ""
echo "### Next step"
echo ""
echo "Trigger **Benchmark — run on Fargate** to launch a run against this image."
} >> "$GITHUB_STEP_SUMMARY"
+45
View File
@@ -0,0 +1,45 @@
name: Update Benchmark README Section
on:
push:
branches:
- main
paths:
- "docs/benchmarks/results.md"
workflow_dispatch:
jobs:
update-benchmark-readme:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Update README benchmark section
run: uv run python -m tests.benchmarks.toolcall_model_benchmark.readme_updater
- name: Commit changes
run: |
git config user.name "benchmark-readme-action"
git config user.email "benchmark-readme-action@noreply.github.com"
git add README.md
git diff --staged --quiet || git commit -m "docs(benchmark): update README with latest benchmark results"
git pull --rebase origin main
git push
+191
View File
@@ -0,0 +1,191 @@
name: Benchmark — run on Fargate (any adapter)
# Manually-triggered benchmark run on AWS Fargate.
#
# Adapter-agnostic. This workflow does NOT know or care which benchmark
# is running. The ``config`` input names a YAML file; the YAML names an
# adapter (``benchmark: <name>``); the framework's registry resolves it
# to a registered ``BenchmarkAdapter`` subclass. The same workflow runs
# any adapter packaged into the bench image — CloudOpsBench today,
# ToolCallBench / future adapters tomorrow, with zero changes
# here. See ``tests/benchmarks/_framework/registry.py``.
#
# Why ECS RunTask instead of a GitHub-hosted ubuntu runner: a full bench
# grid runs for hours and writes hundreds of MB of artifacts — Fargate
# handles it cleanly, has AWS Secrets Manager wired in via tests/benchmarks/cloudopsbench/infra/,
# and writes to the per-run S3 bucket. The workflow's job is just to
# launch the task and print where to watch.
#
# Trigger from the GitHub UI:
# Actions → "Benchmark run (manual)" → Run workflow → fill inputs
#
# Pre-reqs (one-time, see tests/benchmarks/cloudopsbench/infra/README.md):
# - tests/benchmarks/cloudopsbench/infra/ Terraform applied
# - Bench image pushed to ECR via benchmark-image.yml
# - Repo secrets seeded into AWS Secrets Manager via benchmark-seed-secret.yml
# - Repo variables set:
# AWS_ACCOUNT_ID, BENCH_ECS_CLUSTER, BENCH_TASK_DEFINITION_FAMILY,
# BENCH_SUBNET_IDS (comma-separated), BENCH_SECURITY_GROUP_ID
on:
workflow_dispatch:
inputs:
config:
# No adapter-specific default — the operator must point at a
# specific config. Leaving the field empty surfaces the choice
# rather than silently dispatching the CloudOpsBench smoke. The
# path is relative to the container's repo root. Examples:
# tests/benchmarks/cloudopsbench/configs/cloudopsbench_smoke.yml
# tests/benchmarks/<adapter>/configs/<config>.yml
description: 'Path to YAML config inside the container (e.g. tests/benchmarks/<adapter>/configs/<config>.yml)'
required: true
dev_mode:
description: 'Dev mode (skip integrity gates, no pre-reg needed)'
type: boolean
default: true
# Note: there is intentionally no `image_tag` input here. ECS RunTask
# container overrides do NOT support overriding the image URI — that lives
# on the task definition itself. The image actually pulled is whatever
# was registered the last time `terraform apply -var=image_tag=<tag>` ran
# in tests/benchmarks/cloudopsbench/infra/. Adding a workflow input here would silently mislead
# operators into thinking they control the image per run. To change the
# image, push it via `Benchmark image — build + push to ECR`, then re-apply
# Terraform with the new tag.
permissions:
contents: read
id-token: write # required for AWS OIDC role assumption
concurrency:
group: benchmark-run
cancel-in-progress: false
jobs:
launch:
name: launch ECS task
if: github.repository == 'Tracer-Cloud/opensre'
runs-on: ubuntu-latest
timeout-minutes: 15 # we only LAUNCH the task; we don't wait for it
env:
AWS_REGION: us-east-1
steps:
- name: Verify required repo variables
# Fail loudly BEFORE the AWS auth step if any required repo variable
# is missing. Without this, an unset var would surface downstream as
# an opaque error like "Task Definition can not be blank" from
# describe-task-definition. See tests/benchmarks/cloudopsbench/infra/AWS_BENCH_SETUP.md step 4 for how
# to set these.
env:
AWS_ACCOUNT_ID: ${{ vars.AWS_ACCOUNT_ID }}
BENCH_ECS_CLUSTER: ${{ vars.BENCH_ECS_CLUSTER }}
BENCH_TASK_DEFINITION_FAMILY: ${{ vars.BENCH_TASK_DEFINITION_FAMILY }}
BENCH_SUBNET_IDS: ${{ vars.BENCH_SUBNET_IDS }}
BENCH_SECURITY_GROUP_ID: ${{ vars.BENCH_SECURITY_GROUP_ID }}
run: |
missing=()
for var in AWS_ACCOUNT_ID BENCH_ECS_CLUSTER BENCH_TASK_DEFINITION_FAMILY \
BENCH_SUBNET_IDS BENCH_SECURITY_GROUP_ID; do
if [ -z "${!var:-}" ]; then
missing+=("$var")
fi
done
if [ "${#missing[@]}" -gt 0 ]; then
echo "::error::Missing repo variable(s): ${missing[*]}"
echo "::error::Set them under Settings → Secrets and variables → Actions → Variables."
echo "::error::Values come from \`cd tests/benchmarks/cloudopsbench/infra && terraform output\` — see tests/benchmarks/cloudopsbench/infra/AWS_BENCH_SETUP.md step 4."
exit 1
fi
echo "All 5 required repo variables are set."
- name: Configure AWS credentials (OIDC role assumption)
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/opensre-bench-github-actions
role-session-name: bench-run-${{ github.run_id }}
aws-region: us-east-1
- name: Launch ECS RunTask
# Inputs bound to env vars so the shell does the interpolation —
# GitHub Actions template syntax (${{ ... }}) is expanded before
# the shell sees it, which would let a crafted input inject shell
# metacharacters. Env vars are quoted at use-site.
env:
BENCH_CONFIG: ${{ inputs.config }}
BENCH_DEV_FLAG: ${{ inputs.dev_mode == true && '--dev' || '' }}
CLUSTER: ${{ vars.BENCH_ECS_CLUSTER }}
TASK_FAMILY: ${{ vars.BENCH_TASK_DEFINITION_FAMILY }}
SUBNETS: ${{ vars.BENCH_SUBNET_IDS }}
SECURITY_GROUP: ${{ vars.BENCH_SECURITY_GROUP_ID }}
run: |
set -euo pipefail
# Surface the image the task definition will actually pull, so
# operators see the truth (not a workflow input that ECS ignores).
IMAGE_URI=$(aws ecs describe-task-definition \
--task-definition "$TASK_FAMILY" \
--query 'taskDefinition.containerDefinitions[0].image' \
--output text)
# Overrides JSON: container env vars passed at runtime. The
# container's entrypoint reads BENCH_CONFIG + BENCH_DEV_FLAG
# and invokes:
# uv run python -m tests.benchmarks._framework.cli run \
# "$BENCH_CONFIG" $BENCH_DEV_FLAG
OVERRIDES=$(jq -nc \
--arg config "$BENCH_CONFIG" \
--arg dev_flag "$BENCH_DEV_FLAG" \
'{
containerOverrides: [{
name: "bench",
environment: [
{name: "BENCH_CONFIG", value: $config},
{name: "BENCH_DEV_FLAG", value: $dev_flag}
]
}]
}')
TASK_ARN=$(aws ecs run-task \
--cluster "$CLUSTER" \
--task-definition "$TASK_FAMILY" \
--launch-type FARGATE \
--network-configuration "awsvpcConfiguration={subnets=[$SUBNETS],securityGroups=[$SECURITY_GROUP],assignPublicIp=ENABLED}" \
--overrides "$OVERRIDES" \
--query 'tasks[0].taskArn' \
--output text)
if [ -z "$TASK_ARN" ] || [ "$TASK_ARN" = "None" ]; then
echo "::error::run-task returned no taskArn"
exit 1
fi
TASK_ID="${TASK_ARN##*/}"
echo "task_arn=$TASK_ARN" >> "$GITHUB_OUTPUT"
echo "task_id=$TASK_ID" >> "$GITHUB_OUTPUT"
# Job summary — visible in the workflow run page
{
echo "## Bench run launched"
echo ""
echo "- Image (from task definition): \`$IMAGE_URI\`"
echo "- Config: \`$BENCH_CONFIG\`"
echo "- Dev mode: \`${BENCH_DEV_FLAG:-(off)}\`"
echo "- Task ARN: \`$TASK_ARN\`"
echo ""
echo "### Watch it"
echo ""
echo "Stream logs locally:"
echo ""
echo '```bash'
echo "aws logs tail /ecs/opensre-bench --follow"
echo '```'
echo ""
echo "Or via AWS Console:"
echo "https://us-east-1.console.aws.amazon.com/ecs/v2/clusters/$CLUSTER/tasks/$TASK_ID"
echo ""
echo "Artifacts land in the bench S3 bucket under \`runs/\` when the task completes."
echo ""
echo "_To change the running image: push a new tag via \`Benchmark image — build + push to ECR\`, then \`cd tests/benchmarks/cloudopsbench/infra && terraform apply -var=image_tag=<tag>\`._"
} >> "$GITHUB_STEP_SUMMARY"
+102
View File
@@ -0,0 +1,102 @@
name: Benchmark secret — seed GitHub repo secret into AWS Secrets Manager
# Manually-triggered workflow that copies a GitHub repo secret into AWS
# Secrets Manager at opensre-bench/llm/<secret>. The bench container reads
# its LLM API keys from Secrets Manager at runtime; this workflow is how
# you put them there without touching a developer laptop.
#
# The `secret` dropdown enforces which target is valid — must match one
# of the four IAM-granted secret ARNs in tests/benchmarks/cloudopsbench/infra/iam_oidc.tf
# (anthropic / openai / deepseek / hf_token).
#
# Why a workflow instead of a developer running `aws secretsmanager
# put-secret-value` locally: keeps keys off developer laptops + out of
# shell history, and centralizes rotation — rotate the value in the GH
# repo secret and re-run this workflow to propagate.
#
# Pre-reqs:
# - tests/benchmarks/cloudopsbench/infra/ Terraform has been applied (the secret resource exists)
# - Corresponding GitHub repo secret is set:
# ANTHROPIC_API_KEY / OPENAI_API_KEY / DEEPSEEK_API_KEY / HF_TOKEN
#
# Order of operations:
# 1. terraform-bench.yml (plan) on PR
# 2. terraform apply locally
# 3. this workflow (workflow_dispatch) — pick the secret to seed
# from the dropdown, repeat once per LLM provider key
on:
workflow_dispatch:
inputs:
secret:
description: Which secret to seed (must match the AWS resource name)
required: true
type: choice
options:
- anthropic_api_key
- openai_api_key
- deepseek_api_key
- hf_token
permissions:
contents: read
id-token: write # required for AWS OIDC role assumption
concurrency:
group: bench-seed-${{ inputs.secret }}
cancel-in-progress: false
jobs:
seed:
name: seed ${{ inputs.secret }}
if: github.repository == 'Tracer-Cloud/opensre'
runs-on: ubuntu-latest
env:
AWS_REGION: us-east-1
SECRET_ID: opensre-bench/llm/${{ inputs.secret }}
steps:
- name: Configure AWS credentials (OIDC role assumption)
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/opensre-bench-github-actions
role-session-name: github-actions-seed-${{ inputs.secret }}-${{ github.run_id }}
aws-region: us-east-1
- name: Verify secret resource exists
run: |
if ! aws secretsmanager describe-secret --secret-id "$SECRET_ID" >/dev/null 2>&1; then
echo "::error::Secret $SECRET_ID not found. Run \`terraform apply\` in tests/benchmarks/cloudopsbench/infra/ first."
exit 1
fi
- name: Put secret value
# All four candidate values are bound at workflow level. The case
# statement picks the one matching the chosen target — unused
# values stay as masked env vars (GH redacts secret refs in logs).
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
HF_TOKEN: ${{ secrets.HF_TOKEN }}
TARGET: ${{ inputs.secret }}
run: |
case "$TARGET" in
anthropic_api_key) V="$ANTHROPIC_API_KEY" ;;
openai_api_key) V="$OPENAI_API_KEY" ;;
deepseek_api_key) V="$DEEPSEEK_API_KEY" ;;
hf_token) V="$HF_TOKEN" ;;
*)
echo "::error::Unknown target: $TARGET (dropdown should have prevented this)"
exit 1
;;
esac
if [ -z "$V" ]; then
echo "::error::GitHub repo secret for $TARGET is unset. Configure it under Settings > Secrets and variables > Actions."
exit 1
fi
aws secretsmanager put-secret-value \
--secret-id "$SECRET_ID" \
--secret-string "$V" >/dev/null
echo "Seeded $SECRET_ID (length=${#V})."
+37
View File
@@ -0,0 +1,37 @@
name: Celebrate merged pull requests
# Runs after a PR lands (merged into the target branch). Uses pull_request_target so the
# token can comment on merged PRs from forks; checkout is base/default only — no PR head.
on:
pull_request_target:
types: [closed]
permissions:
contents: read
pull-requests: write
jobs:
celebrate-merge:
name: Post-merge celebration comment
runs-on: ubuntu-latest
if: >-
github.event.pull_request.merged == true &&
github.event.pull_request.user.type != 'Bot' &&
github.event.pull_request.user.login != 'dependabot[bot]'
steps:
- uses: actions/checkout@v5
- name: Build celebration comment
id: celebrate
env:
DISCORD_INVITE_URL: https://discord.com/invite/opensre
CONTRIBUTOR_LOGIN: ${{ github.event.pull_request.user.login }}
run: python3 .github/scripts/merged_pr_celebration_message.py
- name: Comment on merged PR
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: >
gh pr comment "${{ github.event.pull_request.number }}"
--repo "${{ github.repository }}"
--body-file comment.md
+109
View File
@@ -0,0 +1,109 @@
# Optional Windows CI when PR label `ci:windows` is present.
# Separate workflow + concurrency group so Windows label runs are isolated.
name: CI Labels (Windows)
on:
pull_request:
branches: [main]
types: [opened, synchronize, reopened, labeled, unlabeled]
permissions:
contents: read
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
# Keep in sync with ci.yml — single source of truth for lint/typecheck/coverage targets.
PYTHON_SOURCE_PATHS: "config core integrations platform surfaces tools"
concurrency:
group: ci-labels-windows-${{ github.ref }}
cancel-in-progress: true
jobs:
windows-quality:
if: contains(github.event.pull_request.labels.*.name, 'ci:windows')
name: windows quality
runs-on: windows-latest
timeout-minutes: 15
continue-on-error: true
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Lint
run: uv run python -m ruff check ${{ env.PYTHON_SOURCE_PATHS }} tests/
- name: Format check
run: uv run python -m ruff format --check ${{ env.PYTHON_SOURCE_PATHS }} tests/
- name: Typecheck
run: uv run python -m mypy ${{ env.PYTHON_SOURCE_PATHS }}
windows-test:
if: contains(github.event.pull_request.labels.*.name, 'ci:windows')
name: windows test
needs: [windows-quality]
runs-on: windows-latest
timeout-minutes: 25
continue-on-error: true
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
JWT_TOKEN: ${{ secrets.JWT_TOKEN }}
TRACER_ORG_ID: ${{ secrets.TRACER_ORG_ID }}
TRACER_WEB_APP_URL: ${{ secrets.TRACER_WEB_APP_URL }}
GRAFANA_READ_TOKEN: ${{ secrets.GRAFANA_READ_TOKEN }}
GCLOUD_HOSTED_METRICS_ID: ${{ secrets.GCLOUD_HOSTED_METRICS_ID }}
GCLOUD_HOSTED_METRICS_URL: ${{ secrets.GCLOUD_HOSTED_METRICS_URL }}
GCLOUD_HOSTED_LOGS_ID: ${{ secrets.GCLOUD_HOSTED_LOGS_ID }}
GCLOUD_HOSTED_LOGS_URL: ${{ secrets.GCLOUD_HOSTED_LOGS_URL }}
GCLOUD_RW_API_KEY: ${{ secrets.GCLOUD_RW_API_KEY }}
GCLOUD_OTLP_ENDPOINT: ${{ secrets.GCLOUD_OTLP_ENDPOINT }}
GCLOUD_OTLP_AUTH_HEADER: ${{ secrets.GCLOUD_OTLP_AUTH_HEADER }}
PYTHONUTF8: "1"
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Run full tests
run: >-
uv run python -m pytest -n auto -v
--cov=config --cov=core --cov=platform.deployment.aws --cov=platform.deployment --cov=integrations --cov=platform --cov=surfaces --cov=tools
--cov-report=term-missing
--cov-report=html
--ignore=tests/e2e/kubernetes_local_alert_simulation
--ignore=tests/synthetic
-m "not synthetic and not integration"
+468
View File
@@ -0,0 +1,468 @@
name: CI
on:
push:
branches: [main]
paths:
- "surfaces/**"
- "config/**"
- "core/**"
- "platform/deployment/**"
- "integrations/**"
- "platform/**"
- "tools/**"
- "tests/**"
- "pyproject.toml"
- "uv.lock"
- "pytest.ini"
- "ruff.toml"
- "mypy.ini"
- "Makefile"
- ".importlinter"
- ".importlinter.strict"
- ".github/ci/**"
- ".github/workflows/ci.yml"
pull_request:
branches: [main]
paths:
- "surfaces/**"
- "config/**"
- "core/**"
- "platform/deployment/**"
- "integrations/**"
- "platform/**"
- "tools/**"
- "tests/**"
- "pyproject.toml"
- "uv.lock"
- "pytest.ini"
- "ruff.toml"
- "mypy.ini"
- "Makefile"
- ".importlinter"
- ".importlinter.strict"
- ".github/ci/**"
- ".github/workflows/ci.yml"
permissions:
contents: read
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
PYTHON_SOURCE_PATHS: "config core integrations platform surfaces tools"
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
jobs:
quality:
name: quality (${{ matrix.os }})
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 15
continue-on-error: ${{ matrix.os == 'windows-latest' }}
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Lint
run: uv run python -m ruff check $PYTHON_SOURCE_PATHS tests/
- name: Format check
run: uv run python -m ruff format --check $PYTHON_SOURCE_PATHS tests/
- name: Typecheck
run: uv run python -m mypy $PYTHON_SOURCE_PATHS
- name: Import graph
run: uv run python .github/ci/check_imports.py --strict
# Full layered contracts (.importlinter.strict), not the minimal default
# .importlinter config used by ``make check-imports``.
- name: Import boundaries
run: uv run lint-imports --config .importlinter.strict
test:
name: test (${{ matrix.shard }})
strategy:
fail-fast: false
max-parallel: 5
matrix:
include:
- os: ubuntu-latest
shard: tools-runtime
# ``tests/core/agent`` is excluded here; it runs in cli-runtime,
# which pins openai for its live action-planning oracles.
extra_pytest_args: >-
--ignore=tests/core/agent
pytest_paths: >-
tests/tools
tests/core
tests/platform
tests/utils
tests/masking
tests/watch_dog
- os: ubuntu-latest
shard: cli-runtime
# Live shell action-agent contracts are validated on openai,
# matching interactive-shell-live.yml. The
# default anthropic tool-call model (haiku) cannot reliably perform
# compound action planning, so pin this shard to openai.
# ``tests/core/agent`` runs here because it contains the live
# action-planning oracles (``test_live_action_planning`` and
# ``test_live_turn_execution_oracle``) that need the openai pin —
# plus the static ``test_import_boundaries`` guard the surfaces
# restructure (#3299) added.
llm_provider: openai
pytest_paths: >-
tests/cli
tests/interactive_shell
tests/agent
tests/core/agent
tests/fleet_monitoring
tests/analytics
tests/tools/investigation
tests/delivery
tests/sandbox
tests/hermes
- os: ubuntu-latest
shard: integrations-and-misc
pytest_paths: >-
tests/integrations
tests/deployment
tests/benchmarks
tests/chaos_engineering
tests/shared
tests/config
tests/github_ci
tests/packaging
tests/scheduler
gateway/tests
- os: ubuntu-latest
shard: e2e-general
pytest_paths: >-
tests/e2e
extra_pytest_args: >-
--ignore=tests/e2e/kubernetes
--ignore=tests/e2e/openclaw
--ignore=tests/e2e/upstream_lambda
--ignore=tests/e2e/upstream_prefect_ecs_fargate
--ignore=tests/e2e/upstream_apache_flink_ecs
- os: ubuntu-latest
shard: e2e-provider-and-openclaw
pytest_paths: >-
tests/e2e/openclaw
tests/e2e/upstream_lambda
tests/e2e/upstream_prefect_ecs_fargate
tests/e2e/upstream_apache_flink_ecs
tests/e2e/kubernetes
runs-on: ${{ matrix.os }}
timeout-minutes: 30
env:
# Per-shard LLM provider for live_llm contracts; defaults to anthropic.
LLM_PROVIDER: ${{ matrix.llm_provider || 'anthropic' }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
JWT_TOKEN: ${{ secrets.JWT_TOKEN }}
GRAFANA_READ_TOKEN: ${{ secrets.GRAFANA_READ_TOKEN }}
GCLOUD_HOSTED_METRICS_ID: ${{ secrets.GCLOUD_HOSTED_METRICS_ID }}
GCLOUD_HOSTED_METRICS_URL: ${{ secrets.GCLOUD_HOSTED_METRICS_URL }}
GCLOUD_HOSTED_LOGS_ID: ${{ secrets.GCLOUD_HOSTED_LOGS_ID }}
GCLOUD_HOSTED_LOGS_URL: ${{ secrets.GCLOUD_HOSTED_LOGS_URL }}
GCLOUD_RW_API_KEY: ${{ secrets.GCLOUD_RW_API_KEY }}
GCLOUD_OTLP_ENDPOINT: ${{ secrets.GCLOUD_OTLP_ENDPOINT }}
GCLOUD_OTLP_AUTH_HEADER: ${{ secrets.GCLOUD_OTLP_AUTH_HEADER }}
PYTHONUTF8: "1"
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Validate pytest marker
env:
# Fork PRs do not receive repository secrets; skip live_llm contracts
# (see interactive-shell-live.yml). Index a JSON string array so the
# value is always a marker string — never a bare boolean from
# ``a && 'x' || 'y'`` (boolean false → ``-m false`` → xdist
# ``N workers [0 items]``, which can still exit 0).
PYTEST_MARKER_EXPR: ${{ fromJSON('["not synthetic", "not (synthetic or live_llm)"]')[github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true] }}
run: |
echo "PYTEST_MARKER_EXPR=${PYTEST_MARKER_EXPR}"
case "${PYTEST_MARKER_EXPR}" in
"not synthetic"|"not (synthetic or live_llm)") ;;
*)
echo "Refusing unexpected PYTEST_MARKER_EXPR: '${PYTEST_MARKER_EXPR}'" >&2
exit 1
;;
esac
- name: Run tests
env:
COVERAGE_FILE: .coverage.${{ matrix.shard }}
PYTEST_MARKER_EXPR: ${{ fromJSON('["not synthetic", "not (synthetic or live_llm)"]')[github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true] }}
# Keep ``run: >`` (folded) so ``matrix.pytest_paths`` stays one argv list.
# A ``run: |`` + ``\`` rewrite can drop/split those paths and yield
# ``N workers [0 items]`` even when the marker is correct.
run: >
uv run python -m pytest -n auto -v
${{ matrix.pytest_paths }}
--cov=config
--cov=core
--cov=platform.deployment.aws --cov=platform.deployment
--cov=integrations
--cov=platform
--cov=surfaces
--cov=tools
--cov-report=
--ignore=tests/e2e/kubernetes_local_alert_simulation
--ignore=tests/synthetic
${{ matrix.extra_pytest_args }}
-m "${PYTEST_MARKER_EXPR}"
- name: Upload shard coverage data
if: >-
always() &&
github.event_name == 'push' &&
github.ref == 'refs/heads/main'
uses: actions/upload-artifact@v4
with:
name: coverage-data-${{ matrix.shard }}
path: .coverage.${{ matrix.shard }}*
if-no-files-found: error
include-hidden-files: true
retention-days: 7
coverage-report:
name: coverage-report
runs-on: ubuntu-latest
needs: [test]
if: >-
always() &&
github.event_name == 'push' &&
github.ref == 'refs/heads/main'
timeout-minutes: 15
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Download shard coverage artifacts
uses: actions/download-artifact@v4
with:
pattern: coverage-data-*
path: ./
merge-multiple: true
- name: Combine coverage and build report
run: |
uv run python -m coverage combine .
uv run python -m coverage html
- name: Upload combined coverage
uses: actions/upload-artifact@v4
with:
name: coverage-report-ubuntu-latest
path: |
htmlcov/
.coverage
retention-days: 7
test-kubernetes:
runs-on: ubuntu-latest
continue-on-error: true
timeout-minutes: 15
if: >-
github.repository == 'Tracer-Cloud/opensre' &&
(github.event_name == 'push' ||
(
github.event_name == 'pull_request' &&
(
contains(github.event.pull_request.title, 'k8s') ||
contains(github.event.pull_request.title, 'kubernetes')
)
))
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Run Kubernetes tests
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: us-east-1
run: uv run python -m tests.e2e.kubernetes.test_local
should-run-thorough:
runs-on: ubuntu-latest
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && github.repository == 'Tracer-Cloud/opensre'
outputs:
thorough: ${{ steps.filter.outputs.thorough }}
steps:
- uses: actions/checkout@v5
- uses: dorny/paths-filter@v4
id: filter
with:
filters: |
thorough:
- ".github/workflows/ci.yml"
- "pyproject.toml"
- "uv.lock"
- "platform/deployment/**"
- "integrations/**"
- "tools/investigation/**"
- "tools/**"
- "tests/e2e/cloudwatch_demo/**"
- "tests/e2e/upstream_lambda/**"
- "tests/e2e/upstream_prefect_ecs_fargate/**"
- "tests/e2e/upstream_apache_flink_ecs/**"
test-thorough:
runs-on: ubuntu-latest
needs: [test, should-run-thorough]
if: >-
github.event_name == 'push' &&
github.ref == 'refs/heads/main' &&
github.repository == 'Tracer-Cloud/opensre' &&
needs.should-run-thorough.outputs.thorough == 'true'
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
include:
- name: cloudwatch-demo
command: python3 -m tests.e2e.cloudwatch_demo.test_aws
- name: upstream-lambda
command: python3 -m tests.e2e.upstream_lambda.test_agent_e2e
- name: prefect-ecs-fargate
command: python3 -m tests.e2e.upstream_prefect_ecs_fargate.test_agent_e2e
- name: flink-ecs
command: python3 -m tests.e2e.upstream_apache_flink_ecs.test_agent_e2e
name: test-thorough (${{ matrix.name }})
env:
# Thorough e2e suites make live LLM calls; run them on openai because the
# default anthropic account is credit-exhausted. Requires a valid, funded
# OPENAI_API_KEY secret.
LLM_PROVIDER: openai
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
JWT_TOKEN: ${{ secrets.JWT_TOKEN }}
TRACER_API_URL: ${{ secrets.TRACER_API_URL }}
TRACER_INGEST_TOKEN: ${{ secrets.TRACER_INGEST_TOKEN }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: us-east-1
AWS_DEFAULT_REGION: us-east-1
GRAFANA_READ_TOKEN: ${{ secrets.GRAFANA_READ_TOKEN }}
GCLOUD_HOSTED_METRICS_ID: ${{ secrets.GCLOUD_HOSTED_METRICS_ID }}
GCLOUD_HOSTED_METRICS_URL: ${{ secrets.GCLOUD_HOSTED_METRICS_URL }}
GCLOUD_HOSTED_LOGS_ID: ${{ secrets.GCLOUD_HOSTED_LOGS_ID }}
GCLOUD_HOSTED_LOGS_URL: ${{ secrets.GCLOUD_HOSTED_LOGS_URL }}
GCLOUD_RW_API_KEY: ${{ secrets.GCLOUD_RW_API_KEY }}
CLOUDWATCH_VERIFY_LOGS: ${{ matrix.name == 'cloudwatch-demo' && '0' || '1' }}
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Install tracer CLI
run: |
curl -sSL https://install.tracer.cloud | sh -s user_36fbN6K6FwEgJFHsQv1pUByo8K6
- name: Initialize tracer (optional)
run: |
sudo tracer init --token ${{ secrets.JWT_TOKEN }}
continue-on-error: true
timeout-minutes: 2
- name: Run test
run: uv run ${{ matrix.command }}
timeout-minutes: 30
+118
View File
@@ -0,0 +1,118 @@
name: Closed-loop learning (weekly miss triage)
# Weekly reminder workflow for the closed-loop learning process documented
# in docs/closed-loop-learning.mdx. Runs at 09:00 UTC every Monday and
# also supports manual triggering.
#
# The workflow does not consume the per-user ``~/.opensre/misses.jsonl``
# store directly (that lives on engineer machines, not on the runner).
# Its purpose is to:
# 1) Open or refresh a tracking issue that nudges the on-call engineer
# to run ``opensre misses stats --since 7d`` and then
# ``opensre misses export --since 7d --top 10 --out tests/benchmarks/production_misses/``.
# 2) Verify the ``misses_command`` CLI surface is still wired and exits
# cleanly, so the weekly process is not blocked by a CLI regression.
on:
schedule:
- cron: "0 9 * * 1"
workflow_dispatch: {}
permissions:
contents: read
issues: write
jobs:
smoke:
name: Verify opensre misses CLI
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Smoke test misses CLI
run: |
uv run opensre misses --help
uv run opensre misses list --json
uv run opensre misses stats --json
remind:
name: Open weekly triage reminder
needs: smoke
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v5
- name: Open weekly tracking issue (close last week's first)
uses: actions/github-script@v7
with:
script: |
// One issue per week. Previous week's tracker is closed before
// opening this week's so triage history is per-week (GitHub's
// natural unit) rather than one ever-growing comment thread.
// Uses the existing "pending triage" label so we do not
// auto-create a new label on first run.
const week = new Date().toISOString().slice(0, 10);
const titlePrefix = "Weekly closed-loop learning triage";
const title = `${titlePrefix} — week of ${week}`;
const body = [
"Weekly nudge per docs/closed-loop-learning.mdx.",
"",
"Steps for this week's on-call:",
"- [ ] `opensre misses stats --since 7d`",
"- [ ] Review recurring `(alert, taxonomy)` pairs",
"- [ ] `opensre misses export --since 7d --top 10 --out tests/benchmarks/production_misses/`",
"- [ ] Open a PR labelled `benchmark` with the new scenarios",
"- [ ] Trigger the benchmark workflow on the PR branch",
].join("\n");
const { data: openIssues } = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
state: "open",
labels: "pending triage",
per_page: 100,
});
const stale = openIssues.filter(
(i) => i.title.startsWith(titlePrefix) && i.title !== title,
);
for (const issue of stale) {
await github.rest.issues.update({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue.number,
state: "closed",
state_reason: "completed",
});
}
const existing = openIssues.find((i) => i.title === title);
if (existing) {
return;
}
await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title,
body,
labels: ["pending triage"],
});
+43
View File
@@ -0,0 +1,43 @@
name: CodeQL
on:
push:
branches: [main]
pull_request:
branches: [main]
# Skip CodeQL for docs-only PRs (Markdown/MDX and the docs site tree) to
# save CI minutes. Not a required check, so skipping cannot block merge.
# `push` to main and the weekly schedule keep full coverage regardless.
paths-ignore:
- '**/*.md'
- '**/*.mdx'
- 'docs/**'
schedule:
- cron: "0 6 * * 1"
permissions:
contents: read
security-events: write
actions: read
jobs:
analyze:
if: github.repository == 'Tracer-Cloud/opensre'
name: Analyze (python)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
with:
languages: python
build-mode: none
config-file: .github/codeql/codeql-config.yml
queries: security-and-quality
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4
with:
category: /language:python
+148
View File
@@ -0,0 +1,148 @@
name: Docker Publish
on:
release:
types: [published]
schedule:
- cron: "30 2 * * *"
workflow_dispatch:
inputs:
tag:
description: "Release tag to build (e.g. v0.1.2026.7.8). Defaults to the latest release."
required: false
type: string
permissions:
contents: read
packages: write
concurrency:
group: docker-publish
cancel-in-progress: false
jobs:
build-and-push:
if: github.repository == 'Tracer-Cloud/opensre'
runs-on: ubuntu-latest
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Resolve release tag and image tags
id: meta
env:
EVENT_NAME: ${{ github.event_name }}
RELEASE_TAG: ${{ github.event.release.tag_name }}
RELEASE_PRERELEASE: ${{ github.event.release.prerelease }}
DISPATCH_TAG: ${{ inputs.tag }}
shell: bash
run: |
set -euo pipefail
image="ghcr.io/${GITHUB_REPOSITORY,,}"
latest_tag="$(gh api "repos/${GITHUB_REPOSITORY}/releases/latest" --jq .tag_name)"
case "$EVENT_NAME" in
release)
if [ "$RELEASE_PRERELEASE" = "true" ]; then
echo "Skipping prerelease $RELEASE_TAG (rolling main builds are not published to GHCR)."
echo "build=false" >> "$GITHUB_OUTPUT"
exit 0
fi
tag="$RELEASE_TAG"
;;
workflow_dispatch)
tag="${DISPATCH_TAG:-$latest_tag}"
gh release view "$tag" --repo "$GITHUB_REPOSITORY" >/dev/null
;;
*)
tag="$latest_tag"
;;
esac
version="${tag#v}"
# The scheduled run rebuilds the newest release; skip it when that
# version is already in the registry (e.g. the release event or a
# dispatch already published it).
if [ "$EVENT_NAME" = "schedule" ] \
&& docker manifest inspect "${image}:${version}" >/dev/null 2>&1; then
echo "Image ${image}:${version} already published; nothing to do."
echo "build=false" >> "$GITHUB_OUTPUT"
exit 0
fi
tags="${image}:${version}"
if [ "$tag" = "$latest_tag" ]; then
tags="${tags}"$'\n'"${image}:latest"
else
echo "Tag $tag is older than latest release $latest_tag; not moving :latest."
fi
{
echo "build=true"
echo "tag=${tag}"
echo "version=${version}"
echo "image=${image}"
echo "tags<<EOF"
echo "$tags"
echo "EOF"
} >> "$GITHUB_OUTPUT"
- uses: actions/checkout@v5
if: steps.meta.outputs.build == 'true'
with:
ref: ${{ steps.meta.outputs.tag }}
- name: Sync release version into pyproject.toml
if: steps.meta.outputs.build == 'true'
shell: bash
run: python3 platform/packaging/sync_release_version.py --tag "${{ steps.meta.outputs.tag }}"
- name: Set up QEMU
if: steps.meta.outputs.build == 'true'
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
if: steps.meta.outputs.build == 'true'
uses: docker/setup-buildx-action@v3
- name: Build and push image
if: steps.meta.outputs.build == 'true'
uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: |
org.opencontainers.image.title=opensre
org.opencontainers.image.description=OpenSRE unified image (MODE=web FastAPI health app, MODE=gateway Telegram gateway)
org.opencontainers.image.source=https://github.com/${{ github.repository }}
org.opencontainers.image.version=${{ steps.meta.outputs.version }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Smoke test published image
if: steps.meta.outputs.build == 'true'
shell: bash
run: |
set -euo pipefail
image="${{ steps.meta.outputs.image }}:${{ steps.meta.outputs.version }}"
docker pull "$image"
version_output="$(docker run --rm --entrypoint opensre "$image" --version)"
printf '%s\n' "$version_output"
case "$version_output" in
*"${{ steps.meta.outputs.version }}"*) ;;
*)
echo "Image version mismatch: expected ${{ steps.meta.outputs.version }}" >&2
exit 1
;;
esac
+186
View File
@@ -0,0 +1,186 @@
# OpenClaw end-to-end CI.
#
# Triggers, in order of intent:
# 1. PRs that touch OpenClaw-relevant paths (auto, primary signal)
# 2. Pushes to main that touch the same paths (post-merge regression catch)
# 3. Daily schedule at 06:00 UTC (catches environment drift —
# OpenClaw version bumps, MCP SDK
# breaks, npm install regressions)
# 4. Adding the `ci:openclaw` label to a PR (manual re-run on matching PRs)
# 5. Workflow_dispatch from the Actions UI (manual force-run on any ref,
# no PR needed)
#
# The path filter at the trigger level is the primary gate, so non-OpenClaw PRs
# pay no CI cost. The label is only useful as a re-trigger on PRs that already
# matched paths; for force-running against an unrelated branch, use the manual
# dispatch button in the Actions UI.
name: CI (OpenClaw E2E)
on:
pull_request:
branches: [main]
types: [opened, synchronize, reopened, labeled, unlabeled]
paths:
- "integrations/openclaw.py"
- "tools/OpenClawMCPTool/**"
- "tests/e2e/openclaw/**"
- "tests/utils/alert_factory/**"
- ".github/workflows/e2e-openclaw.yml"
- "docs/openclaw.mdx"
- ".tool-versions"
push:
branches: [main]
paths:
- "integrations/openclaw.py"
- "tools/OpenClawMCPTool/**"
- "tests/e2e/openclaw/**"
- "tests/utils/alert_factory/**"
- ".github/workflows/e2e-openclaw.yml"
- "docs/openclaw.mdx"
- ".tool-versions"
schedule:
- cron: "0 6 * * *"
workflow_dispatch:
permissions:
contents: read
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
concurrency:
group: ci-openclaw-${{ github.ref }}
cancel-in-progress: true
jobs:
# Quality gates (lint / format-check / mypy) are run by the main
# ``ci.yml`` workflow for every PR. We don't duplicate them here —
# this workflow's sole job is the OpenClaw e2e suite. If you reach
# this workflow, your PR has already cleared the quality gates.
openclaw-test:
if: github.repository == 'Tracer-Cloud/opensre'
name: openclaw test
runs-on: ubuntu-latest
timeout-minutes: 25
# No ``env:`` block by design. CI runs the boot + use-case
# scenarios only — the full-RCA sub-tests call an LLM, cost API
# credits, and take ~25s each, so we deliberately don't wire any
# LLM secret in. They auto-skip via
# ``LLM_CREDENTIAL_SKIP_REASON`` when no key is present in env.
# Contributors with ANTHROPIC_API_KEY / OPENAI_API_KEY /
# GEMINI_API_KEY set locally still run them via
# ``make test-openclaw``. Lang Smith env vars stay scoped to the
# local-RCA path for the same reason — no value tracing the
# use-case-only CI runs.
steps:
- uses: actions/checkout@v5
- name: Set up Node (OpenClaw requires >=22.12; version pinned in .tool-versions)
uses: actions/setup-node@v4
with:
node-version-file: ".tool-versions"
- name: Cache npm downloads
uses: actions/cache@v4
with:
path: ~/.npm
key: ${{ runner.os }}-openclaw-npm-${{ hashFiles('.tool-versions') }}
restore-keys: |
${{ runner.os }}-openclaw-npm-
- name: Install OpenClaw CLI
# ``continue-on-error: true`` so an npm-registry hiccup, a package-name
# rename on OpenClaw's side, or a network blip does not kill the whole
# workflow. The "Probe OpenClaw availability" step below catches the
# failure and downgrades the run to "tests skipped" with a clear
# explanation rather than crashing.
continue-on-error: true
id: install_openclaw
run: npm install -g openclaw --prefer-offline --no-audit
- name: Probe OpenClaw availability
# Whether or not ``npm install`` succeeded, check if ``openclaw`` is
# actually runnable. Failure modes we surface explicitly:
# - CLI not installed (npm step failed silently)
# - CLI installed but Node version mismatch (older runner image)
# - CLI installed but the bundled JS errored on first run
# ``openclaw_runnable`` env var feeds the next step's gating.
run: |
set +e
if ! command -v openclaw >/dev/null 2>&1; then
echo "::warning::openclaw CLI is not on PATH (npm install may have failed); the e2e suite will be skipped this run."
echo "openclaw_runnable=false" >> "$GITHUB_ENV"
exit 0
fi
version_output="$(openclaw --version 2>&1)"
if [ $? -ne 0 ]; then
echo "::warning::openclaw is on PATH but errors when run:"
echo "$version_output"
echo "::warning::Skipping e2e suite this run."
echo "openclaw_runnable=false" >> "$GITHUB_ENV"
exit 0
fi
echo "✓ OpenClaw verified: $version_output"
echo "openclaw_runnable=true" >> "$GITHUB_ENV"
- name: Note LLM credential policy
# CI deliberately does not wire any LLM key, so the full-RCA
# sub-tests skip every run. This step is informational — it
# makes the policy obvious in the run log instead of leaving a
# silent skip that a reviewer has to dig for. Run RCA locally
# with ``make test-openclaw`` if you have a key configured.
run: |
echo "︎ CI policy: full-RCA sub-tests are skipped (LLM credentials intentionally not wired)."
echo " Boot + use-case sub-tests still run and gate the merge."
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Run OpenClaw E2E suite
# Skips with a clear message if the OpenClaw probe failed earlier,
# so the workflow finishes green with a documented "couldn't run"
# status rather than a confusing red on the install step.
run: |
if [ "$openclaw_runnable" != "true" ]; then
echo "::warning::Skipping pytest invocation — openclaw CLI unavailable in this environment. See earlier warnings."
exit 0
fi
uv run python -m pytest -m e2e -v tests/e2e/openclaw/
- name: Upload gateway logs on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: openclaw-gateway-logs
path: /tmp/openclaw-e2e-logs/
if-no-files-found: ignore
retention-days: 7
- name: Summary
# Always runs, regardless of earlier step outcomes. The probe
# step above downgrades a missing-CLI / wrong-Node environment
# to "tests skipped" rather than failing the workflow, so this
# summary is the one place a reviewer sees why fewer tests ran
# than expected. Real pytest failures still fail the job.
if: always()
run: |
echo "### OpenClaw E2E run summary"
echo " OpenClaw runnable: ${openclaw_runnable:-unknown}"
echo " Full-RCA sub-tests: skipped by CI policy (no LLM key wired)"
echo " Run RCA locally with: make test-openclaw"
@@ -0,0 +1,52 @@
# When someone with zero merged PRs in this repo (and not OWNER/MEMBER/COLLABORATOR) comments
# on an open issue labeled `good first issue`, assign them and reply — only if the issue has no
# assignees yet (first eligible commenter wins; includes when a human pre-assigned someone).
# issue_comment also fires for PR review threads; those payloads include issue.pull_request.
name: Good first issue — auto-assign
on:
issue_comment:
types: [created]
concurrency:
group: good-first-issue-assign-${{ github.repository }}-${{ github.event.issue.number }}
cancel-in-progress: false
permissions:
contents: read
issues: write
pull-requests: read
jobs:
assign:
name: Assign new contributor on good first issue
runs-on: ubuntu-latest
if: >-
github.event.issue.state == 'open' &&
github.event.comment.user.type != 'Bot' &&
!github.event.issue.pull_request
steps:
- uses: actions/checkout@v5
- name: Decide assign + notice body
id: decide
env:
GITHUB_EVENT_PATH: ${{ github.event_path }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: python3 .github/scripts/good_first_issue_assign.py
- name: Add assignee and notify
if: steps.decide.outputs.should_assign == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ISSUE_NUMBER: ${{ github.event.issue.number }}
REPO_FULL: ${{ github.repository }}
COMMENTER_LOGIN: ${{ github.event.comment.user.login }}
run: |
gh issue edit "${ISSUE_NUMBER}" \
--repo "${REPO_FULL}" \
--add-assignee "${COMMENTER_LOGIN}"
gh issue comment "${ISSUE_NUMBER}" \
--repo "${REPO_FULL}" \
--body-file assign_comment.md
@@ -0,0 +1,46 @@
name: Greptile PR reminder
# One-time nudge when a PR is opened or reopened. Uses pull_request_target so the token can
# comment on PRs from forks. Does not checkout the PR head — only posts a static reminder (see
# SECURITY: pull_request_target guidance in GitHub Actions docs).
on:
pull_request_target:
types: [opened, reopened]
permissions:
contents: read
pull-requests: write
jobs:
remind:
name: Post Greptile review reminder
runs-on: ubuntu-latest
if: github.event.pull_request.user.type != 'Bot'
steps:
- name: Comment with Greptile guidance
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
BASE_REF: ${{ github.event.pull_request.base.ref }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
set -euo pipefail
base_ref="${BASE_REF}"
contributing_url="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/blob/${base_ref}/CONTRIBUTING.md#greptile-code-review"
pr="${PR_NUMBER}"
repo="${GITHUB_REPOSITORY}"
{
echo '### Greptile code review'
echo ''
echo "This repo uses **Greptile** for automated review. Before merge, aim for **Confidence Score: 5/5** with **zero unresolved** review threads — see [CONTRIBUTING.md](${contributing_url})."
echo ''
echo '**Run a review** — add a PR comment with:'
echo ''
echo '```'
echo '@greptile review'
echo '```'
echo ''
echo 'Give it **~5-10 minutes** (sometimes longer) for results, then fix feedback and re-trigger until you reach **Confidence Score: 5/5**.'
echo ''
echo '**Optional:** automate with the [greploop skill](https://skills.sh/greptileai/skills/greploop).'
} > comment.md
gh pr comment "$pr" --repo "$repo" --body-file comment.md
+50
View File
@@ -0,0 +1,50 @@
name: Hermes Tests
on:
pull_request:
paths:
- "tests/synthetic/hermes_rca/**"
- "tests/e2e/hermes/**"
- "tests/synthetic/mock_hermes_backend/**"
- "tools/HermesSessionEvidenceTool/**"
- ".github/workflows/hermes-tests.yml"
- "Makefile"
- "pytest.ini"
schedule:
- cron: "17 3 * * *"
workflow_dispatch:
jobs:
synthetic:
name: Hermes synthetic offline
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Run Hermes synthetic tests
run: uv run pytest tests/synthetic/hermes_rca -q
- name: Run Hermes offline suite
run: uv run python -m tests.synthetic.hermes_rca.run_suite --offline-only
e2e:
name: Hermes e2e/meta
runs-on: ubuntu-latest
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Run Hermes e2e/meta tests
run: uv run pytest tests/e2e/hermes -q
@@ -0,0 +1,160 @@
name: Interactive Shell Live (PR + post-merge)
on:
pull_request:
branches: [main]
paths:
- "core/agent/**"
- "core/agent_harness/**"
- "tests/core/agent/**"
- "surfaces/interactive_shell/**"
- "core/runtime/**"
- "tools/**"
- "integrations/**"
- "pytest.ini"
- "pyproject.toml"
- "uv.lock"
- ".github/workflows/interactive-shell-live.yml"
push:
branches: [main]
paths:
- "core/agent/**"
- "core/agent_harness/**"
- "tests/core/agent/**"
- "surfaces/interactive_shell/**"
- "core/runtime/**"
- "tools/**"
- "integrations/**"
- "pytest.ini"
- "pyproject.toml"
- "uv.lock"
- ".github/workflows/interactive-shell-live.yml"
workflow_dispatch:
inputs:
shard_indexes:
description: "Comma-separated shard indexes to run (0-7)"
required: false
default: "0,1,2,3,4,5,6,7"
permissions:
contents: read
concurrency:
group: interactive-shell-live-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
turn-checks:
name: turn-checks (no-LLM)
runs-on: ubuntu-latest
timeout-minutes: 10
env:
OPENSRE_DISABLE_KEYRING: "1"
PYTHONUTF8: "1"
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Deterministic turn + fixture integrity
run: >-
uv run python -m pytest -n auto -v
tests/core/agent/
-m "not live_llm"
turn-live:
# Run on push to main, manual dispatch, and same-repo PRs. Fork PRs have no
# secrets, so they skip the live job (the no-LLM turn-checks job above
# still gates them) instead of failing the credential-validation step.
if: >-
github.repository == 'Tracer-Cloud/opensre' &&
(github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false)
name: turn-live shard ${{ matrix.shard_index }}
runs-on: ubuntu-latest
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
shard_index: ${{ fromJSON(github.event_name == 'workflow_dispatch' && format('[{0}]', github.event.inputs.shard_indexes) || '[0, 1, 2, 3, 4, 5, 6, 7]') }}
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
LLM_PROVIDER: openai
OPENSRE_DISABLE_KEYRING: "1"
TURN_SHARD_TOTAL: "8"
TURN_SHARD_INDEX: ${{ matrix.shard_index }}
PYTHONUTF8: "1"
# @live gather scenarios (316, 333335, 337) require these secrets in CI.
# Missing values fail the job in preflight and via skip_or_fail in tests.
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
SENTRY_ORG_SLUG: tracer-30
SENTRY_PROJECT_SLUG: python
SENTRY_URL: https://sentry.io
DD_API_KEY: ${{ secrets.DD_API_KEY }}
DD_APP_KEY: ${{ secrets.DD_APP_KEY }}
DD_SITE: datadoghq.com
GRAFANA_READ_TOKEN: ${{ secrets.GRAFANA_READ_TOKEN }}
GRAFANA_INSTANCE_URL: ${{ secrets.GRAFANA_INSTANCE_URL }}
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Validate live turn credentials
run: |
missing=()
if [ -z "${OPENAI_API_KEY}" ]; then
missing+=(OPENAI_API_KEY)
fi
if [ -z "${SENTRY_AUTH_TOKEN}" ]; then
missing+=(SENTRY_AUTH_TOKEN)
fi
if [ -z "${DD_API_KEY}" ]; then
missing+=(DD_API_KEY)
fi
if [ -z "${DD_APP_KEY}" ]; then
missing+=(DD_APP_KEY)
fi
if [ ${#missing[@]} -gt 0 ]; then
echo "::error::Missing secrets for live turn tests: ${missing[*]}"
exit 1
fi
- name: Run sharded live turn suite
run: >-
uv run python -m pytest -n auto -v
-m live_llm
tests/core/agent/test_turn_scenarios.py
-k "test_live_turn_execution_oracle or test_live_action_planning"
+678
View File
@@ -0,0 +1,678 @@
name: Release
on:
push:
branches: [main]
paths-ignore:
- "docs/**"
- "tests/benchmarks/cloudopsbench/infra/**"
- "platform/deployment/install-proxy/**"
- "tests/e2e/kubernetes/helm/scripts/**"
- "tests/**"
- "**/*.md"
- "**/*.mdx"
- ".claude/**"
schedule:
- cron: "30 0 * * *"
workflow_dispatch:
inputs:
channel:
description: "Release channel to publish."
required: false
default: release
type: choice
options:
- release
- main
tag:
description: "Optional tag to release (e.g. v0.1.2026.6.26 or v0.1). Ignored for the main channel."
required: false
type: string
permissions:
contents: write
actions: read
models: read
concurrency:
group: release-${{ github.event_name == 'push' && 'main' || (github.event_name == 'workflow_dispatch' && inputs.channel == 'main' && 'main') || 'stable' }}
cancel-in-progress: true
jobs:
prepare:
if: github.repository == 'Tracer-Cloud/opensre'
runs-on: ubuntu-latest
outputs:
channel: ${{ steps.meta.outputs.channel }}
tag_name: ${{ steps.meta.outputs.tag_name }}
version_name: ${{ steps.meta.outputs.version_name }}
steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Resolve release metadata
id: meta
env:
EVENT_NAME: ${{ github.event_name }}
DISPATCH_CHANNEL: ${{ inputs.channel }}
DISPATCH_TAG: ${{ inputs.tag }}
shell: bash
run: |
set -euo pipefail
channel="release"
if [ "$EVENT_NAME" = "push" ]; then
channel="main"
fi
if [ "$EVENT_NAME" = "workflow_dispatch" ] && [ "$DISPATCH_CHANNEL" = "main" ]; then
channel="main"
fi
if [ "$channel" = "main" ]; then
if [ "$EVENT_NAME" = "workflow_dispatch" ] && [ -n "$DISPATCH_TAG" ]; then
echo "The main channel does not accept a custom tag." >&2
exit 1
fi
year="$(date -u +%Y)"
month="$(date -u +%-m)"
day="$(date -u +%-d)"
short_sha="${GITHUB_SHA:0:7}"
main_version="0.1.${year}.${month}.${day}+main.${short_sha}"
echo "channel=main" >> "$GITHUB_OUTPUT"
echo "tag_name=main-build" >> "$GITHUB_OUTPUT"
echo "version_name=${main_version}" >> "$GITHUB_OUTPUT"
exit 0
fi
if [ "$EVENT_NAME" = "workflow_dispatch" ] && [ -n "$DISPATCH_TAG" ]; then
tag_name="$DISPATCH_TAG"
else
year="$(date -u +%Y)"
month="$(date -u +%-m)"
day="$(date -u +%-d)"
# Keep the v0.1 prefix so the tag makes clear the product is still
# v0.1, while the date stays useful (e.g. v0.1.2026.6.26).
tag_name="v0.1.${year}.${month}.${day}"
fi
echo "channel=release" >> "$GITHUB_OUTPUT"
echo "tag_name=${tag_name}" >> "$GITHUB_OUTPUT"
echo "version_name=${tag_name#v}" >> "$GITHUB_OUTPUT"
verify:
runs-on: ubuntu-latest
needs: prepare
if: github.event_name != 'push'
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Lint
run: make lint
- name: Type check
run: make typecheck
- name: CLI smoke tests
run: make test-cli-smoke
build-python-dist:
if: needs.prepare.outputs.channel == 'release' && needs.verify.result == 'success'
runs-on: ubuntu-latest
needs: [verify, prepare]
env:
TAG_NAME: ${{ needs.prepare.outputs.tag_name }}
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Sync release version
shell: bash
run: python platform/packaging/sync_release_version.py --tag "$TAG_NAME"
- name: Build Python distributions
run: |
uv sync --frozen --extra release-dist
uv run python -m build
uv run twine check dist/*
- name: Verify Python distribution version
shell: bash
run: |
set -euo pipefail
VERSION="${TAG_NAME#v}"
test -f "dist/opensre-${VERSION}.tar.gz"
test -f "dist/opensre-${VERSION}-py3-none-any.whl"
- name: Upload Python distributions
uses: actions/upload-artifact@v4
with:
name: release-python-dist
path: dist/*
if-no-files-found: error
build-binaries:
if: always() && (needs.prepare.outputs.channel == 'main' || needs.verify.result == 'success')
needs: [verify, prepare]
strategy:
fail-fast: false
matrix:
include:
- runner: ubuntu-22.04
target: linux-x64
binary_name: opensre
archive_ext: tar.gz
pyinstaller_mode: onedir
- runner: ubuntu-22.04-arm
target: linux-arm64
binary_name: opensre
archive_ext: tar.gz
pyinstaller_mode: onedir
- runner: macos-15-intel
target: darwin-x64
binary_name: opensre
archive_ext: tar.gz
pyinstaller_mode: onedir
- runner: macos-latest
target: darwin-arm64
binary_name: opensre
archive_ext: tar.gz
pyinstaller_mode: onedir
- runner: windows-latest
target: windows-x64
binary_name: opensre.exe
archive_ext: zip
pyinstaller_mode: onefile
# windows-arm64 is currently excluded from the default release matrix:
# cryptography does not publish win_arm64 wheels, so dependency install
# falls back to a source build that requires an OpenSSL toolchain on the
# GitHub-hosted runner.
runs-on: ${{ matrix.runner }}
env:
RELEASE_CHANNEL: ${{ needs.prepare.outputs.channel }}
TAG_NAME: ${{ needs.prepare.outputs.tag_name }}
VERSION_NAME: ${{ needs.prepare.outputs.version_name }}
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Sync binary version
shell: bash
run: python platform/packaging/sync_release_version.py --version "$VERSION_NAME"
- name: Install binary build dependencies
shell: bash
run: uv sync --frozen --extra release-binary
- name: Stage stdlib platform module for bundling
shell: bash
run: |
set -euo pipefail
# The first-party ``platform`` package shadows the stdlib ``platform``
# module. PyInstaller does not lay out the stdlib as loose ``.py`` files,
# so bundle a copy of the genuine module that platform/__init__.py can
# load from ``sys._MEIPASS`` at runtime (see _FROZEN_STDLIB_DIR).
rm -rf .stdlib_vendor
mkdir -p .stdlib_vendor
uv run python -c "import os, shutil, sysconfig; src = os.path.join(sysconfig.get_path('stdlib'), 'platform.py'); shutil.copy(src, os.path.join('.stdlib_vendor', 'platform.py')); print('Staged stdlib platform from ' + src)"
- name: Build binary
run: >-
uv run pyinstaller surfaces/cli/__main__.py
--name opensre
--${{ matrix.pyinstaller_mode }}
--clean
--noconfirm
--collect-data surfaces.cli
--collect-data config
--copy-metadata opensre
--collect-data litellm
--hidden-import tiktoken_ext
--hidden-import tiktoken_ext.openai_public
--collect-submodules integrations
--collect-submodules surfaces.interactive_shell
--collect-submodules tools
--add-data "platform:platform"
--add-data ".stdlib_vendor:_opensre_stdlib_platform"
- name: Smoke test binary (Unix)
if: runner.os != 'Windows'
shell: bash
run: |
set -uo pipefail
# Capture the exit status explicitly so a crashing binary still prints
# its stdout/stderr (under `set -e` the failed substitution aborted the
# step before the output was ever shown, hiding the real error).
# onefile builds place the executable at ./dist/<name>; onedir builds
# produce a ./dist/opensre/ directory containing the executable. A bare
# `-x` test matches the onedir directory too (dirs carry the search
# bit), so require a regular file before treating it as the binary.
if [ -f "./dist/${{ matrix.binary_name }}" ] && [ -x "./dist/${{ matrix.binary_name }}" ]; then
BINARY_PATH="./dist/${{ matrix.binary_name }}"
else
BINARY_PATH="./dist/opensre/${{ matrix.binary_name }}"
fi
set +e
VERSION_OUTPUT="$("$BINARY_PATH" --version 2>&1)"
VERSION_STATUS=$?
set -e
printf '%s\n' "$VERSION_OUTPUT"
if [ "$VERSION_STATUS" -ne 0 ]; then
printf '::error::%s --version exited with status %s\n' "${{ matrix.binary_name }}" "$VERSION_STATUS" >&2
exit "$VERSION_STATUS"
fi
case "$VERSION_OUTPUT" in
*"$VERSION_NAME"*) ;;
*)
printf 'Binary version mismatch: expected %s but saw %s\n' "$VERSION_NAME" "$VERSION_OUTPUT" >&2
exit 1
;;
esac
"$BINARY_PATH" -h >/dev/null
LITELLM_DATA="./dist/opensre/_internal/litellm/model_prices_and_context_window_backup.json"
if [ ! -f "$LITELLM_DATA" ]; then
echo "LiteLLM package data missing from Unix onedir bundle: ${LITELLM_DATA}" >&2
exit 1
fi
- name: Check Linux binary glibc compatibility
if: runner.os == 'Linux'
shell: bash
run: |
set -euo pipefail
max_glibc="$(
find dist/opensre -type f \( -name opensre -o -name '*.so' -o -name '*.so.*' \) -print0 \
| xargs -0 strings 2>/dev/null \
| grep -Eo 'GLIBC_[0-9]+\.[0-9]+(\.[0-9]+)?' \
| sort -Vu \
| tail -n 1 \
|| true
)"
echo "Max required glibc symbol: ${max_glibc:-none}"
if [ -n "$max_glibc" ] && [ "$(printf '%s\n' "$max_glibc" GLIBC_2.35 | sort -V | tail -n 1)" != "GLIBC_2.35" ]; then
echo "Linux binary requires ${max_glibc}; pin the Linux runner back to Ubuntu 22.04 or lower dependency wheel requirements." >&2
exit 1
fi
- name: Smoke test binary (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
$versionOutput = & ".\dist\${{ matrix.binary_name }}" --version 2>&1 | Out-String
$versionText = $versionOutput.Trim()
Write-Host $versionText
$expectedVersion = $env:VERSION_NAME
if ($versionText -notmatch [regex]::Escape($expectedVersion)) {
throw "Binary version mismatch. Expected '$expectedVersion' but saw '$versionText'."
}
& ".\dist\${{ matrix.binary_name }}" -h | Out-Null
- name: Package binary archive (Unix)
if: runner.os != 'Windows'
shell: bash
run: |
set -euo pipefail
if [ "$RELEASE_CHANNEL" = "release" ]; then
ASSET_BASENAME="opensre_${TAG_NAME#v}_${{ matrix.target }}"
else
ASSET_BASENAME="opensre_main_${{ matrix.target }}"
fi
tar -C dist -czf "${ASSET_BASENAME}.tar.gz" "${{ matrix.binary_name }}"
shasum -a 256 "${ASSET_BASENAME}.tar.gz" > "${ASSET_BASENAME}.tar.gz.sha256"
- name: Package binary archive (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
if ($env:RELEASE_CHANNEL -eq "release") {
$assetBaseName = "opensre_$($env:TAG_NAME.TrimStart('v'))_${{ matrix.target }}"
} else {
$assetBaseName = "opensre_main_${{ matrix.target }}"
}
Compress-Archive -Path "dist\${{ matrix.binary_name }}" -DestinationPath "${assetBaseName}.zip"
$hash = (Get-FileHash -Algorithm SHA256 "${assetBaseName}.zip").Hash.ToLowerInvariant()
Set-Content -Path "${assetBaseName}.zip.sha256" -Value "$hash ${assetBaseName}.zip"
- name: Upload binary archive
uses: actions/upload-artifact@v4
with:
name: ${{ env.RELEASE_CHANNEL == 'main' && 'main-' || '' }}release-${{ matrix.target }}
path: |
opensre_*_${{ matrix.target }}.${{ matrix.archive_ext }}
opensre_*_${{ matrix.target }}.${{ matrix.archive_ext }}.sha256
if-no-files-found: error
publish-release:
if: needs.prepare.outputs.channel == 'release'
runs-on: ubuntu-latest
needs:
- prepare
- build-python-dist
- build-binaries
steps:
- uses: actions/checkout@v5
with:
ref: ${{ github.event.repository.default_branch }}
fetch-depth: 0
- name: Download release artifacts
uses: actions/download-artifact@v4
with:
pattern: release-*
path: release-assets
merge-multiple: true
- name: Resolve release context
id: release_ctx
env:
TAG_NAME: ${{ needs.prepare.outputs.tag_name }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
shell: bash
run: |
set -euo pipefail
default_branch="${{ github.event.repository.default_branch }}"
git fetch origin "$default_branch" --tags --force
target_sha="$(git rev-parse "origin/$default_branch")"
previous_tag="$(
git tag --list 'v0.1.[0-9][0-9][0-9][0-9].[0-9]*.[0-9]*' --sort=-v:refname \
| grep -v -x "$TAG_NAME" \
| head -n 1 \
|| true
)"
range_spec="$target_sha"
if [ -n "$previous_tag" ]; then
range_spec="${previous_tag}..${target_sha}"
fi
{
printf 'target_sha=%s\n' "$target_sha"
printf 'previous_tag=%s\n' "$previous_tag"
printf 'range_spec=%s\n' "$range_spec"
} >> "$GITHUB_OUTPUT"
- name: Create release notes
env:
TAG_NAME: ${{ needs.prepare.outputs.tag_name }}
RANGE_SPEC: ${{ steps.release_ctx.outputs.range_spec }}
PREVIOUS_TAG: ${{ steps.release_ctx.outputs.previous_tag }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
shell: bash
run: |
set -euo pipefail
{
echo "## Changelog"
echo
if [ -n "$PREVIOUS_TAG" ]; then
echo "_Changes since ${PREVIOUS_TAG}_"
else
echo "_Changes up to ${TAG_NAME}_"
fi
echo
git log "$RANGE_SPEC" --no-merges -n 100 --pretty='- %s (%h) — %an'
echo
} > GENERATED_CHANGELOG.md
# Summarize commits into prose via GitHub Models (gpt-4o-mini).
# Falls back to raw changelog if the API is unavailable.
raw_commits="$(git log "$RANGE_SPEC" --no-merges --pretty='%s' | head -40 || true)"
if [ -n "$raw_commits" ]; then
api_body="$(jq -n --arg commits "$raw_commits" '{
model: "gpt-4o-mini",
messages: [
{
role: "system",
content: "You are writing a Discord release announcement for an open-source SRE CLI tool called opensre. Summarize the git commits into 2-4 short, punchy prose sentences — no bullet points, no headers, no markdown. Write in present tense, active voice. Focus on what users will notice: new features, fixes, performance, integrations. Be specific but concise."
},
{role: "user", content: ("Commits:\n" + $commits)}
],
max_tokens: 300,
temperature: 0.4
}')"
curl -sS --max-time 20 \
-H "Authorization: Bearer $GITHUB_TOKEN" \
-H "Content-Type: application/json" \
"https://models.inference.ai.azure.com/chat/completions" \
-d "$api_body" 2>/dev/null \
| jq -r '.choices[0].message.content // empty' 2>/dev/null \
| tr -d '\r' > DISCORD_NARRATIVE.md || true
if [ -s DISCORD_NARRATIVE.md ]; then
echo "LLM narrative generated ($(wc -c < DISCORD_NARRATIVE.md) bytes)."
else
echo "LLM summary unavailable; Discord will use raw changelog."
rm -f DISCORD_NARRATIVE.md
fi
fi
CB='```'
{
printf '## Install\n\n'
printf '### cURL (macOS / Linux)\n\n%sbash\ncurl -fsSL https://install.opensre.com | bash -s -- --release --version %s\n%s\n\n' "$CB" "${TAG_NAME#v}" "$CB"
printf '### Homebrew (macOS / Linux)\n\n%sbash\nbrew tap tracer-cloud/tap\nbrew install tracer-cloud/tap/opensre\n%s\n\n' "$CB" "$CB"
printf '### PowerShell (Windows)\n\n%spowershell\n$env:OPENSRE_INSTALL_CHANNEL="release"; $env:OPENSRE_VERSION="%s"; irm https://install.opensre.com | iex\n%s\n\n' "$CB" "${TAG_NAME#v}" "$CB"
printf '### Python\n\n%sbash\npipx install opensre\n%s\n\n' "$CB" "$CB"
} > RELEASE_NOTES.md
cat GENERATED_CHANGELOG.md >> RELEASE_NOTES.md
- name: Create GitHub release
id: github_release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG_NAME: ${{ needs.prepare.outputs.tag_name }}
TARGET_SHA: ${{ steps.release_ctx.outputs.target_sha }}
shell: bash
run: |
set -euo pipefail
if gh release view "$TAG_NAME" --repo "${{ github.repository }}" >/dev/null 2>&1; then
echo "created=false" >> "$GITHUB_OUTPUT"
echo "Release $TAG_NAME already exists; nothing to do."
exit 0
fi
VERSION="${TAG_NAME#v}"
# Every release-channel publish is the newest stable build at this
# point, so mark it as GitHub's Latest. The title keeps the full
# version (e.g. "OpenSRE 0.1.2026.6.26") so the v0.1 line is clear.
gh release create "$TAG_NAME" \
release-assets/* \
--repo "${{ github.repository }}" \
--target "$TARGET_SHA" \
--title "OpenSRE ${VERSION}" \
--notes-file RELEASE_NOTES.md \
--latest
echo "created=true" >> "$GITHUB_OUTPUT"
- name: Announce on Discord
if: steps.github_release.outputs.created == 'true' && !github.event.repository.fork && !github.event.repository.private
continue-on-error: true
env:
DISCORD_WEBHOOK_URL_UPDATE: ${{ secrets.DISCORD_WEBHOOK_URL_UPDATE }}
DISCORD_RELEASES_ROLE_ID: ${{ secrets.DISCORD_RELEASES_ROLE_ID }}
DISCORD_RELEASE_LOGO_EMOJI: ${{ secrets.DISCORD_RELEASE_LOGO_EMOJI }}
DISCORD_RELEASE_LOGO_URL: ${{ secrets.DISCORD_RELEASE_LOGO_URL }}
RELEASE_TAG: ${{ needs.prepare.outputs.tag_name }}
RELEASE_URL: ${{ github.server_url }}/${{ github.repository }}/releases/tag/${{ needs.prepare.outputs.tag_name }}
shell: bash
run: |
set -euo pipefail
if [ -z "${DISCORD_WEBHOOK_URL_UPDATE:-}" ]; then
echo "DISCORD_WEBHOOK_URL_UPDATE is not set; skipping Discord announcement."
exit 0
fi
# Prefer LLM-generated narrative; fall back to raw changelog.
if [ -f DISCORD_NARRATIVE.md ] && [ -s DISCORD_NARRATIVE.md ]; then
changelog_file="DISCORD_NARRATIVE.md"
elif [ -f GENERATED_CHANGELOG.md ]; then
changelog_file="GENERATED_CHANGELOG.md"
else
echo "No changelog available." > /tmp/discord_fallback.md
changelog_file="/tmp/discord_fallback.md"
fi
# Use an external Python script to build the JSON payload.
# This avoids jq issues with multiline LLM output containing control characters.
payload="$(CHANGELOG_FILE="$changelog_file" python3 .github/scripts/build-discord-payload.py)"
curl --fail -sS --max-time 30 -X POST -H "Content-Type: application/json" \
-d "$payload" "$DISCORD_WEBHOOK_URL_UPDATE"
- name: Sync Homebrew tap formula
if: steps.github_release.outputs.created == 'true'
continue-on-error: true
env:
HOMEBREW_TAP_GITHUB_TOKEN: ${{ secrets.HOMEBREW_TAP_GITHUB_TOKEN }}
VERSION: ${{ needs.prepare.outputs.tag_name }}
ASSET_DIR: release-assets
shell: bash
run: |
set -euo pipefail
if [ -z "${HOMEBREW_TAP_GITHUB_TOKEN:-}" ]; then
echo "HOMEBREW_TAP_GITHUB_TOKEN is not set; skipping Homebrew tap sync."
exit 0
fi
export VERSION="${VERSION#v}"
bash .github/scripts/sync-homebrew-tap-formula.sh
publish-main-release:
if: always() && needs.prepare.outputs.channel == 'main' && needs.build-binaries.result == 'success'
runs-on: ubuntu-latest
needs:
- prepare
- build-binaries
steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0
# The rolling `main-build` tag is force-moved to each new main commit
# below. Whenever main has advanced past a commit that touches
# .github/workflows/**, the default GITHUB_TOKEN (the
# github-actions[bot] GitHub App) is refused with "refusing to allow a
# GitHub App to create or update workflow ... without `workflows`
# permission" — and that scope cannot be granted via the permissions
# block. MAIN_BUILD_TAG_TOKEN must be a token that carries workflow
# write access (classic PAT with `repo` + `workflow`, a fine-grained
# PAT with Contents + Workflows: write, or a GitHub App token). It is
# persisted so the tag push below authenticates with it. Falls back to
# GITHUB_TOKEN so the rest of the job still runs if the secret is unset
# (the tag push will then fail as before until the secret is added).
token: ${{ secrets.MAIN_BUILD_TAG_TOKEN || secrets.GITHUB_TOKEN }}
persist-credentials: true
- name: Download main release artifacts
uses: actions/download-artifact@v4
with:
pattern: main-release-*
path: main-release-assets
merge-multiple: true
- name: Create release notes
env:
VERSION_NAME: ${{ needs.prepare.outputs.version_name }}
shell: bash
run: |
set -euo pipefail
short_sha="$(printf '%s' "$GITHUB_SHA" | cut -c1-7)"
built_at="$(date -u +"%Y-%m-%d %H:%M UTC")"
CB='```'
{
printf '## Main build\n\nRolling binary build from `main`.\n\n'
printf -- '- Version: `%s`\n- Commit: `%s`\n- Built: %s\n\n' "$VERSION_NAME" "$short_sha" "$built_at"
printf '### Install\n\nmacOS / Linux:\n\n%sbash\ncurl -fsSL https://install.opensre.com | bash\n%s\n\n' "$CB" "$CB"
printf 'Equivalent explicit main channel:\n\n%sbash\ncurl -fsSL https://install.opensre.com | bash -s -- --main\n%s\n\n' "$CB" "$CB"
printf 'Windows:\n\n%spowershell\nirm https://install.opensre.com | iex\n%s\n' "$CB" "$CB"
} > MAIN_RELEASE_NOTES.md
- name: Move main build tag to the latest commit
shell: bash
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git tag -f "${{ needs.prepare.outputs.tag_name }}" "$GITHUB_SHA"
git push origin "refs/tags/${{ needs.prepare.outputs.tag_name }}" --force
- name: Publish rolling main release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
shell: bash
run: |
set -euo pipefail
tag_name="${{ needs.prepare.outputs.tag_name }}"
if gh release view "$tag_name" --repo "${{ github.repository }}" >/dev/null 2>&1; then
gh release upload "$tag_name" main-release-assets/* --repo "${{ github.repository }}" --clobber
gh release edit "$tag_name" \
--repo "${{ github.repository }}" \
--title "Main" \
--notes-file MAIN_RELEASE_NOTES.md
exit 0
fi
gh release create "$tag_name" \
main-release-assets/* \
--repo "${{ github.repository }}" \
--target "$GITHUB_SHA" \
--title "Main" \
--notes-file MAIN_RELEASE_NOTES.md \
--prerelease
@@ -0,0 +1,84 @@
name: Synthetic Deterministic Tests
# Regression guardrail: run every offline (no-LLM) synthetic test on every
# PR and merge so pipeline refactors can't silently break investigations.
#
# Tests that need a live LLM are excluded via the marker filter below.
# Those suites live in interactive-shell-live.yml and hermes-tests.yml instead.
on:
push:
branches: [main]
paths:
- "surfaces/**"
- "config/**"
- "core/**"
- "integrations/**"
- "platform/**"
- "tools/**"
- "tests/synthetic/**"
- "pyproject.toml"
- "uv.lock"
- "pytest.ini"
- ".github/workflows/synthetic-deterministic.yml"
pull_request:
branches: [main]
paths:
- "surfaces/**"
- "config/**"
- "core/**"
- "integrations/**"
- "platform/**"
- "tools/**"
- "tests/synthetic/**"
- "pyproject.toml"
- "uv.lock"
- "pytest.ini"
- ".github/workflows/synthetic-deterministic.yml"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: synthetic-det-${{ github.ref }}
cancel-in-progress: true
jobs:
offline:
name: Synthetic offline (deterministic)
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install dependencies
run: uv sync --frozen --extra dev
- name: Run deterministic synthetic tests
# Excludes markers that require a live LLM:
# synthetic — full RCA scenario suites (EKS, RDS, Hermes, Dagster …)
# axis2 — adversarial scenario suites (also call the LLM)
# live_llm — explicit live-credential gate
# e2e — requires live infrastructure
run: |
uv run pytest tests/synthetic \
-m "not synthetic and not live_llm and not e2e and not axis2" \
-q
- name: Run hermes_rca offline scenario suite
run: uv run python -m tests.synthetic.hermes_rca.run_suite --offline-only
+282
View File
@@ -0,0 +1,282 @@
name: terraform-bench (plan)
# Runs terraform fmt + init + validate + plan against tests/benchmarks/cloudopsbench/infra/ on every PR
# that touches the bench infra. Posts the plan output as a sticky PR comment so
# reviewers can see what would change before approving.
#
# Never applies — apply is developer-local in v1 (see tests/benchmarks/cloudopsbench/infra/README.md).
# When apply-from-CI is desirable later, add a separate workflow that runs on
# merge-to-main with a different (higher-privileged) AWS role.
on:
pull_request:
paths:
- "tests/benchmarks/cloudopsbench/infra/**"
- ".github/workflows/terraform-bench.yml"
workflow_dispatch:
permissions:
contents: read
pull-requests: write
id-token: write # required for AWS OIDC role assumption
concurrency:
group: terraform-bench-${{ github.workflow }}-${{ github.head_ref || github.ref }}
cancel-in-progress: true
jobs:
plan:
name: terraform plan
if: github.repository == 'Tracer-Cloud/opensre' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository)
runs-on: ubuntu-latest
defaults:
run:
working-directory: tests/benchmarks/cloudopsbench/infra
env:
AWS_REGION: us-east-1
# tfplan + plan.txt artifacts are not uploaded; plan output goes into the
# PR comment. If you want a downloadable plan binary, add an
# actions/upload-artifact step after `terraform show`.
TF_IN_AUTOMATION: "true"
TF_INPUT: "0"
steps:
- name: Checkout
uses: actions/checkout@v5
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: "1.7.5"
# Wrapper MUST stay enabled (default) — it's what populates
# `steps.<id>.outputs.stdout` for the `terraform show` step that
# feeds the sticky PR comment. Disabling it makes the comment
# render with a blank plan body. See greptile review on initial
# PR for details.
- name: Configure AWS credentials (OIDC role assumption)
# No long-lived AWS keys. The role is provisioned by tests/benchmarks/cloudopsbench/infra/
# Terraform; ARN is hardcoded here because Terraform outputs aren't
# available before the workflow can run. Account ID is the
# tracer-cloud account.
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/opensre-bench-terraform-plan
role-session-name: github-actions-terraform-plan-${{ github.run_id }}
aws-region: us-east-1
- name: terraform fmt
id: fmt
run: terraform fmt -check -recursive -no-color
continue-on-error: true
- name: terraform init
id: init
run: terraform init -no-color
- name: terraform validate
id: validate
run: terraform validate -no-color
- name: terraform plan
id: plan
run: terraform plan -no-color -input=false -lock=false -out=tfplan
continue-on-error: true
- name: terraform show
id: show
if: steps.plan.outcome == 'success'
run: terraform show -no-color tfplan
- name: Post plan to PR
if: github.event_name == 'pull_request' && always()
uses: actions/github-script@v7
env:
PLAN_STDOUT: ${{ steps.show.outputs.stdout }}
FMT_OUTCOME: ${{ steps.fmt.outcome }}
INIT_OUTCOME: ${{ steps.init.outcome }}
VALIDATE_OUTCOME: ${{ steps.validate.outcome }}
PLAN_OUTCOME: ${{ steps.plan.outcome }}
WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
with:
script: |
const fs = require('fs');
const MARKER = '<!-- terraform-bench-plan-comment -->';
const fmt = process.env.FMT_OUTCOME;
const init = process.env.INIT_OUTCOME;
const validate = process.env.VALIDATE_OUTCOME;
const plan = process.env.PLAN_OUTCOME;
const url = process.env.WORKFLOW_URL;
const icon = (outcome) => outcome === 'success' ? '✅' :
outcome === 'failure' ? '❌' :
outcome === 'skipped' ? '⏭️' : '⚠️';
let planBody = process.env.PLAN_STDOUT || '';
// GitHub PR comments cap at 65536 chars. Reserve ~1000 for surrounding text.
const MAX = 60000;
let truncated = false;
if (planBody.length > MAX) {
planBody = planBody.slice(-MAX);
truncated = true;
}
const body = [
MARKER,
'### terraform-bench plan',
'',
'| step | outcome |',
'| --- | --- |',
`| fmt | ${icon(fmt)} \`${fmt}\` |`,
`| init | ${icon(init)} \`${init}\` |`,
`| validate | ${icon(validate)} \`${validate}\` |`,
`| plan | ${icon(plan)} \`${plan}\` |`,
'',
fmt !== 'success' ? '> Run `terraform fmt -recursive` in `tests/benchmarks/cloudopsbench/infra/` to fix formatting.' : '',
'',
plan === 'success'
? `<details><summary>Plan output${truncated ? ' (truncated — see <a href="' + url + '">workflow logs</a> for full output)' : ''}</summary>\n\n\`\`\`hcl\n${planBody}\n\`\`\`\n\n</details>`
: `Plan did not run successfully — see [workflow logs](${url}) for details.`,
'',
`_Updated by [\`terraform-bench.yml\`](${url})._`,
].join('\n');
// Find an existing comment to update, else create a new one.
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body && c.body.includes(MARKER));
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body,
});
}
- name: Fail if any step errored
if: |
steps.fmt.outcome == 'failure' ||
steps.init.outcome == 'failure' ||
steps.validate.outcome == 'failure' ||
steps.plan.outcome == 'failure'
run: |
echo "::error::One or more terraform steps failed. See PR comment + workflow logs."
exit 1
# ---------------------------------------------------------------------- #
# Security + lint scanners — run in parallel with plan. #
# #
# Suppression policy (intentional skips) is defined per-tool: #
# - checkov: tests/benchmarks/cloudopsbench/infra/.checkov.yaml #
# - tfsec: tests/benchmarks/cloudopsbench/infra/.tfsec/config.yml #
# - tflint: tests/benchmarks/cloudopsbench/infra/.tflint.hcl #
# Each suppression includes a justification at the source. Day-one #
# findings should be REAL findings — change configs, not the workflow, #
# if a rule needs to be skipped. #
# #
# checkov + tfsec hard-fail (security blocks PRs). tflint soft-fails #
# because its rules cover code quality / deprecations, not security — #
# we want visibility without blocking. #
# ---------------------------------------------------------------------- #
tflint:
name: tflint
runs-on: ubuntu-latest
defaults:
run:
working-directory: tests/benchmarks/cloudopsbench/infra
steps:
- name: Checkout
uses: actions/checkout@v5
- name: Setup tflint
uses: terraform-linters/setup-tflint@v4
with:
tflint_version: latest
- name: tflint --init (download plugins)
run: tflint --init
- name: tflint
id: tflint
run: tflint --recursive --format compact
continue-on-error: true
- name: Surface tflint findings (non-blocking)
if: steps.tflint.outcome == 'failure'
run: |
echo "::warning::tflint reported findings. Code-quality only — does not block PR. Review in workflow logs."
tfsec:
name: tfsec
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
# tfsec via Docker, bypassing aquasecurity/tfsec-action.
#
# Why direct Docker run: the action shells out to api.github.com to
# resolve the latest release binary, and the aquasecurity org has an
# IP allow list that blocks GitHub Actions runner IPs — making the
# action effectively unusable from CI. The aquasec/tfsec image on
# Docker Hub has no such restriction.
#
# Migration path (v2): tfsec is in maintenance mode; rule set lives on
# in Trivy. Replace with `aquasec/trivy:latest config /src` when
# ready — same .tfsec/config.yml is recognized.
#
# Suppressions: tests/benchmarks/cloudopsbench/infra/.tfsec/config.yml. tfsec exits 1 on any
# finding by default, which fails the step — exactly the hard-fail
# behavior we want. Add a justified exclude to the config rather
# than disabling the step.
- name: tfsec scan (via Docker)
run: |
docker run --rm \
-v "${{ github.workspace }}/tests/benchmarks/cloudopsbench/infra:/src" \
aquasec/tfsec:latest \
/src \
--config-file=/src/.tfsec/config.yml \
--no-color
checkov:
name: checkov
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: "3.12"
# Suppressions live in tests/benchmarks/cloudopsbench/infra/.checkov.yaml. soft_fail is
# FALSE — real findings block the PR. Add a justified skip-check
# to the config rather than disabling here.
- name: checkov scan
uses: bridgecrewio/checkov-action@v12
with:
directory: tests/benchmarks/cloudopsbench/infra
framework: terraform
config_file: tests/benchmarks/cloudopsbench/infra/.checkov.yaml
quiet: true
soft_fail: false
# Re-enable SARIF upload when CodeQL alerts are wanted:
# output_format: cli,sarif
# output_file_path: console,checkov-results.sarif
@@ -0,0 +1,76 @@
name: Tracer Demo Prefect ECS
on:
workflow_dispatch:
permissions:
contents: read
jobs:
tracer-demo-prefect-ecs:
if: github.repository == 'Tracer-Cloud/opensre'
runs-on: ubuntu-latest
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
JWT_TOKEN: ${{ secrets.JWT_TOKEN }}
TRACER_ORG_ID: ${{ secrets.TRACER_ORG_ID }}
TRACER_WEB_APP_URL: ${{ secrets.TRACER_WEB_APP_URL }}
TRACER_API_URL: ${{ secrets.TRACER_API_URL }}
TRACER_INGEST_TOKEN: ${{ secrets.TRACER_INGEST_TOKEN }}
AWS_REGION: us-east-1
steps:
- uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install Python dependencies
run: uv sync --frozen --extra dev
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: Install tracer CLI
run: |
curl -sSL https://install.tracer.cloud | CLI_BRANCH=dev sh -s user_36fbN6K6FwEgJFHsQv1pUByo8K6
- name: Initialize tracer
run: |
sudo tracer init --token ${{ secrets.JWT_TOKEN }}
- name: Export Tracer run id
shell: bash
run: |
RUN_ID=$(tracer info --json | python3 -c "import sys,json; print(json.load(sys.stdin)['run']['id'])")
PIPELINE_NAME=$(tracer info --json | python3 -c "import sys,json; print(json.load(sys.stdin)['pipeline']['name'])")
ORG_SLUG=$(tracer info --json | python3 -c "import sys,json; print(json.load(sys.stdin)['pipeline']['organization'])")
echo "TRACER_RUN_ID=$RUN_ID" >> $GITHUB_ENV
echo "TRACER_PIPELINE_NAME=$PIPELINE_NAME" >> $GITHUB_ENV
echo "TRACER_ORG_SLUG=$ORG_SLUG" >> $GITHUB_ENV
# Optional but helpful: stable trace id for ingest rows
echo "TRACER_TRACE_ID=trace_$RUN_ID" >> $GITHUB_ENV
echo "Exported TRACER_RUN_ID=$RUN_ID"
- name: Run Prefect ECS Demo
continue-on-error: true
run: |
uv run python -m tests.e2e.upstream_prefect_ecs_fargate.test_agent_e2e
+85
View File
@@ -0,0 +1,85 @@
name: Trigger K8s Alert
on:
workflow_dispatch:
inputs:
verify:
description: "Verify logs in Datadog + post to Slack"
type: boolean
default: true
schedule:
- cron: "0 9 * * 1" # every Monday 9am UTC
permissions:
contents: read
jobs:
trigger-alert:
if: github.repository == 'Tracer-Cloud/opensre'
runs-on: ubuntu-latest
env:
AWS_REGION: us-east-1
DD_API_KEY: ${{ secrets.DD_API_KEY }}
DD_APP_KEY: ${{ secrets.DD_APP_KEY }}
DD_SITE: datadoghq.com
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
SLACK_DEVS_ALERTS_CHANNEL_ID: C09S7GDG60J
steps:
- uses: actions/checkout@v5
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ env.AWS_REGION }}
- name: Install uv
uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
cache-dependency-glob: |
pyproject.toml
uv.lock
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install project
run: uv sync --frozen
- name: Trigger pipeline failure via centralized config (~10s)
id: trigger
run: |
echo "trigger_epoch=$(date +%s)" >> "$GITHUB_OUTPUT"
set +e
uv run python -m tests.e2e.kubernetes.trigger_alert | tee /tmp/trigger.log
trigger_exit=${PIPESTATUS[0]}
set -e
echo "trigger_completed_epoch=$(date +%s)" >> "$GITHUB_OUTPUT"
if grep -q '"status": "accepted_timeout"' /tmp/trigger.log; then
echo "trigger_accepted_timeout=true" >> "$GITHUB_OUTPUT"
else
echo "trigger_accepted_timeout=false" >> "$GITHUB_OUTPUT"
fi
exit "$trigger_exit"
- name: Report trigger result
run: |
echo "Trigger command completed successfully"
- name: Verify Datadog + Slack (~5 min)
if: ${{ inputs.verify == true || github.event_name == 'schedule' }}
run: |
POST_TRIGGER_WAIT=""
if [ "${{ steps.trigger.outputs.trigger_accepted_timeout }}" = "true" ]; then
POST_TRIGGER_WAIT="--post-trigger-wait 90"
fi
uv run python -m tests.e2e.kubernetes.trigger_alert \
--verify-only \
--since-epoch ${{ steps.trigger.outputs.trigger_completed_epoch }} \
--dd-max-wait 300 \
$POST_TRIGGER_WAIT