tracer-cloud--opensre/docs/DEVELOPMENT.md

Development guide

Contributor-focused workflows: local setup details stay in SETUP.md at the repo root (Windows, troubleshooting, MCP/OpenClaw).

Clone and install

git clone https://github.com/Tracer-Cloud/opensre.git
cd opensre
make install

make install runs uv sync --frozen --extra dev and the analytics install helper. Use uv run opensre … from the repo root so you always hit this checkout’s .venv, not another opensre on your PATH.

opensre onboard
opensre investigate -i tests/e2e/kubernetes/fixtures/datadog_k8s_alert.json

Quality gates (same as CI)

From the repo root:

make lint          # ruff check
make format-check  # ruff format --check (CI-enforced)
make typecheck     # mypy config core gateway integrations platform surfaces tools
make test-cov      # pytest + coverage (default unit suite)

One-shot (includes heavier test-full): make check.

Before a PR, run at least make lint, make format-check, make typecheck, and make test-cov (see CONTRIBUTING.md).

Interactive shell action policy

Action-planner behavior, postprocessing transforms, compatibility seams, and the rule-extension checklist are documented in docs/interactive-shell-action-policy.md.

Package architecture

The seven first-party packages, the four-tier layering (which package may import which), the folder diagram, per-layer responsibilities, and cross-layer flows are documented in docs/ARCHITECTURE.md.

Tool registry — surface-scoped, lazy loading

Loading every vendor tool at startup was slow. A static index (tools/registry_index.py) reads tool metadata by scanning the source, without importing executors, so a turn loads only the tools it needs.

• get_registered_tools(surface) imports only that surface's tool modules.
• get_tool_descriptors(surface) returns metadata with no executor import.
• load_tool(descriptor) imports the executor, only when a tool runs.

Adding a vendor tool is a @tool/BaseTool module; the index finds it and no other vendor is imported. tests/tools/test_registry_index.py checks the index matches the imported registry exactly, so they cannot drift.

Investigation pipeline architecture

The six-stage investigation pipeline (resolve integrations → extract alert → plan → ReAct evidence loop → diagnose → deliver), the loop's guardrails (tool cap, stagnation breaker, context budget, duplicate detection), and diagrams are documented in docs/investigation-pipeline-architecture.md.

Investigation tool calling

Tool schemas, provider adapters (transports/sdk/agent_clients.py), and investigation message shapes are documented in docs/investigation-tool-calling.md (all LLM providers, not vendor-specific).

Interactive shell: REPL watchdog demo

PR reviewers expect a visible demo (terminal log or screenshot) in the PR under Demo/Screenshot, not only tests. Copy the exact steps from this section into your PR description, then attach your terminal output or recording.

1. uv run opensre (TTY).
2. /trust on (or confirm the elevated-action prompt when running /watch).
3. /watch <pid> --max-cpu 80 — expect task … started. (use a real PID, e.g. the shell’s Python process).
4. /watches — table columns include id, pid, kind, status, thresholds, last sample.
5. /unwatch <task_id> or /cancel <task_id> — then /watches again; status should show cancelled.
6. Optional: lower --max-cpu so a threshold trips; after Telegram sends, the REPL prints one line: [task …] alarm fired: … (telegram delivered).

Automated equivalent (runs in make test-cov):
uv run pytest tests/interactive_shell/test_watchdog_repl_e2e_demo.py -v --tb=short

Longer transcript (optional): tests/interactive_shell/repl_watchdog_demo.md.

VS Code dev container

The dev container is defined under .devcontainer/. It builds from .devcontainer/Dockerfile (Python 3.13), then postCreateCommand creates .venv-devcontainer and runs pip install -e '.[dev]' (not uv). Docker Desktop, OrbStack, Colima, or another compatible runtime must be available on the host.

Benchmark

make benchmark

To refresh README benchmark copy from cached results (no LLM calls): make benchmark-update-readme.

Deployment

Full deployment instructions, prerequisites, and environment variable reference: DEPLOYMENT.md

Quick reference:

Path	Commands
EC2 (Docker/ECR — web + gateway)	make build-image → make deploy / make destroy
Gateway (AMI + systemd — gateway only)	make bake-gateway → make deploy-gateway / make destroy-gateway
Hosted (Railway / ECS / Vercel)	Deploy with repo Dockerfile; set LLM_PROVIDER + API key

Hosted runtime (Railway / ECS / Vercel)

1. Deploy this repository as a standard Python/FastAPI app using the repo Dockerfile or your host's native Python workflow.
2. Set LLM_PROVIDER and the matching API key (for example ANTHROPIC_API_KEY, OPENAI_API_KEY — see .env.example).
3. Add DATABASE_URI and REDIS_URI for hosted layouts that need persistence.
4. Add integration and storage env vars your deployment needs.

Minimal LLM env:

export LLM_PROVIDER=anthropic
export ANTHROPIC_API_KEY=...

For Railway: ensure the project has Postgres and Redis services and that the OpenSRE service has DATABASE_URI and REDIS_URI set before deploying. Set OPENSRE_DEPLOYMENT_METHOD=railway for telemetry labeling.

Telemetry and privacy

opensre ships with two telemetry stacks, both opt-out:

• PostHog — anonymous product analytics (commands used, success/failure, rough runtime, CLI/Python/OS/arch, and limited command metadata).
• Sentry — crashes and errors (stack traces, environment, release).

Events are tagged with entrypoint, opensre.runtime, and deployment_method. Sensitive headers, paths, and secret-shaped keys are scrubbed before send.

A random install ID is stored under ~/.opensre/anonymous_id. PostHog distinct_id is scoped to that ID. Telemetry is off in GitHub Actions and pytest.

First-launch GitHub login

On the first interactive launch (all platforms, except CI/CD and test harnesses), OpenSRE requires a GitHub device-flow sign-in before the REPL prompt. On success it sets github_username as a PostHog person property (via $identify/$set, which forces $process_person_profile: True for that one event — this is the only intentional PII OpenSRE sends) and emits a github_login_completed event. A configured GitHub integration suppresses re-prompting on later launches.

The existing kill-switches still apply: OPENSRE_NO_TELEMETRY / DO_NOT_TRACK make the $identify and github_login_completed calls no-ops, but the login itself still runs. Set OPENSRE_SKIP_GITHUB_LOGIN=1 to bypass the login gate entirely (also auto-bypassed in CI — CI=true, GITHUB_ACTIONS=true — and in pytest).

Kill-switch matrix

Env var	PostHog	Sentry
OPENSRE_NO_TELEMETRY=1	disabled	disabled
DO_NOT_TRACK=1	disabled	disabled
OPENSRE_ANALYTICS_DISABLED=1	disabled	unaffected
OPENSRE_SENTRY_DISABLED=1	unaffected	disabled
OPENSRE_SENTRY_LOGGING_DISABLED=1	unaffected	disables logger.error/logger.exception forwarding to Sentry; capture_exception unaffected

Full opt-out:

export OPENSRE_NO_TELEMETRY=1

Sentry DSN

Self-hosted users can set SENTRY_DSN to their project; unset uses the bundled default. SENTRY_DSN= (empty) drops events in before_send.

Deployment tagging

Set OPENSRE_DEPLOYMENT_METHOD to railway, ec2, vercel, or local (default local) to label Sentry events.

Local PostHog event log

By default, outbound PostHog payloads are also appended to ~/.opensre/posthog_events.txt (rotates at 1000 lines). Disable:

export OPENSRE_ANALYTICS_LOG_EVENTS=0

We do not collect alert contents, file contents, hostnames, credentials, raw CLI arguments, or PII by design.