tracer-cloud--opensre/docs/signoz.mdx

# SigNoz Integration

Query logs, metrics, and traces from [SigNoz](https://signoz.io) via the Query Range API.

OpenSRE uses `POST /api/v5/query_range` for `query_signoz_logs`, `query_signoz_metrics`, and
`query_signoz_traces`. Configure **`SIGNOZ_URL`** and **`SIGNOZ_API_KEY`** (service account key).

## Quick Start

If you already run SigNoz, skip to env setup below.
If you want a local stack, use the [official SigNoz Docker setup](https://signoz.io/docs/install/docker/).

```bash
docker compose -f signoz/docker/docker-compose.yaml up -d
# ... later
docker compose -f signoz/docker/docker-compose.yaml down
```

Local Docker serves the UI and API on **http://localhost:8080** (not 3301).

### 1. Create an API key

In SigNoz: **Settings → Service Accounts** → create a service account → **Keys** → **Add Key**.
Copy the key (shown once).

### 2. Configure environment variables

```bash
export SIGNOZ_URL="http://localhost:8080"
export SIGNOZ_API_KEY="<service-account-api-key>"
```

### 3. Verify connectivity

```bash
uv run opensre integrations verify signoz
```

Expected output mentions the SigNoz Query API (`/api/v2/metrics`, `/api/v5/query_range`).

### 4. Use the tools

When `alert_source` is `signoz`, the agent auto-seeds three tools before the ReAct loop:

- **`query_signoz_logs`** — Search logs by service, severity, and time window.
- **`query_signoz_metrics`** — Query CPU, memory, and request-rate signals.
- **`query_signoz_traces`** — Query error spans, latency percentiles, and dependencies.

## Webhook Configuration

SigNoz emits Prometheus-style webhook payloads. To trigger OpenSRE investigations automatically:

1. In SigNoz, go to **Settings → Notification Channels**.
2. Create a **Webhook** channel pointing at your OpenSRE instance:
   ```
   POST https://your-opensre-instance/investigate
   ```
3. The agent will detect `alert_source: signoz` from the payload and auto-query logs, metrics, and traces.

## CLI Setup

```bash
opensre integrations setup signoz
```

You will be prompted for SigNoz URL and API key only.

## Supported Metrics (V1)

| Alias | Actual SigNoz Metric |
|-------|---------------------|
| `cpu_usage` | `system_cpu_usage` |
| `memory_usage` | `system_memory_usage` |
| `request_rate` | `signoz_calls_total` |

You can also pass any raw metric name known to SigNoz.
For latency percentiles (p95/p99), prefer `query_signoz_traces`.

## API reference

- Endpoint: `POST {SIGNOZ_URL}/api/v5/query_range`
- Auth header: `SigNoz-Api-Key: <YOUR_API_KEY>`
- Validation probe: `GET {SIGNOZ_URL}/api/v2/metrics`

## Example Investigation

```bash
uv run opensre investigate --input-json '{
  "alert_source": "signoz",
  "alert_name": "HighErrorRate",
  "pipeline_name": "payment-service",
  "severity": "critical",
  "commonLabels": {
    "service_name": "payment-service",
    "severity": "critical"
  },
  "commonAnnotations": {
    "summary": "Error rate exceeded 5%"
  }
}'
```

## Troubleshooting

| Symptom | Fix |
|---------|-----|
| `SigNoz configuration is incomplete` | Set both `SIGNOZ_URL` and `SIGNOZ_API_KEY`. |
| `HTTP 401` on verify | Regenerate the service account key; check URL (local Docker: port **8080**). |
| `No logs returned` | Confirm telemetry exists in SigNoz and filter fields match (`service.name` for logs). |
| `No metrics returned` | Verify the metric name in SigNoz Metrics Explorer; empty results return a warning for unknown metrics. |