chore: import upstream snapshot with attribution

2026-07-13 12:31:57 +08:00
commit d718c5a372
986 changed files with 74597 additions and 0 deletions
@@ -0,0 +1 @@
+postgres-logs/
@@ -0,0 +1,425 @@
+# Distributed Tracing and Logs for SQLPage with OpenTelemetry and Grafana
+
+SQLPage has built-in support for [OpenTelemetry](https://opentelemetry.io/) (OTel),
+an open standard for collecting traces, metrics, and logs from your applications.
+When enabled, every HTTP request to SQLPage produces a **trace** — a timeline of
+everything that happened to serve that request, from receiving it to querying the
+database and rendering the response. SQLPage also emits structured request-aware
+logs, which this example forwards to Grafana Loki so you can inspect logs and traces
+side by side.
+
+This is useful for:
+
+- **Debugging slow pages**: see exactly which SQL query is taking the longest.
+- **Diagnosing connection pool exhaustion**: see how long requests wait for a database connection.
+- **End-to-end visibility**: follow a single user request from your reverse proxy (nginx, Caddy, etc.)
+  through SQLPage and into PostgreSQL.
+
+## Quick start (this example)
+
+This directory contains a ready-to-run Docker Compose stack that demonstrates
+the full tracing, logging, and PostgreSQL metrics pipeline. No prior
+OpenTelemetry experience is needed.
+
+### Prerequisites
+
+- [Docker](https://docs.docker.com/get-docker/) and
+  [Docker Compose](https://docs.docker.com/compose/install/) installed on your machine.
+
+### Run
+
+```bash
+cd examples/telemetry
+docker compose up --build
+```
+
+This starts eight services:
+
+| Service          | Role                                                      | Port          |
+|------------------|-----------------------------------------------------------|---------------|
+| **nginx**        | Reverse proxy, creates the root trace span                | `localhost:80`  |
+| **SQLPage**      | Your application, sends traces to the collector           | (internal 8080) |
+| **PostgreSQL**   | Database                                                  | (internal 5432) |
+| **Prometheus**    | Stores PostgreSQL metrics scraped from the OTel Collector | (internal 9090) |
+| **Tempo**        | Trace storage backend                                     | (internal 3200) |
+| **Loki**         | Log storage backend                                       | (internal 3100) |
+| **OTel Collector** | Receives traces, PostgreSQL metrics, and SQLPage logs   | `localhost:4318`, `localhost:1514` |
+| **Grafana**      | Web UI to explore traces and logs                         | `localhost:3000` |
+
+### Explore traces and logs
+
+1. Open the todo app at [http://localhost](http://localhost) — add a few items, click to toggle them.
+2. Open Grafana at [http://localhost:3000](http://localhost:3000).
+3. The default home dashboard now shows recent traces, recent SQLPage logs, and PostgreSQL metrics.
+4. Click any trace ID in the trace table to see the full span waterfall.
+5. In the logs panel, click a `trace_id` derived field to jump straight to the matching trace.
+6. The PostgreSQL metrics panels are populated by the collector's `postgresqlreceiver`.
+7. In the left sidebar, click **Explore** (compass icon) if you want to search manually.
+8. Select **Tempo** to search traces, **Loki** to search logs, or **Prometheus** to query metrics.
+
+### What you will see in a trace
+
+Each HTTP request produces a tree of **spans** (timed operations):
+
+```
+[nginx] GET /todos                         ← root span (created by nginx)
+  └─ [sqlpage] GET /todos                  ← HTTP request span
+       └─ [sqlpage] SQL website/todos.sql  ← SQL file execution
+            ├─ db.pool.acquire             ← time waiting for a DB connection
+            └─ db.query                    ← the actual SQL query
+                 db.query.text = "SELECT title, ..."
+                 db.system.name = "postgresql"
+```
+
+Key attributes on each span:
+
+| Span               | Key attributes                                                |
+|---------------------|--------------------------------------------------------------|
+| HTTP request        | `http.request.method`, `http.route`, `http.response.status_code`, `user_agent.original` |
+| SQL file execution  | `code.file.path` — which `.sql` file was executed            |
+| `db.pool.acquire`   | `db.client.connection.pool.name`; `sqlpage.db.pool.size` — current pool size when acquiring |
+| `db.query`          | `db.query.text` — the full SQL text; `db.system.name` — database type |
+
+### What you will see in the logs
+
+SQLPage writes one structured log line per event, for example:
+
+```text
+ts=2026-03-08T20:56:15.000Z level=info target=sqlpage::access msg="200 OK" http.request.method=GET url.path=/ trace_id=4f2d...
+```
+
+Request-completion access logs use the target `sqlpage::access` and are written to stdout.
+Diagnostic logs, warnings, and internal errors are written to stderr. Docker and most
+container log drivers collect both streams by default, but custom log pipelines that read
+only stderr need to collect stdout as well to keep access logs.
+
+The OpenTelemetry Collector receives these SQLPage container logs through Docker's syslog
+logging driver and forwards them to Loki.
+The homepage dashboard filters to the `sqlpage` service so you can see request logs update
+live while you use the sample app.
+
+### PostgreSQL correlation and explain plans
+
+SQLPage automatically sets the
+[`application_name`](https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-APPLICATION-NAME)
+on each database connection to include the W3C
+[traceparent](https://www.w3.org/TR/trace-context/#traceparent-header).
+This means you can:
+
+- See trace IDs in `pg_stat_activity` when monitoring live queries:
+  ```sql
+  SELECT application_name, query, state FROM pg_stat_activity;
+  -- application_name: sqlpage 00-abc123...-def456...-01
+  ```
+- Include trace IDs in PostgreSQL logs by adding `%a` to
+  [`log_line_prefix`](https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-LINE-PREFIX).
+
+This example also enables PostgreSQL's
+[`auto_explain`](https://www.postgresql.org/docs/current/auto-explain.html)
+extension for queries slower than 25 ms. The plans are logged in JSON and keep
+the SQLPage trace context in the `app=[...]` prefix, so Grafana's Loki
+`trace_id` derived field links each slow-query plan back to the originating
+SQLPage trace.
+
+### Testing pool pressure
+
+To simulate database connection pool exhaustion (a common production issue),
+reduce the pool size to 1 in `sqlpage/sqlpage.json`:
+
+```json
+{
+    "listen_on": "0.0.0.0:8080",
+    "max_database_pool_connections": 1
+}
+```
+
+Restart (`docker compose restart sqlpage`), then open several browser tabs
+to `http://localhost` simultaneously. In Grafana, you will see `db.pool.acquire`
+spans with longer durations as requests queue up waiting for the single connection.
+
+---
+
+## How it works
+
+### Enabling tracing in SQLPage
+
+Tracing is **built into SQLPage** — there is nothing to install or compile.
+It activates automatically when you set the `OTEL_EXPORTER_OTLP_ENDPOINT`
+environment variable. When this variable is not set, SQLPage behaves exactly
+as before (plain text logs, no tracing overhead).
+
+**Minimal setup — just two environment variables:**
+
+```bash
+# Where to send traces (an OTLP-compatible endpoint)
+export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4318"
+
+# A name to identify this service in traces
+export OTEL_SERVICE_NAME="sqlpage"
+
+# Now start SQLPage as usual
+sqlpage
+```
+
+These are [standard OpenTelemetry environment variables](https://opentelemetry.io/docs/specs/otel/protocol/exporter/)
+understood by all OTel-compatible tools. SQLPage reads them directly — no
+`sqlpage.json` configuration is needed for tracing.
+
+### The role of each component
+
+**OpenTelemetry** is a standard, not a product. It defines a protocol (OTLP) for
+sending trace data. Here is how the pieces fit together:
+
+```
+ Traces:  SQLPage -> OTel Collector -> Tempo -> Grafana
+ Logs:    SQLPage -> Docker syslog logging driver -> OTel Collector -> Loki -> Grafana
+ Metrics: PostgreSQL -> OTel Collector postgresqlreceiver -> Prometheus -> Grafana
+```
+
+- **SQLPage** generates trace data and sends it via the OTLP HTTP protocol.
+- A **collector** (optional) receives traces and forwards them to one or more backends.
+  Useful for buffering, sampling, or fanning out to multiple destinations.
+  You can skip the collector and send directly from SQLPage to most backends.
+- The **OTel Collector** also receives SQLPage container logs and forwards them to Loki.
+- **Tempo** stores traces, **Loki** stores logs, and **Grafana** lets you search both.
+
+### Trace context propagation
+
+When a reverse proxy (like nginx) sits in front of SQLPage, you want the trace
+to start at nginx and continue into SQLPage as a single, connected trace.
+This works via the
+[W3C Trace Context](https://www.w3.org/TR/trace-context/) standard:
+nginx adds a `traceparent` HTTP header to the request it forwards to SQLPage,
+and SQLPage reads it to continue the same trace.
+
+Most modern reverse proxies and load balancers support this.
+For nginx specifically, use the [`ngx_otel_module`](https://nginx.org/en/docs/ngx_otel_module.html)
+(included in the `nginx:otel` Docker image).
+
+---
+
+## Setup guides by deployment scenario
+
+### Self-hosted with Grafana Tempo and Loki
+
+This is what the Docker Compose example in this directory uses.
+[Grafana Tempo](https://grafana.com/oss/tempo/) is a free, open-source trace backend, and
+[Grafana Loki](https://grafana.com/oss/loki/) is the corresponding log backend.
+
+**Components:**
+- [Grafana Tempo](https://grafana.com/docs/tempo/latest/) stores the traces.
+- [Grafana Loki](https://grafana.com/docs/loki/latest/) stores the logs.
+- [Grafana](https://grafana.com/docs/grafana/latest/) provides the web UI.
+- An [OTel Collector](https://opentelemetry.io/docs/collector/) receives SQLPage traces,
+  SQLPage logs, and PostgreSQL metrics in this example.
+
+**SQLPage environment variables:**
+
+```bash
+OTEL_EXPORTER_OTLP_ENDPOINT=http://<collector-or-tempo-host>:4318
+OTEL_SERVICE_NAME=sqlpage
+```
+
+**Links:**
+- [Tempo installation guide](https://grafana.com/docs/tempo/latest/setup/)
+- [OTel Collector installation](https://opentelemetry.io/docs/collector/installation/)
+
+### Self-hosted with Jaeger
+
+[Jaeger](https://www.jaegertracing.io/) is another popular open-source tracing
+backend. Version 2+ natively accepts OTLP — no collector needed.
+
+**Start Jaeger with one command:**
+
+```bash
+docker run -d --name jaeger \
+  -p 16686:16686 \
+  -p 4317:4317 \
+  -p 4318:4318 \
+  jaegertracing/jaeger:latest
+```
+
+**SQLPage environment variables:**
+
+```bash
+OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
+OTEL_SERVICE_NAME=sqlpage
+```
+
+Open the Jaeger UI at [http://localhost:16686](http://localhost:16686) to explore traces.
+
+**Links:**
+- [Jaeger getting started](https://www.jaegertracing.io/docs/latest/getting-started/)
+
+### Grafana Cloud
+
+[Grafana Cloud](https://grafana.com/products/cloud/) has a free tier that
+includes trace storage. SQLPage can send traces directly — no collector needed.
+
+**SQLPage environment variables:**
+
+```bash
+OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp-gateway-prod-<region>.grafana.net/otlp
+OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <base64-of-instance_id:api_token>"
+OTEL_SERVICE_NAME=sqlpage
+```
+
+Replace:
+- `<region>` with your Grafana Cloud region (e.g., `us-east-0`, `eu-west-2`).
+  Find it in your Grafana Cloud portal under **My Account** > **Tempo**.
+- `<base64-of-instance_id:api_token>` with the Base64 encoding of
+  `<instance-id>:<cloud-api-token>`. Generate a token in your Grafana Cloud
+  portal under **My Account** > **API Keys**.
+
+  On macOS/Linux, generate the Base64 value with:
+  ```bash
+  echo -n "123456:glc_your_token_here" | base64
+  ```
+
+**Links:**
+- [Send data via OTLP to Grafana Cloud](https://grafana.com/docs/grafana-cloud/send-data/otlp/send-data-otlp/)
+
+### Datadog
+
+[Datadog](https://www.datadoghq.com/) supports OTLP ingestion through the
+Datadog Agent.
+
+**1. Run the Datadog Agent** with OTLP ingest enabled:
+
+```bash
+docker run -d --name datadog-agent \
+  -e DD_API_KEY=<your-datadog-api-key> \
+  -e DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_HTTP_ENDPOINT=0.0.0.0:4318 \
+  -e DD_SITE=datadoghq.com \
+  -p 4318:4318 \
+  gcr.io/datadoghq/agent:latest
+```
+
+**2. Point SQLPage to the Agent:**
+
+```bash
+OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
+OTEL_SERVICE_NAME=sqlpage
+```
+
+Traces appear in the Datadog **APM > Traces** section.
+
+**Links:**
+- [OTLP ingestion in the Datadog Agent](https://docs.datadoghq.com/opentelemetry/setup/otlp_ingest_in_the_agent/)
+
+### Honeycomb
+
+[Honeycomb](https://www.honeycomb.io/) accepts OTLP directly — no collector needed.
+
+**SQLPage environment variables:**
+
+```bash
+OTEL_EXPORTER_OTLP_ENDPOINT=https://api.honeycomb.io
+OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=<your-api-key>"
+OTEL_SERVICE_NAME=sqlpage
+```
+
+For the EU region, use `https://api.eu1.honeycomb.io` instead.
+
+**Links:**
+- [Send data with OpenTelemetry — Honeycomb docs](https://docs.honeycomb.io/send-data/opentelemetry/)
+
+### New Relic
+
+[New Relic](https://newrelic.com/) accepts OTLP directly.
+
+**SQLPage environment variables:**
+
+```bash
+OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.nr-data.net
+OTEL_EXPORTER_OTLP_HEADERS="api-key=<your-newrelic-license-key>"
+OTEL_SERVICE_NAME=sqlpage
+```
+
+For the EU region, use `https://otlp.eu01.nr-data.net` instead.
+
+Find your Ingest License Key in the New Relic UI under
+**API Keys** (type: `INGEST - LICENSE`).
+
+**Links:**
+- [New Relic OTLP endpoint configuration](https://docs.newrelic.com/docs/opentelemetry/best-practices/opentelemetry-otlp/)
+
+### Axiom
+
+[Axiom](https://axiom.co/) accepts OTLP directly.
+
+**SQLPage environment variables:**
+
+```bash
+OTEL_EXPORTER_OTLP_ENDPOINT=https://api.axiom.co
+OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <your-api-token>,X-Axiom-Dataset=<your-dataset>"
+OTEL_SERVICE_NAME=sqlpage
+```
+
+**Links:**
+- [Send OpenTelemetry data to Axiom](https://axiom.co/docs/send-data/opentelemetry)
+
+---
+
+## Environment variable reference
+
+These are [standard OpenTelemetry variables](https://opentelemetry.io/docs/specs/otel/protocol/exporter/),
+not specific to SQLPage.
+
+| Variable                          | Required? | Description                                    | Example                              |
+|-----------------------------------|-----------|------------------------------------------------|--------------------------------------|
+| `OTEL_EXPORTER_OTLP_ENDPOINT`    | Yes       | Base URL of the OTLP receiver                  | `http://localhost:4318`              |
+| `OTEL_SERVICE_NAME`              | No        | Service name shown in traces (default: `unknown_service`) | `sqlpage`                 |
+| `OTEL_EXPORTER_OTLP_HEADERS`    | No        | Comma-separated `key=value` pairs for auth headers | `api-key=abc123`                |
+| `OTEL_EXPORTER_OTLP_PROTOCOL`   | No        | Protocol (default: `http/protobuf`)            | `http/protobuf`                      |
+| `RUST_LOG` or `LOG_LEVEL`         | No        | Filter which spans/logs are emitted            | `sqlpage=debug,tracing_actix_web=info` |
+
+When `OTEL_EXPORTER_OTLP_ENDPOINT` is **not set**, SQLPage uses plain text
+logging only (same behavior as versions before tracing support was added).
+
+---
+
+## Troubleshooting
+
+### No traces appear
+
+1. **Check that SQLPage sees the endpoint.** Look for this line in the startup logs:
+   ```
+   OpenTelemetry tracing enabled (OTEL_EXPORTER_OTLP_ENDPOINT is set)
+   ```
+   If you don't see it, the environment variable is not reaching SQLPage.
+
+2. **Check that the collector/backend is reachable.** From the SQLPage host, try:
+   ```bash
+   curl -v http://<endpoint>:4318/v1/traces
+   ```
+   You should get a response (even if it's an error like "no data"), not a connection refused.
+
+3. **Check the collector logs** for export errors (e.g., authentication failures).
+
+### Traces are disconnected (nginx and SQLPage show as separate traces)
+
+This means the `traceparent` header is not being propagated. Check that:
+
+- Your reverse proxy is configured to inject/propagate the `traceparent` header.
+- For nginx, you need the `ngx_otel_module` with `otel_trace_context propagate`
+  in the location block. Setting `otel_span_name "$request_method $uri"` also keeps
+  the nginx span name aligned with the actual request path. See the `nginx/nginx.conf`
+  in this example.
+
+### Spans are missing (e.g., no `db.query` spans)
+
+The `RUST_LOG` filter might be too restrictive.
+SQLPage emits spans at the `INFO` level by default. Make sure your filter
+includes `sqlpage=info`:
+
+```bash
+RUST_LOG="sqlpage=info,actix_web=info,tracing_actix_web=info"
+```
+
+If you filter individual targets instead of the broader `sqlpage` target, include
+the access-log target too:
+
+```bash
+RUST_LOG="sqlpage::access=info,sqlpage::webserver::http=info,actix_web=info,tracing_actix_web=info"
+```
@@ -0,0 +1,161 @@
+services:
+  nginx:
+    image: nginx:otel
+    ports:
+      - "8080:80"
+    volumes:
+      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
+    depends_on:
+      - sqlpage
+      - otel-collector
+    logging:
+      driver: syslog
+      options:
+        syslog-address: "udp://localhost:1516"
+        syslog-format: "rfc5424micro"
+        tag: "nginx"
+
+  sqlpage:
+    image: lovasoa/sqlpage:main
+    environment:
+      - DATABASE_URL=postgres://sqlpage:sqlpage@postgres:5432/sqlpage
+      - OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
+      - OTEL_METRIC_EXPORT_INTERVAL=1000
+      - OTEL_SERVICE_NAME=sqlpage
+    volumes:
+      - ./website:/var/www
+      - ./sqlpage:/etc/sqlpage
+    depends_on:
+      otel-collector:
+        condition: service_started
+      postgres:
+        condition: service_healthy
+    logging:
+      driver: syslog
+      options:
+        syslog-address: "udp://localhost:1514"
+        syslog-format: "rfc5424micro"
+        tag: "sqlpage"
+
+  postgres:
+    image: postgres:18
+    command:
+      - postgres
+      - -c
+      - shared_preload_libraries=pg_stat_statements,auto_explain
+      - -c
+      - pg_stat_statements.track_utility=off
+      - -c
+      - logging_collector=on
+      - -c
+      - log_destination=jsonlog
+      - -c
+      - log_directory=/var/log/postgresql
+      - -c
+      - log_filename=postgresql
+      - -c
+      - log_file_mode=0644
+      - -c
+      - log_rotation_age=0
+      - -c
+      - log_rotation_size=0
+      - -c
+      - log_line_prefix=ts=%m pid=%p db=%d user=%u app=[%a] 
+      - -c
+      - log_min_messages=info
+      - -c
+      - log_error_verbosity=verbose
+      - -c
+      - log_connections=on
+      - -c
+      - log_disconnections=on
+      - -c
+      - log_duration=on
+      - -c
+      - log_statement=all
+      - -c
+      - log_min_duration_statement=25
+      - -c
+      - auto_explain.log_min_duration=25ms
+      - -c
+      - auto_explain.log_analyze=on
+      - -c
+      - auto_explain.log_buffers=on
+      - -c
+      - auto_explain.log_timing=on
+      - -c
+      - auto_explain.log_verbose=on
+      - -c
+      - auto_explain.log_format=json
+    environment:
+      POSTGRES_USER: sqlpage
+      POSTGRES_PASSWORD: sqlpage
+      POSTGRES_DB: sqlpage
+    volumes:
+      - ./postgres-init:/docker-entrypoint-initdb.d:ro
+      - ./postgres-logs:/var/log/postgresql:z
+    depends_on:
+      postgres-logs:
+        condition: service_completed_successfully
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U sqlpage"]
+      interval: 2s
+      timeout: 5s
+      retries: 5
+
+  postgres-logs:
+    image: alpine
+    command: ["sh", "-c", "touch /var/log/postgresql/postgresql /var/log/postgresql/postgresql.json && chmod 777 /var/log/postgresql && chmod 666 /var/log/postgresql/postgresql*"]
+    volumes:
+      - ./postgres-logs:/var/log/postgresql:z
+
+  otel-collector:
+    image: otel/opentelemetry-collector-contrib:latest
+    user: "0:0"
+    volumes:
+      - ./otel-collector.yaml:/etc/otelcol-contrib/config.yaml:ro
+      - ./postgres-logs:/var/log/postgresql:ro,z
+    depends_on:
+      tempo:
+        condition: service_started
+      postgres:
+        condition: service_started
+      loki:
+        condition: service_started
+
+  prometheus:
+    image: prom/prometheus:v3.2.1
+    command:
+      - --config.file=/etc/prometheus/prometheus.yml
+      - --storage.tsdb.path=/prometheus
+      - --web.enable-otlp-receiver
+    volumes:
+      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
+
+  tempo:
+    image: grafana/tempo:3.0.0
+    ports:
+      - "3200:3200"
+    volumes:
+      - ./tempo.yaml:/etc/tempo/config.yaml:ro
+    command: ["-config.file=/etc/tempo/config.yaml"]
+
+  loki:
+    image: grafana/loki:3.7.2
+    command: ["-config.file=/etc/loki/local-config.yaml"]
+
+  grafana:
+    image: grafana/grafana:latest
+    environment:
+      - GF_AUTH_ANONYMOUS_ENABLED=true
+      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
+      - GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH=/etc/grafana/provisioning/dashboards/sqlpage/sqlpage-home.json
+      - GF_NEWS_NEWS_FEED_ENABLED=false
+    volumes:
+      - ./grafana/datasources.yaml:/etc/grafana/provisioning/datasources/datasources.yaml:ro,z
+      - ./grafana/dashboards.yaml:/etc/grafana/provisioning/dashboards/dashboards.yaml:ro,z
+      - ./grafana:/etc/grafana/provisioning/dashboards/sqlpage:ro,z
+    depends_on:
+      - tempo
+      - loki
+      - prometheus
@@ -0,0 +1,12 @@
+apiVersion: 1
+
+providers:
+  - name: SQLPage Tracing
+    orgId: 1
+    folder: ""
+    type: file
+    disableDeletion: true
+    updateIntervalSeconds: 30
+    allowUiUpdates: false
+    options:
+      path: /etc/grafana/provisioning/dashboards/sqlpage
@@ -0,0 +1,34 @@
+apiVersion: 1
+
+datasources:
+  - name: Tempo
+    type: tempo
+    uid: tempo
+    access: proxy
+    url: http://tempo:3200
+    isDefault: true
+    jsonData:
+      nodeGraph:
+        enabled: true
+      tracesToLogsV2:
+        datasourceUid: loki
+        spanStartTimeShift: "-5m"
+        spanEndTimeShift: "5m"
+        customQuery: true
+        query: '{service_name=~"nginx|sqlpage|postgresql"} | trace_id="$${__span.traceId}"'
+  - name: Loki
+    type: loki
+    uid: loki
+    access: proxy
+    url: http://loki:3100
+    jsonData:
+      derivedFields:
+        - name: trace_id
+          matcherRegex: '(?:trace_id=|00-)([0-9a-f]{32})'
+          datasourceUid: tempo
+          url: '$${__value.raw}'
+  - name: Prometheus
+    type: prometheus
+    uid: prometheus
+    access: proxy
+    url: http://prometheus:9090
@@ -0,0 +1,531 @@
+{
+  "annotations": {
+    "list": [
+      {
+        "builtIn": 1,
+        "datasource": {
+          "type": "grafana",
+          "uid": "-- Grafana --"
+        },
+        "enable": true,
+        "hide": true,
+        "iconColor": "rgba(0, 211, 255, 1)",
+        "name": "Annotations & Alerts",
+        "type": "dashboard"
+      }
+    ]
+  },
+  "editable": true,
+  "fiscalYearStartMonth": 0,
+  "graphTooltip": 0,
+  "links": [],
+  "panels": [
+    {
+      "fieldConfig": {
+        "defaults": {},
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 5,
+        "w": 24,
+        "x": 0,
+        "y": 0
+      },
+      "id": 1,
+      "options": {
+        "code": {
+          "language": "plaintext",
+          "showLineNumbers": false,
+          "showMiniMap": false
+        },
+        "content": "<div style=\"padding: 4px 2px 0; font-size: 15px; line-height: 1.55;\"><h1 style=\"margin: 0 0 10px; font-size: 28px;\">SQLPage Observability</h1><p style=\"margin: 0 0 8px;\">Open <a href=\"http://localhost\" target=\"_blank\" rel=\"noopener noreferrer\">http://localhost</a> and interact with the app. New requests will appear here automatically.</p><p style=\"margin: 0; color: #666;\">This dashboard shows traces, logs, and application metrics exported by SQLPage. Trace waterfalls link to PostgreSQL logs via trace IDs. Metrics include HTTP durations, DB query latencies, and connection pool states.</p></div>",
+        "mode": "html"
+      },
+      "pluginVersion": "12.4.1",
+      "title": "Overview",
+      "type": "text"
+    },
+    {
+      "datasource": {
+        "type": "prometheus",
+        "uid": "prometheus"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "palette-classic"
+          },
+          "custom": {
+            "axisBorderShow": false,
+            "axisCenteredZero": false,
+            "axisColorMode": "text",
+            "axisLabel": "",
+            "axisPlacement": "auto",
+            "barAlignment": 0,
+            "barWidthFactor": 0.6,
+            "drawStyle": "line",
+            "fillOpacity": 10,
+            "gradientMode": "none",
+            "hideFrom": {
+              "legend": false,
+              "tooltip": false,
+              "viz": false
+            },
+            "insertNulls": false,
+            "lineInterpolation": "linear",
+            "lineWidth": 2,
+            "pointSize": 4,
+            "scaleDistribution": {
+              "type": "linear"
+            },
+            "showPoints": "never",
+            "showValues": false,
+            "spanNulls": false,
+            "stacking": {
+              "group": "A",
+              "mode": "none"
+            },
+            "thresholdsStyle": {
+              "mode": "off"
+            }
+          },
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": 0
+              }
+            ]
+          },
+          "unit": "s"
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 8,
+        "w": 12,
+        "x": 0,
+        "y": 5
+      },
+      "id": 10,
+      "options": {
+        "legend": {
+          "calcs": [],
+          "displayMode": "list",
+          "placement": "bottom",
+          "showLegend": true
+        },
+        "tooltip": {
+          "hideZeros": false,
+          "mode": "single",
+          "sort": "none"
+        }
+      },
+      "pluginVersion": "12.4.1",
+      "targets": [
+        {
+          "datasource": {
+            "type": "prometheus",
+            "uid": "prometheus"
+          },
+          "expr": "histogram_quantile(0.95, sum(rate(http_server_request_duration_seconds_bucket[5m])) by (le, http_route))",
+          "legendFormat": "HTTP P95 {{http_route}}",
+          "refId": "A"
+        },
+        {
+          "datasource": {
+            "type": "prometheus",
+            "uid": "prometheus"
+          },
+          "expr": "histogram_quantile(0.95, sum(rate(db_client_operation_duration_seconds_bucket[5m])) by (le, db_operation_name))",
+          "legendFormat": "DB P95 {{db_operation_name}}",
+          "refId": "B"
+        }
+      ],
+      "title": "Request & Query Latency (P95)",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {
+        "type": "prometheus",
+        "uid": "prometheus"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "palette-classic"
+          },
+          "custom": {
+            "axisBorderShow": false,
+            "axisCenteredZero": false,
+            "axisColorMode": "text",
+            "axisLabel": "",
+            "axisPlacement": "auto",
+            "barAlignment": 0,
+            "barWidthFactor": 0.6,
+            "drawStyle": "line",
+            "fillOpacity": 10,
+            "gradientMode": "none",
+            "hideFrom": {
+              "legend": false,
+              "tooltip": false,
+              "viz": false
+            },
+            "insertNulls": false,
+            "lineInterpolation": "linear",
+            "lineWidth": 2,
+            "pointSize": 4,
+            "scaleDistribution": {
+              "type": "linear"
+            },
+            "showPoints": "never",
+            "showValues": false,
+            "spanNulls": false,
+            "stacking": {
+              "group": "A",
+              "mode": "none"
+            },
+            "thresholdsStyle": {
+              "mode": "off"
+            }
+          },
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": 0
+              }
+            ]
+          },
+          "unit": "none"
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 8,
+        "w": 12,
+        "x": 12,
+        "y": 5
+      },
+      "id": 11,
+      "options": {
+        "legend": {
+          "calcs": [],
+          "displayMode": "list",
+          "placement": "bottom",
+          "showLegend": true
+        },
+        "tooltip": {
+          "hideZeros": false,
+          "mode": "single",
+          "sort": "none"
+        }
+      },
+      "pluginVersion": "12.4.1",
+      "targets": [
+        {
+          "datasource": {
+            "type": "prometheus",
+            "uid": "prometheus"
+          },
+          "expr": "sum(db_client_connection_count) by (db_client_connection_state)",
+          "legendFormat": "{{db_client_connection_state}}",
+          "refId": "A"
+        }
+      ],
+      "title": "SQLPage DB Connection Pool",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {
+        "type": "tempo",
+        "uid": "tempo"
+      },
+      "fieldConfig": {
+        "defaults": {
+          "custom": {
+            "align": "auto",
+            "cellOptions": {
+              "type": "auto"
+            },
+            "footer": {
+              "reducers": []
+            },
+            "inspect": false
+          },
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": 0
+              },
+              {
+                "color": "red",
+                "value": 80
+              }
+            ]
+          }
+        },
+        "overrides": [
+          {
+            "matcher": {
+              "id": "byName",
+              "options": "traceID"
+            },
+            "properties": [
+              {
+                "id": "custom.hideFrom.viz",
+                "value": true
+              }
+            ]
+          },
+          {
+            "matcher": {
+              "id": "byName",
+              "options": "traceName"
+            },
+            "properties": [
+              {
+                "id": "custom.cellOptions",
+                "value": {
+                  "type": "data-links"
+                }
+              },
+              {
+                "id": "links",
+                "value": [
+                  {
+                    "targetBlank": false,
+                    "title": "${__value.text}",
+                    "url": "/a/grafana-exploretraces-app/explore?traceId=${__data.fields.traceID}"
+                  }
+                ]
+              }
+            ]
+          },
+          {
+            "matcher": {
+              "id": "byName",
+              "options": "traceDuration"
+            },
+            "properties": [
+              {
+                "id": "custom.width",
+                "value": 120
+              }
+            ]
+          },
+          {
+            "matcher": {
+              "id": "byName",
+              "options": "Service"
+            },
+            "properties": [
+              {
+                "id": "custom.width",
+                "value": 58
+              }
+            ]
+          },
+          {
+            "matcher": {
+              "id": "byName",
+              "options": "Start time"
+            },
+            "properties": [
+              {
+                "id": "custom.width",
+                "value": 204
+              }
+            ]
+          }
+        ]
+      },
+      "gridPos": {
+        "h": 8,
+        "w": 24,
+        "x": 0,
+        "y": 13
+      },
+      "id": 2,
+      "options": {
+        "cellHeight": "sm",
+        "frameIndex": 0,
+        "showHeader": true,
+        "sortBy": [
+          {
+            "desc": true,
+            "displayName": "Start time"
+          }
+        ]
+      },
+      "pluginVersion": "12.4.1",
+      "targets": [
+        {
+          "datasource": {
+            "type": "tempo",
+            "uid": "tempo"
+          },
+          "filters": [
+            {
+              "id": "57ff7584",
+              "operator": "=",
+              "scope": "span"
+            }
+          ],
+          "limit": 50,
+          "metricsQueryType": "range",
+          "query": "{resource.service.name != nil}",
+          "queryType": "traceqlSearch",
+          "refId": "A",
+          "serviceMapUseNativeHistograms": false,
+          "tableType": "traces"
+        }
+      ],
+      "timeFrom": "1h",
+      "title": "Trace List",
+      "transformations": [
+        {
+          "id": "organize",
+          "options": {
+            "excludeByName": {
+              "nested": true
+            },
+            "indexByName": {
+              "startTime": 1,
+              "traceDuration": 4,
+              "traceID": 0,
+              "traceName": 3,
+              "traceService": 2
+            },
+            "renameByName": {
+              "startTime": "Start time",
+              "traceDuration": "Duration",
+              "traceID": "Trace",
+              "traceName": "Route",
+              "traceService": "Service"
+            }
+          }
+        }
+      ],
+      "type": "table"
+    },
+    {
+      "datasource": {
+        "type": "loki",
+        "uid": "loki"
+      },
+      "fieldConfig": {
+        "defaults": {},
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 10,
+        "w": 12,
+        "x": 0,
+        "y": 21
+      },
+      "id": 3,
+      "options": {
+        "dedupStrategy": "none",
+        "enableInfiniteScrolling": false,
+        "enableLogDetails": true,
+        "prettifyLogMessage": false,
+        "showCommonLabels": false,
+        "showControls": false,
+        "showLabels": true,
+        "showTime": true,
+        "sortOrder": "Descending",
+        "unwrappedColumns": false,
+        "wrapLogMessage": true
+      },
+      "pluginVersion": "12.4.1",
+      "targets": [
+        {
+          "datasource": {
+            "type": "loki",
+            "uid": "loki"
+          },
+          "direction": "backward",
+          "editorMode": "builder",
+          "expr": "{service_name=\"sqlpage\"}",
+          "queryType": "range",
+          "refId": "A"
+        }
+      ],
+      "timeFrom": "1h",
+      "title": "SQLPage Logs",
+      "type": "logs"
+    },
+    {
+      "datasource": {
+        "type": "loki",
+        "uid": "loki"
+      },
+      "fieldConfig": {
+        "defaults": {},
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 10,
+        "w": 12,
+        "x": 12,
+        "y": 21
+      },
+      "id": 6,
+      "options": {
+        "dedupStrategy": "none",
+        "enableInfiniteScrolling": false,
+        "enableLogDetails": true,
+        "prettifyLogMessage": false,
+        "showCommonLabels": false,
+        "showControls": false,
+        "showLabels": true,
+        "showTime": true,
+        "sortOrder": "Descending",
+        "unwrappedColumns": false,
+        "wrapLogMessage": true
+      },
+      "pluginVersion": "12.4.1",
+      "targets": [
+        {
+          "datasource": {
+            "type": "loki",
+            "uid": "loki"
+          },
+          "direction": "backward",
+          "editorMode": "builder",
+          "expr": "{service_name=\"postgresql\"}",
+          "queryType": "range",
+          "refId": "A"
+        }
+      ],
+      "timeFrom": "1h",
+      "title": "PostgreSQL Logs",
+      "type": "logs"
+    }
+  ],
+  "preload": false,
+  "refresh": "5s",
+  "schemaVersion": 42,
+  "tags": ["sqlpage", "tracing", "logs", "metrics"],
+  "templating": {
+    "list": []
+  },
+  "time": {
+    "from": "now-1h",
+    "to": "now"
+  },
+  "timepicker": {},
+  "timezone": "browser",
+  "title": "SQLPage Observability Home",
+  "uid": "sqlpage-tracing-home",
+  "version": 1,
+  "weekStart": ""
+}
@@ -0,0 +1,50 @@
+load_module modules/ngx_otel_module.so;
+
+events {}
+
+http {
+    otel_exporter {
+        endpoint otel-collector:4317;
+    }
+
+    otel_service_name nginx;
+
+    map $status $log_level {
+        ~^[45] error;
+        default info;
+    }
+
+    map $status $log_target {
+        ~^[45] nginx.request_error;
+        default nginx.access;
+    }
+
+    log_format otel_request 'ts=$time_iso8601 level=$log_level target=$log_target '
+                            'client.address=$remote_addr method=$request_method path="$uri" '
+                            'status=$status request_time=$request_time '
+                            'connection=$connection '
+                            'upstream_addr="$upstream_addr" upstream_status="$upstream_status" '
+                            'upstream_response_time="$upstream_response_time" '
+                            'referer="$http_referer" user_agent="$http_user_agent" '
+                            'trace_id=$otel_trace_id span_id=$otel_span_id parent_id=$otel_parent_id';
+
+    access_log /dev/stdout otel_request;
+    error_log /dev/stderr warn;
+
+    upstream sqlpage {
+        server sqlpage:8080;
+    }
+
+    server {
+        listen 80;
+
+        location / {
+            otel_trace on;
+            otel_span_name "$request_method $uri";
+            otel_trace_context propagate;
+            proxy_pass http://sqlpage;
+            proxy_set_header Host $host;
+            proxy_set_header X-Real-IP $remote_addr;
+        }
+    }
+}
@@ -0,0 +1,158 @@
+receivers:
+  otlp:
+    protocols:
+      grpc:
+        endpoint: 0.0.0.0:4317
+      http:
+        endpoint: 0.0.0.0:4318
+  syslog/sqlpage:
+    protocol: rfc5424
+    udp:
+      listen_address: 0.0.0.0:1514
+  syslog/nginx:
+    protocol: rfc5424
+    udp:
+      listen_address: 0.0.0.0:1516
+  filelog/postgresql:
+    include:
+      - /var/log/postgresql/postgresql.json
+    start_at: end
+    include_file_path: true
+    operators:
+      - type: json_parser
+        parse_from: body
+        parse_to: attributes
+  postgresql:
+    endpoint: postgres:5432
+    transport: tcp
+    username: sqlpage
+    password: sqlpage
+    tls:
+      insecure: true
+    databases:
+      - sqlpage
+    collection_interval: 1s
+
+processors:
+  transform/sqlpage_logs:
+    error_mode: ignore
+    log_statements:
+      - context: resource
+        statements:
+          - set(resource.attributes["service.name"], "sqlpage")
+      - context: log
+        statements:
+          - set(log.body, log.attributes["message"]) where log.attributes["message"] != nil
+          - merge_maps(log.cache, ExtractPatterns(log.body, "level=(?P<level>[^ ]+)"), "upsert") where IsString(log.body)
+          - merge_maps(log.cache, ExtractPatterns(log.body, "trace_id=(?P<trace_id>[0-9a-f]+)"), "upsert") where IsString(log.body)
+          - set(log.attributes["level"], log.cache["level"]) where log.cache["level"] != nil
+          - set(log.attributes["trace_id"], log.cache["trace_id"]) where log.cache["trace_id"] != nil
+          - set(log.severity_text, "TRACE") where log.cache["level"] == "trace"
+          - set(log.severity_number, SEVERITY_NUMBER_TRACE) where log.cache["level"] == "trace"
+          - set(log.severity_text, "DEBUG") where log.cache["level"] == "debug"
+          - set(log.severity_number, SEVERITY_NUMBER_DEBUG) where log.cache["level"] == "debug"
+          - set(log.severity_text, "INFO") where log.cache["level"] == "info"
+          - set(log.severity_number, SEVERITY_NUMBER_INFO) where log.cache["level"] == "info"
+          - set(log.severity_text, "WARN") where log.cache["level"] == "warn"
+          - set(log.severity_number, SEVERITY_NUMBER_WARN) where log.cache["level"] == "warn"
+          - set(log.severity_text, "ERROR") where log.cache["level"] == "error"
+          - set(log.severity_number, SEVERITY_NUMBER_ERROR) where log.cache["level"] == "error"
+  transform/postgresql_logs:
+    error_mode: ignore
+    log_statements:
+      - context: resource
+        statements:
+          - set(resource.attributes["service.name"], "postgresql")
+      - context: log
+        statements:
+          - set(log.body, log.attributes["message"]) where log.attributes["message"] != nil
+          - merge_maps(log.cache, ExtractPatterns(log.attributes["application_name"], "00-(?P<trace_id>[0-9a-f]{32})-[0-9a-f]{16}-[0-9a-f]{2}"), "upsert") where IsString(log.attributes["application_name"])
+          - set(log.attributes["level"], log.attributes["error_severity"]) where log.attributes["error_severity"] != nil
+          - set(log.attributes["trace_id"], log.cache["trace_id"]) where log.cache["trace_id"] != nil
+          - set(log.severity_text, "DEBUG") where log.attributes["level"] == "DEBUG1" or log.attributes["level"] == "DEBUG2" or log.attributes["level"] == "DEBUG3" or log.attributes["level"] == "DEBUG4" or log.attributes["level"] == "DEBUG5" or log.attributes["level"] == "DEBUG"
+          - set(log.severity_number, SEVERITY_NUMBER_DEBUG) where log.severity_text == "DEBUG"
+          - set(log.severity_text, "INFO") where log.attributes["level"] == "LOG" or log.attributes["level"] == "INFO" or log.attributes["level"] == "NOTICE"
+          - set(log.severity_number, SEVERITY_NUMBER_INFO) where log.severity_text == "INFO"
+          - set(log.severity_text, "WARN") where log.attributes["level"] == "WARNING"
+          - set(log.severity_number, SEVERITY_NUMBER_WARN) where log.severity_text == "WARN"
+          - set(log.severity_text, "ERROR") where log.attributes["level"] == "ERROR" or log.attributes["level"] == "FATAL" or log.attributes["level"] == "PANIC"
+          - set(log.severity_number, SEVERITY_NUMBER_ERROR) where log.severity_text == "ERROR"
+  transform/nginx_logs:
+    error_mode: ignore
+    log_statements:
+      - context: resource
+        statements:
+          - set(resource.attributes["service.name"], "nginx")
+      - context: log
+        statements:
+          - set(log.body, log.attributes["message"]) where log.attributes["message"] != nil
+          - merge_maps(log.cache, ExtractPatterns(log.body, "level=(?P<level>[^ ]+)"), "upsert") where IsString(log.body)
+          - merge_maps(log.cache, ExtractPatterns(log.body, "\\[(?P<nginx_level>info|error|warn|notice|debug)\\]"), "upsert") where IsString(log.body)
+          - merge_maps(log.cache, ExtractPatterns(log.body, "trace_id=(?P<trace_id>[0-9a-f]{32})"), "upsert") where IsString(log.body)
+          - merge_maps(log.cache, ExtractPatterns(log.body, "span_id=(?P<span_id>[0-9a-f]{16})"), "upsert") where IsString(log.body)
+          - set(log.attributes["trace_id"], log.cache["trace_id"]) where log.cache["trace_id"] != nil
+          - set(log.attributes["span_id"], log.cache["span_id"]) where log.cache["span_id"] != nil
+          - set(log.attributes["level"], log.cache["level"]) where log.cache["level"] != nil
+          - set(log.attributes["level"], log.cache["nginx_level"]) where log.cache["level"] == nil and log.cache["nginx_level"] != nil
+          - set(log.severity_text, "DEBUG") where log.attributes["level"] == "debug"
+          - set(log.severity_number, SEVERITY_NUMBER_DEBUG) where log.attributes["level"] == "debug"
+          - set(log.severity_text, "INFO") where log.attributes["level"] == "info" or log.attributes["level"] == "notice"
+          - set(log.severity_number, SEVERITY_NUMBER_INFO) where log.attributes["level"] == "info" or log.attributes["level"] == "notice"
+          - set(log.severity_text, "WARN") where log.attributes["level"] == "warn"
+          - set(log.severity_number, SEVERITY_NUMBER_WARN) where log.attributes["level"] == "warn"
+          - set(log.severity_text, "ERROR") where log.attributes["level"] == "error"
+          - set(log.severity_number, SEVERITY_NUMBER_ERROR) where log.attributes["level"] == "error"
+  batch:
+    timeout: 1s
+
+exporters:
+  otlp_grpc/tempo:
+    endpoint: tempo:4317
+    tls:
+      insecure: true
+  otlp_http/loki:
+    endpoint: http://loki:3100/otlp
+  prometheus:
+    endpoint: 0.0.0.0:9464
+
+service:
+  pipelines:
+    traces:
+      receivers:
+        - otlp
+      processors:
+        - batch
+      exporters:
+        - otlp_grpc/tempo
+    metrics:
+      receivers:
+        - otlp
+        - postgresql
+      processors:
+        - batch
+      exporters:
+        - prometheus
+    logs/sqlpage:
+      receivers:
+        - syslog/sqlpage
+      processors:
+        - transform/sqlpage_logs
+        - batch
+      exporters:
+        - otlp_http/loki
+    logs/postgresql:
+      receivers:
+        - filelog/postgresql
+      processors:
+        - transform/postgresql_logs
+        - batch
+      exporters:
+        - otlp_http/loki
+    logs/nginx:
+      receivers:
+        - syslog/nginx
+      processors:
+        - transform/nginx_logs
+        - batch
+      exporters:
+        - otlp_http/loki
@@ -0,0 +1,5 @@
+\connect postgres
+CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
+
+\connect sqlpage
+CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
@@ -0,0 +1,9 @@
+global:
+  scrape_interval: 10s
+  evaluation_interval: 10s
+
+scrape_configs:
+  - job_name: otel-collector
+    static_configs:
+      - targets: ["otel-collector:9464"]
+
@@ -0,0 +1,6 @@
+CREATE TABLE IF NOT EXISTS todos (
+    id SERIAL PRIMARY KEY,
+    title TEXT NOT NULL,
+    done BOOLEAN NOT NULL DEFAULT false,
+    created_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
@@ -0,0 +1,3 @@
+{
+  "listen_on": "0.0.0.0:8080"
+}
@@ -0,0 +1,19 @@
+stream_over_http_enabled: true
+
+server:
+  http_listen_port: 3200
+
+distributor:
+  receivers:
+    otlp:
+      protocols:
+        grpc:
+          endpoint: 0.0.0.0:4317
+
+storage:
+  trace:
+    backend: local
+    wal:
+      path: /tmp/tempo/wal
+    local:
+      path: /tmp/tempo/blocks
@@ -0,0 +1,47 @@
+GET http://localhost:8080/
+traceparent: 00-00000000000000000000000000000001-0000000000000001-01
+HTTP 200
+[Asserts]
+body contains "Todo List"
+body contains "Add a todo"
+body not contains "An error occurred"
+
+POST http://localhost:8080/add_todo.sql
+traceparent: 00-00000000000000000000000000000002-0000000000000002-01
+[FormParams]
+title: Hurl telemetry todo
+HTTP 302
+[Asserts]
+header "Location" == "/"
+
+GET http://localhost:8080/
+traceparent: 00-00000000000000000000000000000003-0000000000000003-01
+HTTP 200
+[Asserts]
+body contains "Hurl telemetry todo"
+body not contains "An error occurred"
+
+GET http://localhost:3200/api/search?q=%7Bresource.service.name%3D%22sqlpage%22%7D&limit=20
+HTTP 200
+[Asserts]
+body contains "sqlpage"
+body contains "\"traceID\":\"3\""
+
+GET http://localhost:3200/api/search?q=%7Bresource.service.name%3D%22nginx%22%7D&limit=20
+HTTP 200
+[Asserts]
+body contains "nginx"
+body contains "\"traceID\":\"3\""
+
+GET http://localhost:3200/api/traces/3
+HTTP 200
+[Asserts]
+body contains "\"service.name\""
+body contains "\"sqlpage\""
+body contains "\"nginx\""
+body contains "\"name\":\"GET /\""
+body contains "\"name\":\"SELECT\""
+body contains "\"kind\":\"SPAN_KIND_CLIENT\""
+body contains "\"db.query.text\""
+body contains "\"db.system.name\""
+body contains "\"db.operation.name\""
@@ -0,0 +1,8 @@
+-- Toggle an existing todo
+UPDATE todos SET done = $done::boolean WHERE id = $id::int;
+
+-- Insert a new todo if title is provided via the form (POST)
+INSERT INTO todos (title)
+SELECT :title WHERE :title IS NOT NULL AND length(:title) > 0;
+
+SELECT 'redirect' AS component, '/' AS link;
@@ -0,0 +1,11 @@
+SELECT 'list' AS component,
+       'Todo List' AS title;
+
+SELECT title,
+       CASE WHEN done THEN 'complete' ELSE 'pending' END AS description,
+       'add_todo.sql?id=' || id || '&done=' || (NOT done)::text AS link
+FROM todos
+ORDER BY created_at DESC;
+
+SELECT 'form' AS component, 'Add a todo' AS title, 'add_todo.sql' AS action;
+SELECT 'title' AS name, 'What do you need to do?' AS placeholder;
@@ -0,0 +1,7 @@
+SELECT pg_sleep(15);
+
+SELECT 'list' AS component,
+       'Slow Query Complete' AS title;
+
+SELECT 'The slow query finished successfully.' AS title,
+       'This page exists to make PostgreSQL query-sample events easy to capture.' AS description;