4b6817381b
CI (OpenClaw E2E) / openclaw test (push) Has been cancelled
CI / coverage-report (push) Has been cancelled
CI / test-kubernetes (push) Has been cancelled
CI / should-run-thorough (push) Has been cancelled
CI / test-thorough (cloudwatch-demo) (push) Has been cancelled
CI / test-thorough (flink-ecs) (push) Has been cancelled
CI / test-thorough (upstream-lambda) (push) Has been cancelled
CI / test-thorough (prefect-ecs-fargate) (push) Has been cancelled
Release / build-binaries (zip, opensre.exe, onefile, windows-latest, windows-x64) (push) Has been cancelled
Benchmark image — build + push to ECR (any adapter) / build + push (push) Has been cancelled
CI / quality (ubuntu-latest) (push) Has been cancelled
CI / test (tools-runtime) (push) Has been cancelled
CI / test (e2e-general) (push) Has been cancelled
CI / test (cli-runtime) (push) Has been cancelled
CI / test (e2e-provider-and-openclaw) (push) Has been cancelled
CI / test (integrations-and-misc) (push) Has been cancelled
Release / verify (push) Has been cancelled
Release / build-python-dist (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-15-intel, darwin-x64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-latest, darwin-arm64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04, linux-x64) (push) Has been cancelled
Release / publish-release (push) Has been cancelled
Release / publish-main-release (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-checks (no-LLM) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-live shard ${{ matrix.shard_index }} (push) Has been cancelled
Release / prepare (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04-arm, linux-arm64) (push) Has been cancelled
Synthetic Deterministic Tests / Synthetic offline (deterministic) (push) Has been cancelled
112 lines
4.8 KiB
Plaintext
112 lines
4.8 KiB
Plaintext
---
|
|
title: 'How to use Tracer with Grafana'
|
|
sidebarTitle: 'Grafana'
|
|
description: 'Execution-aware context beyond dashboards'
|
|
---
|
|
|
|
Grafana provides visualization and alerting for metrics, logs, and traces collected from external telemetry systems such as Prometheus and node-level exporters. In bioinformatics and HPC environments, it is commonly used to explore system- and application-reported signals for compute-intensive workloads through configurable dashboards.
|
|
|
|
Tracer complements Grafana by observing how tasks, tools, and processes actually execute at runtime and organizing this behavior by pipeline, run, and execution unit.
|
|
|
|
<Note>
|
|
If you're new to Tracer or want a conceptual overview, see [How Tracer fits in your stack](/comparisons/overview).
|
|
</Note>
|
|
|
|
## What Grafana does well
|
|
|
|
Grafana is designed for visualization and alerting. It provides:
|
|
|
|
- Dashboards built from metrics, logs, and traces
|
|
- Flexible queries, panels, and transformations
|
|
- Alerting based on thresholds and rules
|
|
- Support for many telemetry backends and data sources
|
|
|
|
These capabilities make Grafana effective for exploring and monitoring reported telemetry across systems and services.
|
|
|
|
## What Grafana does not observe
|
|
|
|
Grafana visualizes data that is collected elsewhere. It does not observe execution directly and does not have inherent awareness of pipeline or task structure. It does not show:
|
|
|
|
- Execution behavior inside processes or containers
|
|
- CPU vs I/O vs memory contention at runtime
|
|
- Short-lived processes that complete between metric scrapes
|
|
- Idle time masked by aggregate utilization metrics
|
|
- How reported metrics map to pipeline runs, tasks, or tools
|
|
- How cost relates to actual execution rather than to infrastructure uptime
|
|
|
|
This information is not present in dashboards unless it is inferred through instrumentation, exporters, or custom queries.
|
|
|
|
## Why this gap matters
|
|
|
|
Scientific pipelines often involve heterogeneous tools, nested execution, and short-lived subprocesses. When relying on dashboards alone, teams may see that resources were used, but not how work progressed.
|
|
|
|
As a result:
|
|
|
|
- Performance bottlenecks must be inferred from aggregates
|
|
- Idle time can be mistaken for productive work
|
|
- Cost is attributed to hosts or services rather than to execution units
|
|
- Custom dashboards accumulate assumptions about runtime behavior
|
|
|
|
These approaches can be useful, but they have limits when execution behavior is not directly observed.
|
|
|
|
## What Tracer adds
|
|
|
|
Tracer observes execution directly from the host and container runtime and adds:
|
|
|
|
- Observed CPU, memory, disk, and network behavior
|
|
- Visibility into short-lived processes and nested tools
|
|
- Attribution by pipeline, run, task, or execution unit
|
|
- Cost mapping aligned with observed runtime activity
|
|
|
|
These insights are based on what actually runs, not on emitted metrics or dashboard queries.
|
|
|
|
## Example: reported metrics versus observed execution
|
|
|
|
A dashboard shows high instance utilization during a pipeline run. Tracer reveals that:
|
|
|
|
- CPU usage remains low
|
|
- Tasks spend most of their time blocked on disk I/O
|
|
- Multiple short-lived helper processes dominate execution time
|
|
|
|
This indicates an I/O-bound workload rather than insufficient compute. The distinction is visible only when execution behavior is observed directly.
|
|
|
|
## Observability comparison
|
|
|
|
<iframe
|
|
src="https://vakisr.github.io/tracer-radar-chart/radar-chart-grafana.html"
|
|
width="100%"
|
|
height="750"
|
|
style={{ border: 'none', borderRadius: '8px' }}
|
|
title="Grafana vs Grafana+Tracer capabilities comparison"
|
|
/>
|
|
|
|
This comparison highlights the difference between dashboard-level telemetry and execution-level observation.
|
|
|
|
## What Tracer does not replace
|
|
|
|
Tracer is not a general-purpose visualization platform.
|
|
|
|
- It does not replace Grafana dashboards
|
|
- It does not collect arbitrary application metrics
|
|
- Its alerting is focused on execution behavior, not all system events
|
|
- It stores execution-derived metrics it observes, but does not replace metric backends for unrelated or user-defined metrics
|
|
|
|
Grafana remains useful as a shared visualization layer across many systems.
|
|
|
|
## When to use Tracer with Grafana
|
|
|
|
Tracer is most useful alongside Grafana when teams need to:
|
|
|
|
- Understand pipeline behavior beyond reported metrics
|
|
- Diagnose performance issues involving short-lived tasks
|
|
- Attribute resource usage and cost to workflows or tools
|
|
- Reduce manual dashboard configuration and metric correlation
|
|
|
|
Tracer focuses on execution behavior. Grafana continues to provide visualization and alerting for reported telemetry.
|
|
|
|
## Summary
|
|
|
|
Grafana visualizes metrics, logs, and traces collected from many systems. Tracer adds execution-level visibility that shows how pipelines, tasks, and tools actually behave at runtime and how resource usage and cost map to real work. Together, they provide complementary perspectives without overlap.
|
|
|
|
<div style={{ height: '50vh' }}></div>
|