wehub-resource-sync
4b6817381b
CI (OpenClaw E2E) / openclaw test (push) Has been cancelled
CI / coverage-report (push) Has been cancelled
CI / test-kubernetes (push) Has been cancelled
CI / should-run-thorough (push) Has been cancelled
CI / test-thorough (cloudwatch-demo) (push) Has been cancelled
CI / test-thorough (flink-ecs) (push) Has been cancelled
CI / test-thorough (upstream-lambda) (push) Has been cancelled
CI / test-thorough (prefect-ecs-fargate) (push) Has been cancelled
Release / build-binaries (zip, opensre.exe, onefile, windows-latest, windows-x64) (push) Has been cancelled
Benchmark image — build + push to ECR (any adapter) / build + push (push) Has been cancelled
CI / quality (ubuntu-latest) (push) Has been cancelled
CI / test (tools-runtime) (push) Has been cancelled
CI / test (e2e-general) (push) Has been cancelled
CI / test (cli-runtime) (push) Has been cancelled
CI / test (e2e-provider-and-openclaw) (push) Has been cancelled
CI / test (integrations-and-misc) (push) Has been cancelled
Release / verify (push) Has been cancelled
Release / build-python-dist (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-15-intel, darwin-x64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-latest, darwin-arm64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04, linux-x64) (push) Has been cancelled
Release / publish-release (push) Has been cancelled
Release / publish-main-release (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-checks (no-LLM) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-live shard ${{ matrix.shard_index }} (push) Has been cancelled
Release / prepare (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04-arm, linux-arm64) (push) Has been cancelled
Synthetic Deterministic Tests / Synthetic offline (deterministic) (push) Has been cancelled
48 lines
2.3 KiB
Plaintext
48 lines
2.3 KiB
Plaintext
---
|
|
title: "Data Lake Integration"
|
|
description: "Export pipeline telemetry and metrics to your data lake for custom analysis"
|
|
---
|
|
|
|
<video
|
|
autoPlay
|
|
loop
|
|
muted
|
|
playsInline
|
|
style={{ width: '100%', maxWidth: '100%' }}
|
|
>
|
|
<source src="/images/DataLake.mp4" type="video/mp4" />
|
|
Your browser does not support the video tag.
|
|
</video>
|
|
|
|
### Data Lake
|
|
Query all your pipeline runs to analyze sample processing turnaround times, costs per pipeline execution, error correlations and performance trends across projects.
|
|
|
|
### How It Works
|
|
We ingest all your pipeline telemetry, metrics, and logs into a multi-tenant data lake. You can then use the Tracer web interface to query this data to analyze performance, identify bottlenecks, and optimize your pipelines.
|
|
|
|
Identify underperforming steps, track which projects need attention, and monitor how efficiency changes over time. Tracer allows you to export all pipeline telemetry, metrics, and logs to your data lake for custom analysis and long-term retention.
|
|
|
|
You can calculate cost per sample, measure run durations, and compare throughput across pipelines to identify optimization opportunities. Tracer also makes it easier to spot regressions, monitor efficiency trends, and inspect error types so teams can locate bottlenecks, prioritize fixes, and improve their workflows using real execution data.
|
|
|
|
## Key Analytics
|
|
|
|
### Pipeline Success Rate Analysis
|
|
Track overall pipeline health with metrics such as success rate, total executions, and average turnaround times.
|
|
|
|
|
|
|
|
### Most Expensive Pipelines
|
|
Identify your most expensive pipelines for any time window and surface the average cost per run to understand spend drivers.
|
|
|
|
|
|
|
|
### Pipeline Resource Utilization
|
|
Spot pipelines that consistently underuse allocated compute, helping you pinpoint wasted capacity and optimize resource allocation.
|
|
|
|
|
|
|
|
### Longest Tool Execution
|
|
Surface the tools or steps that contribute the most to overall runtime, including the exact commands, so you know where to focus optimization efforts.
|
|
|
|
|