wehub/triggerdotdev--trigger.dev
1
0
Fork 0
Code Issues Pull Requests Actions 1 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:32:57 +08:00
commit cd420f9332
4811 changed files with 884702 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/observability/dashboards.mdx
+102
View File
@@ -0,0 +1,102 @@
---
title: "Dashboards"
description: "Create custom dashboards with real-time metrics powered by TRQL queries."
---
## Overview
In the Trigger.dev dashboard we have built-in dashboards and you can create your own.
Dashboards are powered by [TRQL queries](/observability/query) with widgets that can be displayed as charts, tables, or single values. They automatically refresh to show the latest data.
### Available metrics data
Trigger.dev automatically collects process metrics (CPU, memory) and Node.js runtime metrics (event loop, heap) for all deployed tasks -- no configuration needed. Requires SDK version **4.4.1 or later**. You can also create custom metrics using the `otel.metrics` API from the SDK.
All of this data is available in the `metrics` table for use in dashboard widgets. See [Logging, tracing & metrics](/logging#metrics) for the full list of automatic metrics and how to create custom ones, or the [Query page](/observability/query#metrics-table-columns) for the `metrics` table schema.
![The built-in Metrics dashboard](/images/metrics-built-in.png)
### Visualization types
- **Line chart** - Show trends over time
- **Bar chart** - Compare values across categories
- **Area chart** - Display cumulative trends
- **Table** - Show detailed data in rows
- **Single value** - Display a single metric (count, sum, average, etc.)
You can also add Titles to your dashboard.
## Filtering and time ranges
All widgets on a dashboard use the time range filter applied to the dashboard.
You can also filter the data by:
- Scope: Environment, Project, Organization
- Tasks
- Queues
## Creating custom dashboards
1. In the sidebar click the + icon next to "Dashboards".
2. Name your custom dashboard.
3. From the top-right you can "Add chart" or "Add title".
4. For charts you write [TRQL queries](/observability/query) and choose a visualization type.
5. You can resize and reposition widgets on your dashboards.
## Performance considerations
### Optimize queries for metrics
1. **Use time bucketing** - `timeBucket()` automatically groups by appropriate intervals
2. **Limit result size** - Add `LIMIT` clauses, especially for table widgets
3. **Use approximate functions** - `uniq()` instead of `uniqExact()` for faster approximate counts
## Exporting metric data
Export data from any metric widget:
1. Click the widget menu (three dots)
2. Select "Copy JSON" or "Copy CSV"
## Best practices
1. **Start simple** - Begin with basic metrics and iterate based on insights
2. **Use meaningful names** - Give widgets clear, descriptive titles
3. **Group related metrics** - Organize dashboards by theme (performance, costs, errors)
4. **Test queries first** - Use the Query page to develop and test before adding to dashboards
## Troubleshooting
### Widget shows "No data"
- Check that your query returns results in the Query page
- Verify time filters include the period with data
- Ensure task/queue filters match existing runs
### Widget is slow to load
- Add time range filters to your query
- Use `LIMIT` clauses
- Simplify aggregations
- Check query execution time in Query page
### Chart displays incorrectly
- Verify column names match visualization config
- Check data types (numbers for charts, dates for time series)
- Ensure `timeBucket()` is used for time-series charts
- Review that series columns exist in query results
## Limits
Dashboards are powered by Query so have [the same limits](/observability/query#limits) as Query.
There is a separate concurrency limits for metric widgets.
| Limit | Details |
| :------------------------ | :------------- |
| Concurrent widget queries | 30 per project |
See [Limits](/limits) for details.
docs/observability/query.mdx
+585
View File
@@ -0,0 +1,585 @@
---
title: "Query"
description: "Query allows you to write custom queries against your data using TRQL (Trigger.dev Query Language), a SQL-style language based on ClickHouse SQL. You can query your data through the dashboard, SDK, or REST API."
---
### Available tables
- `runs`: contains all task run data including status, timing, costs, and task output. Run metadata (key-value set in your task) is not available on the Query page.
- `metrics`: contains metrics data for your runs including CPU, memory, and your custom metrics
- `llm_metrics`: contains LLM/AI metrics including token usage, costs, latency, and model performance data from GenAI spans
### `metrics` table columns
| Column | Type | Description |
| :--- | :--- | :--- |
| `metric_name` | string | Metric identifier (e.g., `process.cpu.utilization`) |
| `metric_type` | string | `gauge`, `sum`, or `histogram` |
| `metric_value` | number | The observed value |
| `bucket_start` | datetime | 10-second aggregation bucket start time |
| `run_id` | string | Associated run ID |
| `task_identifier` | string | Task slug |
| `attempt_number` | number | Attempt number |
| `machine_id` | string | Machine that produced the metric |
| `machine_name` | string | Machine preset (e.g., `small-1x`) |
| `worker_version` | string | Worker version |
| `environment_type` | string | `PRODUCTION`, `STAGING`, `DEVELOPMENT`, `PREVIEW` |
| `attributes` | json | Raw JSON attributes for custom data |
See [Logging, tracing & metrics](/logging#automatic-system-and-runtime-metrics) for the full list of automatically collected metrics and how to create custom metrics. You can visualize this data on [Dashboards](/observability/dashboards).
### `prettyFormat()`
Use `prettyFormat()` to format metric values for display:
```sql
SELECT
timeBucket(),
prettyFormat(avg(metric_value), 'bytes') AS avg_memory_usage
FROM metrics
WHERE metric_name = 'process.memory.usage'
GROUP BY timeBucket
ORDER BY timeBucket
LIMIT 1000
```
Available format types: `bytes`, `percent`, `duration`, `durationSeconds`, `quantity`, `costInDollars`.
## Using the Query dashboard
Navigate to the Query page to write and execute queries. The dashboard provides:
- **AI-powered query generation** - Describe what you want in natural language
- **Syntax highlighting** - SQL syntax highlighting for better readability
- **Query history** - Access your previous queries
- **Interactive help** - Built-in documentation for TRQL syntax and functions
- **Export options** - Download results as JSON or CSV
![The Query dashboard](/images/query-chart-usage-percentiles.png)
## Querying from the SDK
Use `query.execute()` to run TRQL queries programmatically from your backend code:
```typescript
import { query } from "@trigger.dev/sdk";
// Basic query with defaults (environment scope, json format)
const result = await query.execute("SELECT run_id, status FROM runs LIMIT 10");
console.log(result.results); // Array<Record<string, any>>
```
### Type-safe queries
Use the `QueryTable` type for nice inferred types in your query results:
```typescript
import { query, type QueryTable } from "@trigger.dev/sdk";
// Type-safe query using QueryTable with specific columns
const typedResult = await query.execute<QueryTable<"runs", "run_id" | "status" | "triggered_at">>(
"SELECT run_id, status, triggered_at FROM runs LIMIT 10"
);
typedResult.results.forEach((row) => {
console.log(row.run_id, row.status); // Fully typed!
});
```
### Query options
```typescript
import { query } from "@trigger.dev/sdk";
const result = await query.execute("SELECT COUNT(*) as count FROM runs", {
// Scope: "environment" (default), "project", or "organization"
scope: "project",
// Time period using shorthand (e.g., "7d", "30d", "1h")
period: "7d",
// Or use explicit time range
// from: new Date("2024-01-01"),
// to: new Date("2024-01-31"),
// Response format: "json" (default) or "csv"
format: "json",
});
```
### CSV export
Export query results as CSV by setting `format: "csv"`:
```typescript
const csvResult = await query.execute("SELECT run_id, status, triggered_at FROM runs", {
format: "csv",
period: "7d",
});
const lines = csvResult.results.split("\n");
console.log(lines[0]); // CSV header row
```
## Querying from the REST API
Execute queries via HTTP POST to `/api/v1/query`:
```sh
curl -X POST https://api.trigger.dev/api/v1/query \
-H "Authorization: Bearer YOUR_SECRET_KEY" \
-H "Content-Type: application/json" \
-d '{
"query": "SELECT run_id, status FROM runs LIMIT 10",
"scope": "environment",
"period": "7d",
"format": "json"
}'
```
See the [API reference](/management/query/execute) for full details.
## TRQL syntax guide
### Basic queries
Select columns from a table:
```sql
SELECT run_id, task_identifier, status
FROM runs
LIMIT 10
```
Alias columns with `AS`:
```sql
SELECT task_identifier AS task, count() AS total
FROM runs
GROUP BY task
```
### Using \*
Note that when you use `SELECT *` we don't return all the columns, we only return the core columns. This is for performance reasons (the underlying ClickHouse database is columnar and selecting lots of columns isn't efficient).
You should specify the columns you want to return.
### Filtering with WHERE
Use comparison operators:
```sql
SELECT run_id, task_identifier FROM runs
WHERE status = 'Failed'
```
Available operators:
```sql
-- Comparison operators
WHERE status = 'Failed' -- Equal
WHERE status != 'Completed' -- Not equal
WHERE attempt_count > 3 -- Greater than
WHERE attempt_count >= 3 -- Greater than or equal
WHERE attempt_count < 5 -- Less than
WHERE attempt_count <= 5 -- Less than or equal
-- IN for multiple values
WHERE status IN ('Failed', 'Crashed')
-- LIKE for pattern matching (% = wildcard)
WHERE task_identifier LIKE 'email%'
-- ILIKE for case-insensitive matching
WHERE task_identifier ILIKE '%send%'
-- BETWEEN for ranges
WHERE triggered_at BETWEEN '2024-01-01' AND '2024-01-31'
-- NULL checks
WHERE completed_at IS NOT NULL
WHERE completed_at IS NULL
-- Array column checks
WHERE has(tags, 'user_12345')
WHERE notEmpty(tags)
WHERE hasAny(tags, array('user_12345', 'user_67890'))
WHERE hasAll(tags, array('user_12345', 'user_67890'))
WHERE indexOf(tags, 'user_12345') > 0
WHERE arrayElement(tags, 1) = 'user_12345'
```
### Sorting and limiting
Sort results with `ORDER BY`:
```sql
SELECT run_id, compute_cost, triggered_at
FROM runs
ORDER BY compute_cost DESC, triggered_at ASC
LIMIT 50
```
### Grouping and aggregation
Use `GROUP BY` with aggregate functions:
```sql
SELECT
task_identifier,
avg(metric_value) AS avg_memory
FROM metrics
WHERE metric_name = 'process.memory.usage'
GROUP BY task_identifier
ORDER BY avg_memory DESC
LIMIT 20
```
## Available functions
TRQL provides a rich set of functions for data analysis.
### Aggregate functions
- `count()` - Count rows
- `countIf(col, cond)` - Count rows matching condition
- `countDistinct(col)` - Count unique values
- `sum(col)` - Sum of values
- `sumIf(col, cond)` - Sum values matching condition
- `avg(col)` - Average of values
- `min(col)` - Minimum value
- `max(col)` - Maximum value
- `median(col)` - Median value (50th percentile)
- `quantile(p)(col)` - Value at percentile p (0-1)
- `stddevPop(col)` - Population standard deviation
- `stddevSamp(col)` - Sample standard deviation
Example:
```sql
SELECT
task_identifier,
count() AS total_runs,
avg(usage_duration) AS avg_duration_ms,
median(usage_duration) AS median_duration_ms,
quantile(0.95)(usage_duration) AS p95_duration_ms
FROM runs
GROUP BY task_identifier
```
### Date/time functions
**Time bucketing:**
```sql
-- Auto-bucket by time period based on query's time range
SELECT timeBucket(), count() AS runs
FROM runs
GROUP BY timeBucket()
```
**Date extraction:**
```sql
SELECT
toYear(triggered_at) AS year,
toMonth(triggered_at) AS month,
toDayOfWeek(triggered_at) AS day_of_week,
toHour(triggered_at) AS hour
FROM runs
```
**Date truncation:**
```sql
SELECT
toStartOfDay(triggered_at) AS day,
count() AS runs_per_day
FROM runs
GROUP BY day
ORDER BY day DESC
```
**Date arithmetic:**
```sql
-- Add/subtract time
SELECT dateAdd('day', 7, triggered_at) AS week_later
FROM runs
-- Calculate differences
SELECT dateDiff('minute', executed_at, completed_at) AS duration_minutes
FROM runs
WHERE completed_at IS NOT NULL
```
Common date functions:
- `now()` - Current date and time
- `today()` - Current date
- `toDate(dt)` - Convert to date
- `toStartOfDay(dt)`, `toStartOfHour(dt)`, `toStartOfMonth(dt)` - Truncate to start of period
- `formatDateTime(dt, format)` - Format datetime as string
### String functions
```sql
SELECT
lower(status) AS status_lower,
upper(status) AS status_upper,
concat(task_identifier, '-', status) AS combined,
substring(run_id, 1, 8) AS short_id,
length(task_identifier) AS name_length
FROM runs
```
Common string functions:
- `length(s)` - String length
- `lower(s)`, `upper(s)` - Case conversion
- `concat(s1, s2, ...)` - Concatenate strings
- `substring(s, offset, len)` - Extract substring
- `trim(s)` - Remove whitespace
- `replace(s, from, to)` - Replace occurrences
- `startsWith(s, prefix)`, `endsWith(s, suffix)` - Check prefixes/suffixes
### Conditional functions
```sql
SELECT
run_id,
if(status = 'Failed', 1, 0) AS is_failed,
multiIf(
status = 'Completed', 'ok',
status = 'Failed', 'bad',
'other'
) AS status_category,
coalesce(completed_at, triggered_at) AS end_time
FROM runs
```
- `if(cond, then, else)` - Conditional expression
- `multiIf(c1, t1, c2, t2, ..., else)` - Multiple conditions (like CASE)
- `coalesce(a, b, ...)` - First non-null value
### Math functions
```sql
SELECT
round(compute_cost, 4) AS cost_rounded,
ceil(usage_duration / 1000) AS duration_seconds_up,
floor(usage_duration / 1000) AS duration_seconds_down,
abs(compute_cost) AS cost_abs
FROM runs
```
### Array functions
Useful for working with tags and other array columns:
```sql
SELECT
run_id,
tags,
length(tags) AS tag_count,
has(tags, 'user_12345') AS is_production,
arrayJoin(tags) AS individual_tag -- Expand array to rows
FROM runs
WHERE notEmpty(tags)
```
### JSON functions
The `output`, `error`, and `metrics.attributes` columns are already JSON, so use dot notation to read or filter on them. You don't need `JSONExtract*` for these (those are for string columns).
```sql
SELECT
run_id,
output.message AS output_message,
output.count AS count,
output.externalId AS external_id
FROM runs
WHERE task_identifier = 'my-task'
AND output.externalId = 'something'
ORDER BY triggered_at DESC
LIMIT 100
```
## Query scopes
Control what data your query can access:
- **`environment`** (default) - Query runs in the current environment only
- **`project`** - Query runs across all environments in the project
- **`organization`** - Query runs across all projects in the organization
```typescript
// Query across all environments in a project
const result = await query.execute("SELECT environment, count() FROM runs GROUP BY environment", {
scope: "project",
});
```
## Time ranges
We recommend avoiding adding `triggered_at` in the actual TRQL query. The dashboard, API, and SDK have a time filter that is applied automatically and is easier to work with. It means the queries can be executed with multiple periods easily.
### Using period shorthand
```typescript
await query.execute("SELECT count() FROM runs", {
period: "4d", // Last 4 days
});
// Supported periods: "1h", "6h", "12h", "1d", "7d", "30d", "90d", etc.
```
### Using explicit dates
```typescript
await query.execute("SELECT count() FROM runs", {
from: new Date("2024-01-01"),
to: new Date("2024-01-31"),
});
// Or use Unix timestamps
await query.execute("SELECT count() FROM runs", {
from: Date.now() - 7 * 24 * 60 * 60 * 1000, // 7 days ago
to: Date.now(),
});
```
## Example queries
### Failed runs (in the last 24 hours)
```sql
SELECT
task_identifier,
run_id,
error,
triggered_at
FROM runs
WHERE status = 'Failed'
ORDER BY triggered_at DESC
```
With the time filter set to 24h.
### Task success rate by day
```sql
SELECT
toDate(triggered_at) AS day,
task_identifier,
countIf(status = 'Completed') AS completed,
countIf(status = 'Failed') AS failed,
round(completed / (completed + failed) * 100, 2) AS success_rate_pct
FROM runs
WHERE status IN ('Completed', 'Failed')
GROUP BY day, task_identifier
ORDER BY day DESC, task_identifier
```
### Top 10 most expensive runs
```sql
SELECT
run_id,
task_identifier,
compute_cost,
usage_duration,
triggered_at
FROM runs
WHERE compute_cost > 0
ORDER BY compute_cost DESC
LIMIT 10
```
### Average compute duration over time
```sql
SELECT
timeBucket() AS time,
task_identifier,
avg(usage_duration) AS avg_duration_ms,
count() AS run_count
FROM runs
WHERE usage_duration IS NOT NULL
GROUP BY time, task_identifier
ORDER BY time ASC
```
### Runs by queue and machine
```sql
SELECT
queue,
machine,
count() AS run_count,
countIf(status = 'Completed') AS completed,
countIf(status = 'Failed') AS failed
FROM runs
GROUP BY queue, machine
ORDER BY queue, machine
```
### CPU utilization over time
Track process CPU utilization bucketed over time.
```sql
SELECT
timeBucket(),
avg(metric_value) AS avg_cpu
FROM metrics
WHERE metric_name = 'process.cpu.utilization'
GROUP BY timeBucket
ORDER BY timeBucket
LIMIT 1000
```
### Memory usage by task (past 7d)
Average process memory usage per task identifier over the last 7 days.
```sql
SELECT
task_identifier,
avg(metric_value) AS avg_memory
FROM metrics
WHERE metric_name = 'process.memory.usage'
GROUP BY task_identifier
ORDER BY avg_memory DESC
LIMIT 20
```
### Available metric names
List all distinct metric names collected in your environment.
```sql
SELECT
metric_name,
count() AS sample_count
FROM metrics
GROUP BY metric_name
ORDER BY sample_count DESC
LIMIT 100
```
## Best practices
1. **Use the built-in time filtering** - The dashboard, API, and SDK have a time filter that is applied automatically and is easier to work with. It means the queries can be executed with multiple periods easily.
2. **Use LIMIT** - Add a `LIMIT` clause to reduce the rows returned if you don't need everything.
3. **Use appropriate aggregations** - For large datasets, use `uniq()` instead of `uniqExact()` for approximate but faster counts
## Limits
We have several limits to prevent abuse and ensure performance:
- **Concurrency limit**: We limit the number of concurrent queries per organization.
- **Row limit**: We limit the number of rows returned to 10k.
- **Time restrictions**: We limit the time period you can query.
- **Time/Memory limit**: We limit the memory a query can use and the time it can run for. As well as other limits like AST complexity.
See [Limits](/limits) for current quota details.