chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,182 @@
|
||||
---
|
||||
title: "Bulk actions"
|
||||
description: "Cancel or replay many runs from the SDK using run IDs or SDK run-list filters."
|
||||
---
|
||||
|
||||
**Bulk actions let you cancel or replay many runs asynchronously from the SDK by selecting runs with run IDs or SDK run-list filters.**
|
||||
|
||||
A bulk action returns a handle immediately. Use the handle to retrieve progress, poll until completion, list previous actions, or abort pending work.
|
||||
|
||||
## Create a bulk replay
|
||||
|
||||
Use `runs.bulk.replay()` to replay every run that matches a filter.
|
||||
|
||||
```ts Your backend code
|
||||
import { runs } from "@trigger.dev/sdk";
|
||||
|
||||
const action = await runs.bulk.replay({
|
||||
filter: {
|
||||
status: "FAILED",
|
||||
taskIdentifier: "sync-customer",
|
||||
period: "24h",
|
||||
},
|
||||
name: "Replay failed customer syncs",
|
||||
targetRegion: "eu-central-1",
|
||||
});
|
||||
|
||||
const completed = await runs.bulk.poll(action.id);
|
||||
console.log(completed.status, completed.counts);
|
||||
```
|
||||
|
||||
`filter` accepts the SDK run-list filter fields, excluding pagination fields such as `limit`, `after`, and `before`. This is the TypeScript SDK shape used by `runs.list()` (`from`, `to`, and `period` are top-level fields), not the nested `filter[createdAt]` query shape shown in the raw HTTP list-runs endpoint. Provide at least one filter field; use `runIds` when you want to target specific runs. Relative time filters such as `period` are resolved when the bulk action is created, so later batches process the same fixed time range.
|
||||
|
||||
<Warning>
|
||||
Filters inherit the same time semantics as `runs.list()`: when you don't pass `from`, `to`, or `period`, the action defaults to the **last 7 days** and won't target older matching runs. To cover a wider range, pass an explicit `period` (such as `"30d"`), a `from` timestamp, or a `from`/`to` pair. This default only applies to `filter` selections; `runIds` selections are never time-bounded.
|
||||
</Warning>
|
||||
|
||||
<ParamField body="filter" type="BulkActionFilter">
|
||||
Selects runs using the SDK run-list filter shape, excluding `limit`, `after`, and `before`. In raw HTTP requests these fields are sent as JSON body properties, so time fields are top-level (`from`, `to`, `period`) instead of nested under `createdAt`.
|
||||
</ParamField>
|
||||
|
||||
<ParamField body="runIds" type="string[]">
|
||||
Selects specific run IDs. Provide either `filter` or `runIds`, not both.
|
||||
</ParamField>
|
||||
|
||||
<ParamField body="name" type="string" optional>
|
||||
A name for the bulk action.
|
||||
</ParamField>
|
||||
|
||||
<ParamField body="targetRegion" type="string" optional>
|
||||
Replays matching runs in a specific region. When omitted, each replay keeps the original run's region. This option is only available for `runs.bulk.replay()`.
|
||||
</ParamField>
|
||||
|
||||
## Create a bulk cancel
|
||||
|
||||
Use `runs.bulk.cancel()` to cancel every run that matches a filter, or specific run IDs.
|
||||
|
||||
```ts Your backend code
|
||||
import { runs } from "@trigger.dev/sdk";
|
||||
|
||||
const action = await runs.bulk.cancel({
|
||||
runIds: ["run_1234", "run_5678"],
|
||||
name: "Cancel selected runs",
|
||||
});
|
||||
|
||||
console.log(action.id);
|
||||
```
|
||||
|
||||
Only runs that are still cancelable when the action reaches them are canceled. Runs that have already reached a final state count as failures in the bulk action summary.
|
||||
|
||||
## Retrieve progress
|
||||
|
||||
Use `runs.bulk.retrieve()` to read the current status and aggregate counts.
|
||||
|
||||
```ts Your backend code
|
||||
import { runs } from "@trigger.dev/sdk";
|
||||
|
||||
const action = await runs.bulk.retrieve("bulk_1234");
|
||||
|
||||
console.log(action.status);
|
||||
console.log(action.counts.total, action.counts.success, action.counts.failure);
|
||||
```
|
||||
|
||||
The returned bulk action object has these fields:
|
||||
|
||||
<ResponseField name="id" type="string">
|
||||
The bulk action ID, starting with `bulk_`.
|
||||
</ResponseField>
|
||||
|
||||
<ResponseField name="type" type="'CANCEL' | 'REPLAY'">
|
||||
The action being performed.
|
||||
</ResponseField>
|
||||
|
||||
<ResponseField name="status" type="'PENDING' | 'COMPLETED' | 'ABORTED'">
|
||||
The current bulk action status.
|
||||
</ResponseField>
|
||||
|
||||
<ResponseField name="counts" type="object">
|
||||
Aggregate processing counts.
|
||||
|
||||
<Expandable title="properties">
|
||||
<ResponseField name="total" type="number">
|
||||
The number of runs selected when the bulk action was created.
|
||||
</ResponseField>
|
||||
<ResponseField name="success" type="number">
|
||||
The number of runs processed successfully.
|
||||
</ResponseField>
|
||||
<ResponseField name="failure" type="number">
|
||||
The number of runs that could not be processed.
|
||||
</ResponseField>
|
||||
</Expandable>
|
||||
</ResponseField>
|
||||
|
||||
<ResponseField name="createdAt" type="Date">
|
||||
The date and time the bulk action was created.
|
||||
</ResponseField>
|
||||
|
||||
<ResponseField name="completedAt" type="Date" optional>
|
||||
The date and time the bulk action completed.
|
||||
</ResponseField>
|
||||
|
||||
## Poll for completion
|
||||
|
||||
Use `runs.bulk.poll()` to wait until the bulk action leaves the `PENDING` state.
|
||||
|
||||
```ts Your backend code
|
||||
import { runs } from "@trigger.dev/sdk";
|
||||
|
||||
const completed = await runs.bulk.poll("bulk_1234", {
|
||||
pollIntervalMs: 2_000,
|
||||
});
|
||||
|
||||
console.log(completed.status);
|
||||
```
|
||||
|
||||
## Abort a bulk action
|
||||
|
||||
Use `runs.bulk.abort()` to stop future batches from being processed.
|
||||
|
||||
```ts Your backend code
|
||||
import { runs } from "@trigger.dev/sdk";
|
||||
|
||||
await runs.bulk.abort("bulk_1234");
|
||||
```
|
||||
|
||||
Abort is best effort. Runs already being processed in the current batch may still finish.
|
||||
|
||||
<Note>
|
||||
Each environment can only run a limited number of bulk replays at the same time. If you start a replay while too many are still in progress, the call fails and returns an error. Wait for an in-progress replay to finish or abort one before starting another. Bulk cancels are not subject to this limit.
|
||||
</Note>
|
||||
|
||||
## List bulk actions
|
||||
|
||||
Use `runs.bulk.list()` to page through previous bulk actions in the current environment.
|
||||
|
||||
```ts Your backend code
|
||||
import { runs } from "@trigger.dev/sdk";
|
||||
|
||||
const page = await runs.bulk.list({ limit: 25 });
|
||||
|
||||
for (const action of page.data) {
|
||||
console.log(action.id, action.status);
|
||||
}
|
||||
```
|
||||
|
||||
List results support the same auto-pagination helpers as other management API list methods:
|
||||
|
||||
```ts Your backend code
|
||||
import { runs } from "@trigger.dev/sdk";
|
||||
|
||||
for await (const action of runs.bulk.list({ limit: 25 })) {
|
||||
console.log(action.id, action.status);
|
||||
}
|
||||
```
|
||||
|
||||
## API reference
|
||||
|
||||
The SDK methods use the bulk actions HTTP API:
|
||||
|
||||
- [Create bulk action](/management/bulk-actions/create)
|
||||
- [List bulk actions](/management/bulk-actions/list)
|
||||
- [Retrieve bulk action](/management/bulk-actions/retrieve)
|
||||
- [Abort bulk action](/management/bulk-actions/abort)
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
title: "Heartbeats"
|
||||
sidebarTitle: "Heartbeats"
|
||||
description: "Keep long-running or CPU-heavy tasks from being marked as stalled."
|
||||
---
|
||||
|
||||
We send a heartbeat from your task to the platform every 30 seconds. If we don't receive a heartbeat within 5 minutes, we mark the run as stalled and stop it with a `TASK_RUN_STALLED_EXECUTING` error.
|
||||
|
||||
Code that blocks the event loop for too long (for example, a tight loop doing synchronous work on a large dataset) can prevent heartbeats from being sent. In that case, use `heartbeats.yield()` inside the loop so the runtime can yield to the event loop and send a heartbeat. You can call it every iteration; the implementation only yields when needed.
|
||||
|
||||
```ts
|
||||
import { task, heartbeats } from "@trigger.dev/sdk";
|
||||
|
||||
export const processLargeDataset = task({
|
||||
id: "process-large-dataset",
|
||||
run: async (payload: { items: string[] }) => {
|
||||
for (const row of payload.items) {
|
||||
await heartbeats.yield();
|
||||
processRow(row);
|
||||
}
|
||||
return { processed: payload.items.length };
|
||||
},
|
||||
});
|
||||
|
||||
function processRow(row: string) {
|
||||
// synchronous CPU-heavy work
|
||||
}
|
||||
```
|
||||
|
||||
If you see `TASK_RUN_STALLED_EXECUTING`, see [Task run stalled executing](/troubleshooting#task-run-stalled-executing) in the troubleshooting guide.
|
||||
|
||||
## Sending progress to Trigger.dev
|
||||
|
||||
To stream progress or status updates to the dashboard and your app, use [run metadata](/runs/metadata). Call `metadata.set()` (or `metadata.append()`) as the task runs. The dashboard and [Realtime](/realtime) (including `runs.subscribeToRun` and the React hooks) receive those updates as they happen. See [Progress monitoring](/realtime/backend/subscribe#progress-monitoring) for a full example.
|
||||
|
||||
## Sending updates to your own system
|
||||
|
||||
Trigger.dev doesn’t push run updates to external services. To send progress or heartbeats to your own backend (for example Supabase Realtime), call your API or client from inside the task when you want to emit an update—e.g. in the same loop where you call `heartbeats.yield()` or `metadata.set()`. Use whatever your stack supports: HTTP, the Supabase client, or another SDK.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 29 KiB |
@@ -0,0 +1,139 @@
|
||||
---
|
||||
title: "Max duration"
|
||||
sidebarTitle: "Max duration"
|
||||
description: "Set a maximum duration for a task to run."
|
||||
---
|
||||
|
||||
The `maxDuration` parameter sets a maximum compute time limit for tasks. When a task exceeds this duration, it will be automatically stopped. This helps prevent runaway tasks and manage compute resources effectively.
|
||||
|
||||
You must set a default maxDuration in your `trigger.config.ts` file, which will apply to all tasks unless overridden:
|
||||
|
||||
```ts /config/trigger.config.ts
|
||||
import { defineConfig } from "@trigger.dev/sdk";
|
||||
|
||||
export default defineConfig({
|
||||
project: "proj_gtcwttqhhtlasxgfuhxs",
|
||||
maxDuration: 60, // 60 seconds or 1 minute
|
||||
});
|
||||
```
|
||||
|
||||
<Note>
|
||||
The minimum maxDuration is 5 seconds. If you want to avoid timeouts, set this value to a very large number of seconds.
|
||||
</Note>
|
||||
|
||||
You can set the `maxDuration` for a run in the following ways:
|
||||
|
||||
- Across all your tasks in the [config](/config/config-file#max-duration)
|
||||
- On a specific task
|
||||
- On a specific run when you [trigger a task](/triggering#maxduration)
|
||||
|
||||
## How it works
|
||||
|
||||
The `maxDuration` is set in seconds, and is compared to the CPU time elapsed since the start of a single execution (which we call [attempts](/runs#attempts)) of the task. The CPU time is the time that the task has been actively running on the CPU, and does not include time spent waiting during the following:
|
||||
|
||||
- `wait.for` calls
|
||||
- `triggerAndWait` calls
|
||||
- `batchTriggerAndWait` calls
|
||||
|
||||
You can inspect the CPU time of a task inside the run function with our `usage` utility:
|
||||
|
||||
```ts /trigger/max-duration.ts
|
||||
import { task, usage } from "@trigger.dev/sdk";
|
||||
|
||||
export const maxDurationTask = task({
|
||||
id: "max-duration-task",
|
||||
maxDuration: 300, // 300 seconds or 5 minutes
|
||||
run: async (payload: any, { ctx }) => {
|
||||
let currentUsage = usage.getCurrent();
|
||||
|
||||
currentUsage.attempt.durationMs; // The CPU time in milliseconds since the start of the run
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
The above value will be compared to the `maxDuration` you set. If the task exceeds the `maxDuration`, it will be stopped with the following error:
|
||||
|
||||
|
||||
|
||||
## Configuring for a task
|
||||
|
||||
You can set a `maxDuration` on a specific task:
|
||||
|
||||
```ts /trigger/max-duration-task.ts
|
||||
import { task } from "@trigger.dev/sdk";
|
||||
|
||||
export const maxDurationTask = task({
|
||||
id: "max-duration-task",
|
||||
maxDuration: 300, // 300 seconds or 5 minutes
|
||||
run: async (payload: any, { ctx }) => {
|
||||
//...
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
This will override the default `maxDuration` set in the config file. If you have a config file with a default `maxDuration` of 60 seconds, and you set a `maxDuration` of 300 seconds on a task, the task will run for 300 seconds.
|
||||
|
||||
You can "turn off" the Max duration set in your config file for a specific task like so:
|
||||
|
||||
```ts /trigger/max-duration-task.ts
|
||||
import { task, timeout } from "@trigger.dev/sdk";
|
||||
|
||||
export const maxDurationTask = task({
|
||||
id: "max-duration-task",
|
||||
maxDuration: timeout.None, // No max duration
|
||||
run: async (payload: any, { ctx }) => {
|
||||
//...
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Configuring for a run
|
||||
|
||||
You can set a `maxDuration` on a specific run when you trigger a task:
|
||||
|
||||
```ts /trigger/max-duration.ts
|
||||
import { maxDurationTask } from "./trigger/max-duration-task";
|
||||
|
||||
// Trigger the task with a maxDuration of 300 seconds
|
||||
const run = await maxDurationTask.trigger(
|
||||
{ foo: "bar" },
|
||||
{
|
||||
maxDuration: 300, // 300 seconds or 5 minutes
|
||||
}
|
||||
);
|
||||
```
|
||||
|
||||
You can also set the `maxDuration` to `timeout.None` to turn off the max duration for a specific run:
|
||||
|
||||
```ts /trigger/max-duration.ts
|
||||
import { maxDurationTask } from "./trigger/max-duration-task";
|
||||
import { timeout } from "@trigger.dev/sdk";
|
||||
|
||||
// Trigger the task with no maxDuration
|
||||
const run = await maxDurationTask.trigger(
|
||||
{ foo: "bar" },
|
||||
{
|
||||
maxDuration: timeout.None, // No max duration
|
||||
}
|
||||
);
|
||||
```
|
||||
|
||||
## maxDuration in run context
|
||||
|
||||
You can access the `maxDuration` set for a run in the run context:
|
||||
|
||||
```ts /trigger/max-duration-task.ts
|
||||
import { task } from "@trigger.dev/sdk";
|
||||
|
||||
export const maxDurationTask = task({
|
||||
id: "max-duration-task",
|
||||
maxDuration: 300, // 300 seconds or 5 minutes
|
||||
run: async (payload: any, { ctx }) => {
|
||||
console.log(ctx.run.maxDuration); // 300
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## maxDuration and lifecycle functions
|
||||
|
||||
When a task run exceeds the `maxDuration`, the lifecycle functions `cleanup`, `onSuccess`, and `onFailure` will not be called.
|
||||
@@ -0,0 +1,734 @@
|
||||
---
|
||||
title: "Run metadata"
|
||||
sidebarTitle: "Metadata"
|
||||
description: "Attach structured data to a run and update it as the task progresses. Use metadata for progress tracking, user context, intermediate results, and more."
|
||||
---
|
||||
|
||||
**Metadata lets you attach up to 256KB of structured data to a run and update it while the task runs.** Subscribers (via [React hooks](/realtime/react-hooks/subscribe) or [backend](/realtime/backend/subscribe)) get those updates in real time, making metadata the simplest way to build progress bars, status indicators, and live dashboards.
|
||||
|
||||
You can access metadata from inside the run function, via the API, Realtime, and in the dashboard. Common uses: progress percentage, current step, user context, intermediate results.
|
||||
|
||||
## Usage
|
||||
|
||||
Add metadata to a run when triggering by passing it as an object to the `trigger` function:
|
||||
|
||||
```ts
|
||||
const handle = await myTask.trigger(
|
||||
{ message: "hello world" },
|
||||
{ metadata: { user: { name: "Eric", id: "user_1234" } } }
|
||||
);
|
||||
```
|
||||
|
||||
You can get the current metadata at any time by calling `metadata.get()` or `metadata.current()` (only inside a run):
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// Get the whole metadata object
|
||||
const currentMetadata = metadata.current();
|
||||
console.log(currentMetadata);
|
||||
|
||||
// Get a specific key
|
||||
const user = metadata.get("user");
|
||||
console.log(user.name); // "Eric"
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Any of these methods can be called anywhere "inside" the run function, or a function called from the run function:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
doSomeWork();
|
||||
},
|
||||
});
|
||||
|
||||
async function doSomeWork() {
|
||||
// Set the value of a specific key
|
||||
metadata.set("progress", 0.5);
|
||||
}
|
||||
```
|
||||
|
||||
If you call any of the metadata methods outside of the run function, they will have no effect:
|
||||
|
||||
```ts
|
||||
import { metadata } from "@trigger.dev/sdk";
|
||||
|
||||
// Somewhere outside of the run function
|
||||
function doSomeWork() {
|
||||
metadata.set("progress", 0.5); // This will do nothing
|
||||
}
|
||||
```
|
||||
|
||||
This means it's safe to call these methods anywhere in your code, and they will only have an effect when called inside the run function.
|
||||
|
||||
<Note>
|
||||
Calling `metadata.current()` or `metadata.get()` outside of the run function will always return
|
||||
undefined.
|
||||
</Note>
|
||||
|
||||
These methods also work inside any task lifecycle hook, either attached to the specific task or the global hooks defined in your `trigger.config.ts` file.
|
||||
|
||||
<CodeGroup>
|
||||
|
||||
```ts myTasks.ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// Your run function work here
|
||||
},
|
||||
onStart: async () => {
|
||||
metadata.set("progress", 0.5);
|
||||
},
|
||||
onSuccess: async () => {
|
||||
metadata.set("progress", 1.0);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
```ts trigger.config.ts
|
||||
import { defineConfig, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export default defineConfig({
|
||||
project: "proj_1234",
|
||||
onStart: async () => {
|
||||
metadata.set("progress", 0.5);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
</CodeGroup>
|
||||
|
||||
## Updates API
|
||||
|
||||
One of the more powerful features of metadata is the ability to update it as the run progresses. This is useful for tracking the progress of a run, storing intermediate results, or storing any other information that changes over time. (Combining metadata with [Realtime](/realtime) can give you a live view of the progress of your runs.)
|
||||
|
||||
All metadata update methods (accept for `flush` and `stream`) are synchronous and will not block the run function. We periodically flush metadata to the database in the background, so you can safely update the metadata inside a run as often as you need to, without worrying about impacting the run's performance.
|
||||
|
||||
### set
|
||||
|
||||
Set the value of a key in the metadata object:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// Do some work
|
||||
metadata.set("progress", 0.1);
|
||||
|
||||
// Do some more work
|
||||
metadata.set("progress", 0.5);
|
||||
|
||||
// Do even more work
|
||||
metadata.set("progress", 1.0);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### del
|
||||
|
||||
Delete a key from the metadata object:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// Do some work
|
||||
metadata.set("progress", 0.1);
|
||||
|
||||
// Do some more work
|
||||
metadata.set("progress", 0.5);
|
||||
|
||||
// Remove the progress key
|
||||
metadata.del("progress");
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### replace
|
||||
|
||||
Replace the entire metadata object with a new object:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// Do some work
|
||||
metadata.set("progress", 0.1);
|
||||
|
||||
// Replace the metadata object
|
||||
metadata.replace({ user: { name: "Eric", id: "user_1234" } });
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### append
|
||||
|
||||
Append a value to an array in the metadata object:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// Do some work
|
||||
metadata.set("progress", 0.1);
|
||||
|
||||
// Append a value to an array
|
||||
metadata.append("logs", "Step 1 complete");
|
||||
|
||||
console.log(metadata.get("logs")); // ["Step 1 complete"]
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### remove
|
||||
|
||||
Remove a value from an array in the metadata object:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// Do some work
|
||||
metadata.set("progress", 0.1);
|
||||
|
||||
// Append a value to an array
|
||||
metadata.append("logs", "Step 1 complete");
|
||||
|
||||
// Remove a value from the array
|
||||
metadata.remove("logs", "Step 1 complete");
|
||||
|
||||
console.log(metadata.get("logs")); // []
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### increment
|
||||
|
||||
Increment a numeric value in the metadata object:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// Do some work
|
||||
metadata.set("progress", 0.1);
|
||||
|
||||
// Increment a value
|
||||
metadata.increment("progress", 0.4);
|
||||
|
||||
console.log(metadata.get("progress")); // 0.5
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### decrement
|
||||
|
||||
Decrement a numeric value in the metadata object:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// Do some work
|
||||
metadata.set("progress", 0.5);
|
||||
|
||||
// Decrement a value
|
||||
metadata.decrement("progress", 0.4);
|
||||
|
||||
console.log(metadata.get("progress")); // 0.1
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### stream
|
||||
|
||||
<Note>
|
||||
As of SDK version **4.1.0**, `metadata.stream()` has been replaced by [Realtime Streams
|
||||
v2](/tasks/streams). We recommend using the new `streams.pipe()` API for better reliability,
|
||||
unlimited stream length, and improved developer experience. The examples below are provided for
|
||||
backward compatibility.
|
||||
</Note>
|
||||
|
||||
Capture a stream of values and make the stream available when using Realtime. See our [Realtime Streams v2](/tasks/streams) documentation for the recommended approach.
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
const readableStream = new ReadableStream({
|
||||
start(controller) {
|
||||
controller.enqueue("Step 1 complete");
|
||||
controller.enqueue("Step 2 complete");
|
||||
controller.enqueue("Step 3 complete");
|
||||
controller.close();
|
||||
},
|
||||
});
|
||||
|
||||
// IMPORTANT: you must await the stream method
|
||||
const stream = await metadata.stream("logs", readableStream);
|
||||
|
||||
// You can read from the returned stream locally
|
||||
for await (const value of stream) {
|
||||
console.log(value);
|
||||
}
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
`metadata.stream` accepts any `AsyncIterable` or `ReadableStream` object. The stream will be captured and made available in the Realtime API. So for example, you could pass the body of a fetch response to `metadata.stream` to capture the response body and make it available in Realtime:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { url: string }) => {
|
||||
logger.info("Streaming response", { url });
|
||||
|
||||
const response = await fetch(url);
|
||||
|
||||
if (!response.body) {
|
||||
throw new Error("Response body is not readable");
|
||||
}
|
||||
|
||||
const stream = await metadata.stream(
|
||||
"fetch",
|
||||
response.body.pipeThrough(new TextDecoderStream())
|
||||
);
|
||||
|
||||
let text = "";
|
||||
|
||||
for await (const chunk of stream) {
|
||||
logger.log("Received chunk", { chunk });
|
||||
|
||||
text += chunk;
|
||||
}
|
||||
|
||||
return { text };
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Or the results of a streaming call to the OpenAI SDK:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
import OpenAI from "openai";
|
||||
|
||||
const openai = new OpenAI({
|
||||
apiKey: process.env.OPENAI_API_KEY,
|
||||
});
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { prompt: string }) => {
|
||||
const completion = await openai.chat.completions.create({
|
||||
messages: [{ role: "user", content: payload.prompt }],
|
||||
model: "gpt-3.5-turbo",
|
||||
stream: true,
|
||||
});
|
||||
|
||||
const stream = await metadata.stream("openai", completion);
|
||||
|
||||
let text = "";
|
||||
|
||||
for await (const chunk of stream) {
|
||||
logger.log("Received chunk", { chunk });
|
||||
|
||||
text += chunk.choices.map((choice) => choice.delta?.content).join("");
|
||||
}
|
||||
|
||||
return { text };
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### flush
|
||||
|
||||
Flush the metadata to the database. The SDK will automatically flush the metadata periodically, so you don't need to call this method unless you need to ensure that the metadata is persisted immediately.
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// Do some work
|
||||
metadata.set("progress", 0.1);
|
||||
|
||||
// Flush the metadata to the database
|
||||
await metadata.flush();
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Fluent API
|
||||
|
||||
All of the update methods can be chained together in a fluent API:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
metadata
|
||||
.set("progress", 0.1)
|
||||
.append("logs", "Step 1 complete")
|
||||
.increment("progress", 0.4)
|
||||
.decrement("otherProgress", 0.1);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Parent & root updates
|
||||
|
||||
Tasks that have been triggered by a parent task (a.k.a. a "child task") can update the metadata of the parent task. This is useful for propagating progress information up the task hierarchy. You can also update the metadata of the root task (root = the initial task that was triggered externally, like from your backend).
|
||||
|
||||
To update the parent task's metadata, use the `metadata.parent` accessor:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myParentTask = task({
|
||||
id: "my-parent-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// Do some work
|
||||
metadata.set("progress", 0.1);
|
||||
|
||||
// Trigger a child task
|
||||
await childTask.triggerAndWait({ message: "hello world" });
|
||||
},
|
||||
});
|
||||
|
||||
export const childTask = task({
|
||||
id: "child-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
// This will update the parent task's metadata
|
||||
metadata.parent.set("progress", 0.5);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
All of the update methods are available on `metadata.parent` and `metadata.root`:
|
||||
|
||||
```ts
|
||||
metadata.parent.set("progress", 0.5);
|
||||
metadata.parent.append("logs", "Step 1 complete");
|
||||
metadata.parent.remove("logs", "Step 1 complete");
|
||||
metadata.parent.increment("progress", 0.4);
|
||||
metadata.parent.decrement("otherProgress", 0.1);
|
||||
metadata.parent.stream("llm", readableStream); // Use streams.pipe() instead (v4.1+)
|
||||
|
||||
metadata.root.set("progress", 0.5);
|
||||
metadata.root.append("logs", "Step 1 complete");
|
||||
metadata.root.remove("logs", "Step 1 complete");
|
||||
metadata.root.increment("progress", 0.4);
|
||||
metadata.root.decrement("otherProgress", 0.1);
|
||||
metadata.root.stream("llm", readableStream); // Use streams.pipe() instead (v4.1+)
|
||||
```
|
||||
|
||||
You can also chain the update methods together:
|
||||
|
||||
```ts
|
||||
metadata.parent
|
||||
.set("progress", 0.1)
|
||||
.append("logs", "Step 1 complete")
|
||||
.increment("progress", 0.4)
|
||||
.decrement("otherProgress", 0.1);
|
||||
```
|
||||
|
||||
### Example
|
||||
|
||||
An example of where you might use parent and root updates is in a task that triggers multiple child tasks in parallel. You could use the parent metadata to track the progress of the child tasks and update the parent task's progress as each child task completes:
|
||||
|
||||
```ts
|
||||
import { CSVRow, UploadedFileData, parseCSVFromUrl } from "@/utils";
|
||||
import { batch, logger, metadata, schemaTask } from "@trigger.dev/sdk";
|
||||
|
||||
export const handleCSVRow = schemaTask({
|
||||
id: "handle-csv-row",
|
||||
schema: CSVRow,
|
||||
run: async (row, { ctx }) => {
|
||||
// Do some work with the row
|
||||
|
||||
// Update the parent task's metadata with the progress of this row
|
||||
metadata.parent.increment("processedRows", 1).append("rowRuns", ctx.run.id);
|
||||
|
||||
return row;
|
||||
},
|
||||
});
|
||||
|
||||
export const handleCSVUpload = schemaTask({
|
||||
id: "handle-csv-upload",
|
||||
schema: UploadedFileData,
|
||||
run: async (file, { ctx }) => {
|
||||
metadata.set("status", "fetching");
|
||||
|
||||
const rows = await parseCSVFromUrl(file.url);
|
||||
|
||||
metadata.set("status", "processing").set("totalRows", rows.length);
|
||||
|
||||
const results = await batch.triggerAndWait<typeof handleCSVRow>(
|
||||
rows.map((row) => ({ id: "handle-csv-row", payload: row }))
|
||||
);
|
||||
|
||||
metadata.set("status", "complete");
|
||||
|
||||
return {
|
||||
file,
|
||||
rows,
|
||||
results,
|
||||
};
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Combined with [Realtime](/realtime), you could use this to show a live progress bar of the CSV processing in your frontend, like this:
|
||||
|
||||
<video
|
||||
src="https://content.trigger.dev/csv-upload-realtime.mp4"
|
||||
preload="auto"
|
||||
controls={true}
|
||||
loop
|
||||
muted
|
||||
autoPlay={true}
|
||||
width="100%"
|
||||
height="100%"
|
||||
/>
|
||||
|
||||
## More metadata task examples
|
||||
|
||||
Using metadata updates in conjunction with our [Realtime React hooks](/realtime/react-hooks/overview) can be a powerful way to build real-time UIs. Here are some example tasks demonstrating how to use metadata in your tasks to track progress, status, and more:
|
||||
|
||||
### Progress tracking
|
||||
|
||||
Track progress with percentage and current step:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const batchProcessingTask = task({
|
||||
id: "batch-processing",
|
||||
run: async (payload: { records: any[] }) => {
|
||||
for (let i = 0; i < payload.records.length; i++) {
|
||||
const record = payload.records[i];
|
||||
|
||||
// Update progress
|
||||
metadata.set("progress", {
|
||||
step: i + 1,
|
||||
total: payload.records.length,
|
||||
percentage: Math.round(((i + 1) / payload.records.length) * 100),
|
||||
currentRecord: record.id,
|
||||
});
|
||||
|
||||
await processRecord(record);
|
||||
}
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### Status updates with logs
|
||||
|
||||
Append log entries while maintaining status:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const deploymentTask = task({
|
||||
id: "deployment",
|
||||
run: async (payload: { version: string }) => {
|
||||
metadata.set("status", "initializing");
|
||||
metadata.append("logs", "Starting deployment...");
|
||||
|
||||
// Step 1
|
||||
metadata.set("status", "building");
|
||||
metadata.append("logs", "Building application...");
|
||||
await buildApplication();
|
||||
|
||||
// Step 2
|
||||
metadata.set("status", "deploying");
|
||||
metadata.append("logs", "Deploying to production...");
|
||||
await deployToProduction();
|
||||
|
||||
// Step 3
|
||||
metadata.set("status", "verifying");
|
||||
metadata.append("logs", "Running health checks...");
|
||||
await runHealthChecks();
|
||||
|
||||
metadata.set("status", "completed");
|
||||
metadata.append("logs", "Deployment successful!");
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### User context and notifications
|
||||
|
||||
Store user information and notification preferences:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const userTask = task({
|
||||
id: "user-task",
|
||||
run: async (payload: { userId: string; action: string }) => {
|
||||
// Set user context in metadata
|
||||
metadata.set("user", {
|
||||
id: payload.userId,
|
||||
action: payload.action,
|
||||
startedAt: new Date().toISOString(),
|
||||
});
|
||||
|
||||
// Update status for user notifications
|
||||
metadata.set("notification", {
|
||||
type: "info",
|
||||
message: `Starting ${payload.action} for user ${payload.userId}`,
|
||||
});
|
||||
|
||||
await performUserAction(payload);
|
||||
|
||||
metadata.set("notification", {
|
||||
type: "success",
|
||||
message: `${payload.action} completed successfully`,
|
||||
});
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Metadata propagation
|
||||
|
||||
Metadata is NOT propagated to child tasks. If you want to pass metadata to a child task, you must do so explicitly:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
await metadata.set("progress", 0.5);
|
||||
await childTask.trigger(payload, { metadata: metadata.current() });
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Type-safe metadata
|
||||
|
||||
The metadata APIs are currently loosely typed, accepting any object that is JSON-serializable:
|
||||
|
||||
```ts
|
||||
// ❌ You can't pass a top-level array
|
||||
const handle = await myTask.trigger(
|
||||
{ message: "hello world" },
|
||||
{ metadata: [{ user: { name: "Eric", id: "user_1234" } }] }
|
||||
);
|
||||
|
||||
// ❌ You can't pass a string as the entire metadata:
|
||||
const handle = await myTask.trigger(
|
||||
{ message: "hello world" },
|
||||
{ metadata: "this is the metadata" }
|
||||
);
|
||||
|
||||
// ❌ You can't pass in a function or a class instance
|
||||
const handle = await myTask.trigger(
|
||||
{ message: "hello world" },
|
||||
{ metadata: { user: () => "Eric", classInstance: new HelloWorld() } }
|
||||
);
|
||||
|
||||
// ✅ You can pass in dates and other JSON-serializable objects
|
||||
const handle = await myTask.trigger(
|
||||
{ message: "hello world" },
|
||||
{ metadata: { user: { name: "Eric", id: "user_1234" }, date: new Date() } }
|
||||
);
|
||||
```
|
||||
|
||||
<Note>
|
||||
If you pass in an object like a Date, it will be serialized to a string when stored in the
|
||||
metadata. That also means that when you retrieve it using `metadata.get()` or
|
||||
`metadata.current()`, you will get a string back. You will need to deserialize it back to a Date
|
||||
object if you need to use it as a Date.
|
||||
</Note>
|
||||
|
||||
We recommend wrapping the metadata API in a [Zod](https://zod.dev) schema (or your validator library of choice) to provide type safety:
|
||||
|
||||
```ts
|
||||
import { task, metadata } from "@trigger.dev/sdk";
|
||||
import { z } from "zod";
|
||||
|
||||
const Metadata = z.object({
|
||||
user: z.object({
|
||||
name: z.string(),
|
||||
id: z.string(),
|
||||
}),
|
||||
date: z.coerce.date(), // Coerce the date string back to a Date object
|
||||
});
|
||||
|
||||
type Metadata = z.infer<typeof Metadata>;
|
||||
|
||||
// Helper function to get the metadata object in a type-safe way
|
||||
// Note: you would probably want to use .safeParse instead of .parse in a real-world scenario
|
||||
function getMetadata() {
|
||||
return Metadata.parse(metadata.current());
|
||||
}
|
||||
|
||||
export const myTask = task({
|
||||
id: "my-task",
|
||||
run: async (payload: { message: string }) => {
|
||||
const metadata = getMetadata();
|
||||
console.log(metadata.user.name); // "Eric"
|
||||
console.log(metadata.user.id); // "user_1234"
|
||||
console.log(metadata.date); // Date object
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Inspecting metadata
|
||||
|
||||
### Dashboard
|
||||
|
||||
You can view the metadata for a run in the Trigger.dev dashboard. The metadata will be displayed in the run details view:
|
||||
|
||||
|
||||
|
||||
### API
|
||||
|
||||
You can use the `runs.retrieve()` SDK function to get the metadata for a run:
|
||||
|
||||
```ts
|
||||
import { runs } from "@trigger.dev/sdk";
|
||||
|
||||
const run = await runs.retrieve("run_1234");
|
||||
|
||||
console.log(run.metadata);
|
||||
```
|
||||
|
||||
See the [API reference](/management/runs/retrieve) for more information.
|
||||
|
||||
## Size limit
|
||||
|
||||
The maximum size of the metadata object is 256KB. If you exceed this limit, the SDK will throw an error. If you are self-hosting Trigger.dev, you can increase this limit by setting the `TASK_RUN_METADATA_MAXIMUM_SIZE` environment variable. For example, to increase the limit to 16KB, you would set `TASK_RUN_METADATA_MAXIMUM_SIZE=16384`.
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
title: "Priority"
|
||||
description: "Specify a priority when triggering a run."
|
||||
---
|
||||
|
||||
You can set a priority when you trigger a run. This allows you to prioritize some of your runs over others, so they are started sooner. This is very useful when:
|
||||
|
||||
- You have critical work that needs to start more quickly (and you have long queues).
|
||||
- You want runs for your premium users to take priority over free users.
|
||||
|
||||
The value for priority is a time offset in seconds that determines the order of dequeuing.
|
||||
|
||||
|
||||
|
||||
If you specify a priority of `10` the run will dequeue before runs that were triggered with no priority 8 seconds ago, like in this example:
|
||||
|
||||
```ts
|
||||
// no priority = 0
|
||||
await myTask.trigger({ foo: "bar" });
|
||||
|
||||
//... imagine 8s pass by
|
||||
|
||||
// this run will start before the run above that was triggered 8s ago (with no priority)
|
||||
await myTask.trigger({ foo: "bar" }, { priority: 10 });
|
||||
```
|
||||
|
||||
If you passed a value of `3600` the run would dequeue before runs that were triggered an hour ago (with no priority).
|
||||
|
||||
<Note>
|
||||
Setting a high priority will not allow you to beat runs from other organizations. It will only affect the order of your own runs.
|
||||
</Note>
|
||||
Reference in New Issue
Block a user