chore: import upstream snapshot with attribution

2026-07-13 13:32:57 +08:00
commit cd420f9332
4811 changed files with 884702 additions and 0 deletions
@@ -0,0 +1,207 @@
+---
+title: "Persistence and replay"
+sidebarTitle: "Persistence and replay"
+description: "How chat.agent rebuilds conversation history at run boot — durable JSON snapshot in object storage plus session.out replay, with a hydrateMessages short-circuit for backend-owned history."
+---
+
+`chat.agent` runs are processes — they boot, stream a turn, and either suspend (waiting for the next message) or exit. When the next message arrives at a session whose previous run already exited, a **fresh** run boots with no in-memory state. Something has to rebuild the conversation history before that turn can produce a coherent response.
+
+This page walks through the **snapshot + replay** model the runtime uses by default, and the [`hydrateMessages`](/ai-chat/lifecycle-hooks#hydratemessages) short-circuit that turns the whole thing off when the customer owns history.
+
+## Why a snapshot at all
+
+The wire is delta-only: each `.in/append` carries at most one new `UIMessage` (see [Client Protocol](/ai-chat/client-protocol#chattaskwirepayload)). A long conversation might be 50 turns deep with megabytes of tool results — the wire never carries that. So when run #2 boots to handle turn 51, the wire alone tells it almost nothing about turns 1–50.
+
+Two existing pieces of durable state already capture everything that happened:
+
+- **`session.in`** — every user message and tool-approval response ever sent.
+- **`session.out`** — every assistant token, tool call, and tool result the agent emitted, ordered.
+
+Replaying `session.out` from the beginning is correct but expensive — bandwidth scales with chat length, and parsing N megabytes of streamed chunks at every boot adds latency. So the runtime writes a **snapshot** after every turn and reads it on the next boot. Replay only covers the gap between the snapshot's cursor and now.
+
+## The model end-to-end
+
+```mermaid
+sequenceDiagram
+  participant User
+  participant Run1 as Run 1 (turn 1)
+  participant Snapshot as Object storage
+  participant SessionOut as session.out
+  participant Run2 as Run 2 (turn 2+)
+
+  User->>Run1: u1
+  Run1->>SessionOut: assistant chunks for a1
+  Run1->>Run1: onTurnComplete
+  Run1->>Snapshot: write { messages: [u1, a1], lastOutEventId, lastOutTimestamp }
+  Note over Run1: idle suspend (or exit)
+
+  User->>Run2: u2 (delta only)
+  Run2->>Snapshot: read snapshot
+  Run2->>SessionOut: subscribe(lastEventId, wait=0)
+  SessionOut-->>Run2: (empty — nothing since snapshot)
+  Note over Run2: accumulator = [u1, a1]
+  Run2->>Run2: append u2 from wire
+  Run2->>SessionOut: assistant chunks for a2
+  Run2->>Run2: onTurnComplete
+  Run2->>Snapshot: write { messages: [u1, a1, u2, a2], ... }
+```
+
+### Run 1 — first turn
+
+The accumulator starts empty. The wire delivers `u1`. After the model finishes, `onTurnComplete` fires, then the runtime serializes the full accumulator and writes:
+
+```json
+{
+  "version": 1,
+  "savedAt": 1715180400000,
+  "messages": [u1, a1],
+  "lastOutEventId": "42",
+  "lastOutTimestamp": 1715180399000
+}
+```
+
+The key is `packets/{projectRef}/{envSlug}/sessions/{sessionId}/snapshot.json` — overwritten every turn, never appended. The write is **awaited**, not fire-and-forget — if the run idle-suspends immediately after, in-flight promises don't reliably complete and the snapshot would be lost.
+
+### Run 2 — boot
+
+A new run boots when the user sends `u2`. Run 1 has long since exited. Run 2 has no in-memory state. The boot sequence:
+
+<Steps>
+  <Step title="Read the snapshot">
+    GET the JSON blob. On 404 (no snapshot yet — first-ever turn) or read error or version mismatch, treat as empty and continue. Snapshot misses are non-fatal — replay alone may still be sufficient.
+  </Step>
+  <Step title="Replay session.out tail">
+    Subscribe to `session.out` with `wait=0` starting from `snapshot.lastOutEventId`. Drain whatever's there and close. Returns:
+    - **Settled messages** — closed assistant turns past the snapshot cursor (the chunks of a turn that completed after the snapshot was written but before the run exited cleanly).
+    - **A partial assistant** — the trailing message if its stream never received a `finish` chunk. The dead run was mid-response when it died. `cleanupAbortedParts` has already stripped streaming-in-progress fragments.
+
+    In the steady state this returns empty. In recovery, it returns whatever the dead run was in the middle of.
+  </Step>
+  <Step title="Replay session.in tail">
+    GET `session.in` records past the last `turn-complete`'s `session-in-event-id` cursor. Returns the user messages the dead run hadn't acknowledged — typically the message that triggered the cancelled / crashed turn, plus anything the customer typed after.
+  </Step>
+  <Step title="Reconstruct the chain (smart default)">
+    Snapshot messages merge with the settled replay (replay wins on `id` collision). Then:
+
+    - If there's a partial assistant **and** at least one in-flight user message, splice `[firstInFlightUser, partialAssistant]` onto the end of the chain. The model sees the prior turn's incomplete attempt and can continue, abandon, or pivot based on the next user message.
+    - Remaining in-flight users dispatch as fresh turns after the recovered first one.
+    - If there's no partial OR no in-flight users, the chain is just the settled chain and any in-flight users dispatch normally.
+
+    Customers can override this entirely via [`onRecoveryBoot`](/ai-chat/patterns/recovery-boot).
+  </Step>
+  <Step title="Append the new wire message">
+    Append `u2` from the wire payload, exactly as on turn 1.
+  </Step>
+</Steps>
+
+The model now sees `[u1, a1, u2]` and produces `a2`. After `onTurnComplete`, the runtime overwrites the snapshot with `[u1, a1, u2, a2]` and the cycle repeats.
+
+### Crash mid-turn — replay carries the load
+
+Suppose Run 1's turn 1 streams partial assistant chunks to `session.out` and then crashes (OOM, exception, server-side cancel) before `onTurnComplete` fires. No snapshot was written. The next run boots and:
+
+1. Snapshot read returns 404 → empty.
+2. `session.out` tail replay picks up the partial assistant chunks emitted before the crash. `cleanupAbortedParts` strips streaming-in-progress fragments but keeps the cleaned trailing message as the `partialAssistant`.
+3. `session.in` tail replay finds the user message the dead run was answering (no `turn-complete` was written, so the cursor never advanced past it).
+4. Smart default splices `[firstInFlightUser, partialAssistant]` onto the chain. Any later user messages (including the customer's follow-up) dispatch as fresh turns.
+5. The model sees full prior context and responds in kind — continuing a cut-off essay on "keep going", answering a fresh question on "actually, what's 7+8?", abandoning the prior work on "scrap that, do X instead".
+
+Replay carries the conversation across the crash boundary with zero customer code. For policies different from "preserve context" — drop the partial entirely, synthesize tool results for an interrupted tool call, write a recovery banner to the UI — register [`onRecoveryBoot`](/ai-chat/patterns/recovery-boot).
+
+## OOM-retry interaction
+
+The runtime already had an OOM-retry path that scans `session.out` for the latest `trigger:turn-complete` timestamp to use as a cutoff for `session.in` (so the retry doesn't re-process completed turns — see [OOM resilience](/ai-chat/patterns/oom-resilience)). The snapshot includes a `lastOutTimestamp` field that is exactly that high-water mark.
+
+When a snapshot exists, the OOM-retry path reads `lastOutTimestamp` directly instead of scanning `session.out`. One fewer stream subscription per retry. Free win.
+
+If no snapshot exists (first turn, or `hydrateMessages` registered), the path falls back to the scan.
+
+## Action turns — no snapshot write
+
+[Action turns](/ai-chat/actions) (`trigger: "action"`) don't fire `onTurnComplete` — they fire `onAction` only. The snapshot write site is gated on `onTurnComplete`, so action turns don't snapshot.
+
+If `onAction` mutates `chat.history.*` and then the run crashes before the next regular turn, the mutation is lost. The user re-fires the action. This matches `chat.history` semantics in general — mutations are persisted at turn boundaries, not action boundaries.
+
+## The `hydrateMessages` short-circuit
+
+When the customer registers a [`hydrateMessages`](/ai-chat/lifecycle-hooks#hydratemessages) hook, the runtime trusts the hook to be the source of truth for history. Snapshot read and replay are **skipped entirely** at boot. The hook fires per turn, returns the canonical chain from the customer's database, and the accumulator is set to whatever the hook returned.
+
+```ts
+import { chat, upsertIncomingMessage } from "@trigger.dev/sdk/ai";
+import { db } from "@/lib/db";
+
+export const myChat = chat.agent({
+  id: "my-chat",
+  hydrateMessages: async ({ chatId, trigger, incomingMessages }) => {
+    const stored = (await db.chat.findUnique({ where: { id: chatId } }))?.messages ?? [];
+
+    // See lifecycle-hooks for the full upsert pattern + rationale:
+    // /ai-chat/lifecycle-hooks#hydratemessages
+    if (upsertIncomingMessage(stored, { trigger, incomingMessages })) {
+      // Upsert, not update: head-start first turns run without a preload
+      // to create the row.
+      await db.chat.upsert({
+        where: { id: chatId },
+        create: { id: chatId, messages: stored },
+        update: { messages: stored },
+      });
+    }
+
+    return stored;
+  },
+  onTurnComplete: async ({ chatId, uiMessages }) => {
+    await db.chat.update({ where: { id: chatId }, data: { messages: uiMessages } });
+  },
+  run: async ({ messages, signal }) => {
+    return streamText({ model: anthropic("claude-sonnet-4-5"), messages, abortSignal: signal });
+  },
+});
+```
+
+What you gain:
+
+- **Zero object-store traffic per turn.** No snapshot read, no snapshot write, no replay subscription. `OBJECT_STORE_*` env vars don't have to be set.
+- **Branching, undo, edit, abuse prevention** — patterns that need a backend-side single source of truth work naturally because the customer mediates every read.
+
+What you give up:
+
+- **You own persistence end-to-end.** A bug in `hydrateMessages` that returns the wrong chain corrupts the conversation visible to the model.
+- **OOM-retry needs a `session.out` scan again** because there's no snapshot to short-circuit it. (Same as the pre-snapshot baseline — not a regression, just a missed optimization.)
+
+The runtime's snapshot+replay is the safer default. `hydrateMessages` is the right choice when you already have authoritative storage for messages and want one consistent persistence path.
+
+## When neither is configured
+
+If `hydrateMessages` is not registered **and** no object store is configured, conversations don't survive run boundaries. A continuation boots empty. The runtime logs a warning at agent registration time so you see this at deploy time, not at user-traffic time.
+
+For local development this is sometimes fine — you're not testing continuations. For production it isn't. Configure one of:
+
+- **Object store** (`OBJECT_STORE_*` env vars on your webapp) — easiest, default behavior.
+- **`hydrateMessages` + your own database** — stronger control, suits multi-tenant apps with audit needs.
+
+## Snapshot key & lifecycle
+
+| Field | Value |
+|---|---|
+| Bucket | Whatever `OBJECT_STORE_BASE_URL` points to |
+| Key prefix | `packets/{projectRef}/{envSlug}/` (server-prefixed) |
+| Key suffix | `sessions/{sessionId}/snapshot.json` |
+| Final key | `packets/{projectRef}/{envSlug}/sessions/{sessionId}/snapshot.json` |
+| Size | Tens of KB typical, capped only by object-store limits |
+| Cadence | Overwritten after every successful `onTurnComplete` |
+
+Snapshots accumulate per-session forever unless you set a lifecycle policy on the bucket. A 90-day expiry on `packets/*/sessions/*/snapshot.json` is a reasonable default if your chats don't typically resume after that window. Closed sessions are not auto-cleaned today.
+
+### MinIO and S3-compatible stores
+
+Snapshot read/write reuses the same object-store layer as Trigger.dev's existing large-payload routes. Anything that already works for large payloads — AWS S3, MinIO (self-host or local development), Cloudflare R2, Tigris, Backblaze B2 — works for snapshots too. `OBJECT_STORE_DEFAULT_PROTOCOL` controls the routing (`s3`, `minio`, etc.) and the SDK picks the right driver automatically. No snapshot-specific config.
+
+For local development against `pnpm run docker`, the bundled MinIO container is enough — set `OBJECT_STORE_DEFAULT_PROTOCOL=minio` and the standard MinIO env vars on the webapp, and continuations work end-to-end against a local stack.
+
+## See also
+
+- [Client Protocol](/ai-chat/client-protocol#how-history-is-rebuilt) — the wire-level view of the same model
+- [`hydrateMessages`](/ai-chat/lifecycle-hooks#hydratemessages) — the short-circuit hook
+- [OOM resilience](/ai-chat/patterns/oom-resilience) — how `session.in` cutoffs interact with snapshots
+- [Database persistence](/ai-chat/patterns/database-persistence) — the canonical persistence pattern using `onTurnComplete`
+- [v4.5 upgrade guide](/ai-chat/upgrade-guide#v45-wire-format-change) — when this model landed and what changed