yvgude--lean-ctx/docs/concepts/context-time-machine.md

Context Time Machine

North-star concept. This document defines the positioning, the vision, the architecture (built on foundations that already exist in the codebase), and a phased roadmap. The GitLab epic and its sub-issues are derived from here. Status: Phases 0–4 have shipped — the headless engine, the dashboard Time Machine, restore/resume, and signed file-based share/import are all live (see §10). What remains is reach, not foundations: a ctxpkg.com registry and a side-by-side git-diff replay view.

TL;DR

The state of the context layer — what the model saw, why, what it cost, what it was allowed to touch, and what it proved — is today ephemeral and fragmented. The Context Time Machine makes that state a git-anchored, signed, navigable artifact (a Context Snapshot) and gives it three verbs: replay, restore, and share. It is not a new pillar bolted onto lean-ctx; it is the time axis through the five things lean-ctx already does — and the first time the discipline of context engineering becomes literally visible.

---

1. Positioning — five equal areas, one category

Token reduction is not played out — it is acute and growing. "Saturated for GitHub hype" is not the same as "solved problem": more AI usage means more tokens means a bigger problem every day. For paying teams the measurable saving is the concrete ROI — the CFO argument. Compression stays the foundation, not something we leave behind.

The category is already ours: the context engineering layer. We do not need to invent a new label. The five areas are equally important — none dominates:

Verb	Area	What it owns
decides	Compression / smart I/O	what the model reads
remembers	Memory architecture	what it learns
guards	Governance / policy	what it may touch
proves	Verification / evidence	what it saved, signed
replays	Time Machine	the temporal view across the other four

What is weak today is the narrative, not the substance. The Time Machine is not the peak of the stack — it is the window onto it. It makes all five areas visible at once and over time: each frame shows what was read and compressed (decides), what was recalled (remembers), what was blocked or redacted (guards), how much was saved and proven (proves) — all navigable (replays). This is how an abstract discipline becomes legible, the way flame graphs made performance observability legible — without elevating any single area above the others.

Token reduction and the Time Machine are linked, not separate. "Token savings are the receipt." The Time Machine shows that receipt in motion: not just "saved 88%", but "at this commit your agent saw X, which saved Y tokens, here is the proof, rewind and check." Compression produces the number; the Time Machine makes it tangible.

2. The thesis — what we are really building

lean-ctx is the enforcement layer of the Context Stack (ECOSYSTEM.md) — the layer that live-decides, filters and manages what the model sees. The official positioning is four verbs (VISION.md, repo description):

decides what they read · remembers what they learn · guards what they touch · proves what they save.

The gap: this layer state is ephemeral and fragmented. The codebase already persists dozens of stores and ships many partial exports (context_package/, savings_ledger/, stats/model.rs) — but there is no unifying concept for "the state of the layer at time T."

The missing dimension is time. We turn the layer state into a freezable, navigable, reproducible, shareable artifact — anchored to git. This is not a sixth area next to the five; it is the temporal axis through all five.

3. The vision — the five-verb loop

The vision in one sentence. The five verbs are equal; replays closes the loop and makes the other four visible:

lean-ctx decides / remembers / guards / proves — and replays: rewind to any commit and see exactly what the model saw, why (which decision / knowledge / compression, at what Φ-score), and at what token ROI — then reproduce, resume, or share that state.

The loop: decides → remembers → guards → proves → replays → back to decides, now informed by the past.

4. Why it fits holistically

This is not a new island. It is the time axis through the subsystems lean-ctx already runs (perceive, compress, remember, route, govern):

• Perceive / compress → replay shows what the layer chose to surface (read modes, entropy filtering, Φ-selection) and how many tokens that saved.
• Remember → the snapshot is the ultimate memory: the entire layer state, not just facts.
• Route / govern → replay shows what was blocked, redacted, or budgeted — visible policy evidence.
• Verify → we reuse the replay hashes and proof artifacts that already exist; replay is their visible experience.

It also slots into the Context Stack as the temporal extension of ctxpkg:

• ctxpkg today = a spatial distillate (knowledge + graph + session + gotchas).
• ctxpkg + git anchor + IR lineage + proof = a temporal artifact.
• A chain of them = the Time Machine. ctxpkg.com = the shareable version history.
• Result: "git for the context layer" — in lean-ctx's own vocabulary (distill → seal → publish → verify → enforce).

5. Architecture — on existing foundations

We build almost nothing from scratch. The building blocks already exist:

Component	File	Role today
ContextIrV1 (record/save/load)	context_ir.rs	Live lineage (source, safety, verification, tokens, freshness) of every tool call — already a temporal record of what flowed through the layer.
ContextProofV1 (write_project_proof)	context_proof.rs	Signed point-in-time snapshot: project/role/profile identity, ledger summary, evidence receipts, replay hashes.
Context ledger (Φ-scores + item states)	context_ledger.rs	The why behind the model's view: Candidate / Included / Excluded / Pinned / Stale.
Playbook + session diff	session/playbook.rs, session_diff.rs	Incremental checkpoints + state diffing — the frames of the timeline.
Container format	context_package/	The portable, signed .ctxpkg envelope to carry a snapshot.
Knowledge timeline	ctx_knowledge/mod.rs	An already-temporal view over knowledge.

What we built on top (Phases 0–4, now shipped):

• Git anchor ✅ — every snapshot pins to a commit / branch / dirty state (builder.rs).
• Unified Context Snapshot ✅ — one CONTEXT_SNAPSHOT_V1 format bundling IR lineage + ledger Φ + ROI + session slice + git anchor, content-addressed and ed25519-signable (types.rs).
• Timeline index ✅ — append-only, crash-safe index.jsonl (timeline.rs).
• Replay experience ✅ — dashboard Time Machine tab: timeline + per-frame detail (git anchor, ROI, lineage, ledger, session). Open: a side-by-side model-view ｜ git-diff split.
• Restore / resume ✅ — snapshot restore merges the session slice and, with --git, checks out the commit (guarded) (restore.rs).
• Share / import ✅ — snapshot publish/import move a signed, verifiable snapshot file between projects (publish.rs). Open: a ctxpkg.com registry + A2A transport.

flowchart LR
  subgraph live [Live layer state]
    IR["Context IR v1 (lineage)"]
    LEDGER["Context ledger (Phi + states)"]
    SESS["SessionState + playbook"]
    KNOW["Knowledge + gotchas"]
    PROOF["Context Proof v1 (signed)"]
  end
  GIT["Git commit anchor"]
  SNAP["Context Snapshot (temporal ctxpkg)"]
  TL["Timeline index (append-only)"]
  subgraph verbs [Three verbs on the state]
    REPLAY["Replay - dashboard time-travel"]
    RESTORE["Restore / resume"]
    SHARE["Publish to ctxpkg.com"]
  end
  IR --> SNAP
  LEDGER --> SNAP
  SESS --> SNAP
  KNOW --> SNAP
  PROOF --> SNAP
  GIT --> SNAP
  SNAP --> TL --> REPLAY
  SNAP --> RESTORE
  SNAP --> SHARE

6. The artifact: Context Snapshot (temporal .ctxpkg)

A Context Snapshot is a git-anchored, signed, point-in-time state of the layer:

• Anchor — commit SHA, branch, dirty-state hash, time window.
• What the model saw — reconstructable MCP instructions + active ledger items (Φ, view mode), as typed items, not raw text.
• Why — decisions, recalled knowledge, compression choices, policy/guard events.
• ROI — savings-ledger slice + CEP snapshot for that window.
• Proof — ContextProofV1 with replay hashes (Ed25519-signable).

Doctrine-compliant: distilled, typed, signed — never raw transcripts (ECOSYSTEM.md). Deterministic per the output-determinism contract (no timestamps/counters in bodies; content-addressed).

7. The three verbs on the state

• Snapshot (freeze) — lean-ctx snapshot create [--sign]: freeze the layer state at the current commit; list / show / verify browse and prove the timeline. Auto-snapshot on git hooks or session checkpoints stays a future policy toggle.
• Replay (time-travel) — dashboard Time Machine tab: scrub the timeline; per frame the detail panel shows git anchor, ROI, lineage and ledger Φ, and the session behind it. A side-by-side "model view ↔ git diff" remains the next step.
• Restore / resume — lean-ctx snapshot restore <id> [--git]: merge the snapshot's session slice (task, decisions, files) into the live session so the next agent resumes, and optionally check out the commit anchor (guarded against a dirty tree).
• Share / import — lean-ctx snapshot publish <id> writes a signed, verifiable *.ctxsnapshot.json; import <file> proves it and adds it to the recipient's timeline. A ctxpkg.com registry for versioned, hosted history is the future reach.

8. Differentiation

We are honest about what others do well (see docs/comparisons/); the point below is structural, not FUD.

• vs. wire-proxy compressors — a stateless proxy that compresses a request in flight has no persistent, git-anchored state over time. It cannot reconstruct what the model saw three commits ago and why. Our snapshots can.
• vs. conversation recorders — tools that record the raw conversation violate the Stack doctrine "distilled, typed, signed knowledge only — never raw transcripts." We store the distilled, typed, signed state of what the model saw. This is not conversation replay; it is context replay — proof-carrying, offline-verifiable, privacy-respecting. The constraint becomes the moat.

One-liner: "Every other tool records the conversation. lean-ctx records the context — distilled, signed, and replayable. Rewind to any commit and see exactly what your agent saw, why, and what it cost."

9. Naming

• Experience / feature: Context Time Machine (dashboard tab: "Time Machine").
• Artifact: Context Snapshot (CONTEXT_SNAPSHOT_V1; shared as *.ctxsnapshot.json).
• CLI (shipped): lean-ctx snapshot create|list|show|verify|restore|publish|import. MCP ctx_snapshot stays a future surface.
• Strategic frame: "ctxpkg goes temporal" / "version control for context."

10. Phased roadmap (→ GitLab epic + sub-issues)

• Phase 0 — concept & contract ✅ (#1023): this document; a Direction bullet in VISION.md and ECOSYSTEM.md; and the CONTEXT_SNAPSHOT_V1 contract.
• Phase 1 — MVP snapshot (headless) ✅ (#1024): lean-ctx snapshot create/list/show/verify on context_ir + context_ledger + session slice + git anchor. Append-only timeline index. Deterministic, ed25519-signable.
• Phase 2 — replay experience ✅ (#1025): dashboard Time Machine tab — timeline + per-frame detail (git anchor, ROI, lineage, ledger Φ, session) over a JSON control-plane API. Open: a side-by-side model-view ｜ git-diff split.
• Phase 3 — restore / resume ✅ (#1026): snapshot restore <id> [--git] — merge the session slice into the live session; optionally check out the commit anchor (guarded against a dirty tree).
• Phase 4 — share / publish ✅ (#1027): snapshot publish/import move a signed, verifiable snapshot file between projects. Open: a ctxpkg.com registry for hosted, versioned history + A2A transport.

11. Open decisions & risks

• Fidelity vs. size — lightweight (session + knowledge + anchor + proof) vs. full (incl. indexes). Recommendation: MVP lightweight, indexes referenced optionally (no full copy).
• Determinism — must honour the output-determinism contract (no volatile fields in bodies).
• Privacy / redaction — reuse ccp privacy modes; guard events stay typed, never raw content.
• Retention / storage — snapshot lifecycle (decay/archive) analogous to the memory lifecycle.
• Auto-snapshot triggers — git hook vs. session checkpoint vs. manual, configurable as policy.

---

See also: VISION.md · ECOSYSTEM.md · ARCHITECTURE.md