chore: import upstream snapshot with attribution

2026-07-13 12:58:35 +08:00
commit 85453da49f
4031 changed files with 710987 additions and 0 deletions
@@ -0,0 +1,73 @@
+# Common Mistakes
+
+Pitfalls that break HyperFrames compositions that can't be caught by the linter.
+
+> **Linter:** Run `lintHyperframeHtml()` on your composition to catch most issues automatically. The linter detects: missing timeline registration, unmuted video, nested video in timed elements, missing `class="clip"`, deprecated attribute names (`data-layer`, `data-end`), and missing dimensions. See `core/src/lint/` for the full rule list.
+
+---
+
+## 1. Animating video element dimensions with GSAP
+
+**Symptom:** Video frames stop updating, or browser performance drops.
+
+**Cause:** GSAP animating `width`, `height`, `top`, `left` directly on a `<video>` element can cause the browser to stop rendering frames.
+
+```js
+// BROKEN — animating video element dimensions
+tl.to("#el-video", { width: 500, height: 280, top: 700, left: 1400 }, 26);
+
+// FIXED — animate a wrapper div, video fills it at 100%
+tl.to("#pip-wrapper", { width: 500, height: 280, top: 700, left: 1400 }, 26);
+```
+
+Use a non-timed wrapper div for visual effects like picture-in-picture. Animate the wrapper; let the video fill it.
+
+---
+
+## 2. Controlling media playback in scripts
+
+**Symptom:** Audio/video playback is out of sync, or plays when it shouldn't.
+
+**Cause:** Calling `video.play()`, `video.pause()`, `audio.currentTime = ...` in your scripts. The framework owns all media playback.
+
+```js
+// BROKEN — conflicts with framework media sync
+document.getElementById("el-video").play();
+document.getElementById("el-audio").currentTime = 5;
+
+// FIXED — don't do this. The framework handles it.
+// Use GSAP for visual animations only:
+tl.to("#el-video", { opacity: 1, duration: 0.5 }, 0);
+```
+
+---
+
+## 3. Composition duration shorter than video
+
+**Symptom:** Video plays for a few seconds then stops. Timeline shows 8-10 seconds even though the video is minutes long.
+
+**Cause:** The composition duration equals the GSAP timeline duration, not `data-duration` on the video. If your last GSAP animation ends at 8 seconds, the composition is 8 seconds long.
+
+```js
+// BROKEN — timeline is only 8s long, video cuts off
+tl.to("#lower-third", { left: -640, duration: 0.6 }, 7.2);
+// Last tween ends at 7.8s → composition = 7.8s
+
+// FIXED — extend timeline to match video length
+tl.to("#lower-third", { left: -640, duration: 0.6 }, 7.2);
+tl.set({}, {}, 283); // ← extends timeline to 283 seconds
+```
+
+`tl.set({}, {}, TIME)` adds a zero-duration tween at the specified time, which extends the timeline without affecting any elements.
+
+---
+
+## Debugging checklist
+
+When something doesn't work, check in this order:
+
+1. Run the linter: `lintHyperframeHtml(html)` — catches most structural issues
+2. Is `window.__timelines["<id>"]` registered? Does the key match `data-composition-id`?
+3. Are GSAP animations only on visual properties (opacity, transform, color)?
+4. Is the GSAP timeline long enough? (`tl.set({}, {}, DURATION)` at the end)
+5. Open browser console — runtime errors show up as `[Browser:ERROR]` or `[Browser:PAGEERROR]`
@@ -0,0 +1,538 @@
+# HyperFrames Schema
+
+Reference for generating and editing HyperFrames HTML compositions. This is your source of truth for how to author compositions.
+
+**New to HyperFrames?** Start with the [quickstart template](./quickstart-template.html) — a copy-paste composition with inline comments explaining every required piece. See [common mistakes](./common-mistakes.md) for pitfalls that break compositions.
+
+For Frame adapters and deterministic frame rendering direction, see [`../../FRAME.md`](../../FRAME.md) and [`../adapters/README.md`](../adapters/README.md).
+
+Producer-canonical parity note:
+
+- Producer render behavior is the source of truth for deterministic parity.
+- Preview should emulate producer seek semantics (`renderSeek`, frame quantization, readiness gates) in parity mode.
+- Non-parity smooth playback can exist, but parity mode is the correctness baseline.
+
+## Overview
+
+HyperFrames uses HTML as the source of truth for describing a video:
+
+- **HTML clips** = video, image, audio, composition
+- **Data attributes** = timing, metadata, styling
+- **CSS** = positioning and appearance
+- **GSAP timeline** = animations and playback sync
+
+### Framework-Managed Behavior
+
+The framework reads data attributes and automatically manages:
+
+- **Primitive clip timeline entries** — the framework reads `data-start`, `data-duration`, and `data-track-index` from primitive clips and adds them to the composition's GSAP timeline. You do not manually add primitive clips to the timeline in scripts.
+- **Media playback** (play, pause, seek) for `<video>` and `<audio>`
+- **Clip lifecycle** — clips are **mounted** (made visible on screen) and **unmounted** (removed from screen) based on `data-start` and `data-duration`
+- **Timeline synchronization** (keeping media in sync with the GSAP master timeline)
+- **Media loading** — the framework waits for all media elements to load before resolving timing and starting playback
+
+Mounting and unmounting controls **presence**, not appearance. A clip that is mounted is on screen; a clip that is unmounted is not. Transitions (fade in, slide in, etc.) are separate — they are animated in scripts and happen _after_ a clip is mounted or _before_ it is unmounted.
+
+The framework does **not** handle transitions, effects, or visual animation — those are driven by GSAP in JavaScript.
+
+Do not manually call `video.play()`, `video.pause()`, set `audio.currentTime`, or mount/unmount clips in scripts. The framework owns media playback and clip lifecycle. Animating visual properties like `opacity` or `transform` for transitions is fine — that's what scripts are for.
+
+## Viewport
+
+Every composition must include `data-width` and `data-height` so scripts and CSS can reference concrete pixel dimensions for layout. Common sizes:
+
+- **Landscape**: `data-width="1920" data-height="1080"`
+- **Portrait**: `data-width="1080" data-height="1920"`
+
+e.g.,
+
+```html
+<div id="main" data-composition-id="my-video" data-start="0" data-width="1920" data-height="1080">
+  <!-- clips -->
+</div>
+```
+
+Every composition's container is full-screen within the viewport by default. The framework applies full-screen sizing to composition containers automatically.
+
+To position or size individual clips (e.g., picture-in-picture, overlay placement), use standard CSS on the element.
+
+## Compositions
+
+A composition is the fundamental grouping unit in HyperFrames. Every clip — video, image, audio — must live inside a composition. The `index.html` file is itself a composition (the top-level one), and it can contain nested compositions within it. Any composition can be imported into another composition as a sub-composition — there is no special "root" type.
+
+A composition carries the same core attributes as any other clip (`id`, `data-start`, `data-track-index`), so it can be placed and timed on a timeline just like a video or image. A composition's length is determined by its GSAP timeline — there is no `data-duration` on compositions. This means compositions can be nested: a composition clip inside another composition behaves like a self-contained video within the parent timeline.
+
+### Composition File Structure
+
+**Each composition should be defined in its own HTML file.** This keeps compositions modular, reusable, and maintainable. The file contains the complete composition: HTML structure, inline styles, and script.
+
+```
+project/
+├── index.html              # Root composition
+├── compositions/
+│   ├── intro-anim.html     # Intro animation composition
+│   ├── caption-overlay.html # Caption composition
+│   └── outro-title.html     # Outro composition
+```
+
+**Composition file format** (`compositions/intro-anim.html`):
+
+```html
+<!-- Define the composition as a template that can be loaded -->
+<template id="intro-anim-template">
+  <div data-composition-id="intro-anim" data-width="1920" data-height="1080">
+    <div class="title">Welcome!</div>
+    <div class="subtitle">Let's get started</div>
+
+    <style>
+      [data-composition-id="intro-anim"] .title {
+        font-size: 72px;
+        color: white;
+        text-align: center;
+      }
+      [data-composition-id="intro-anim"] .subtitle {
+        font-size: 36px;
+        color: #ccc;
+        text-align: center;
+      }
+    </style>
+
+    <script>
+      const tl = gsap.timeline({ paused: true });
+      tl.from(".title", { opacity: 0, y: -50, duration: 1 });
+      tl.from(".subtitle", { opacity: 0, y: 50, duration: 1 }, 0.5);
+      window.__timelines["intro-anim"] = tl;
+    </script>
+  </div>
+</template>
+```
+
+### Loading Compositions
+
+Use the `data-composition-src` attribute to load a composition from an external HTML file. The framework will automatically fetch the template and instantiate it:
+
+```html
+<div id="comp-1" data-composition-id="my-video" data-start="0" data-width="1920" data-height="1080">
+  <!-- Primitive clips -->
+  <video id="el-1" data-start="0" data-duration="10" data-track-index="0" src="..."></video>
+  <video id="el-2" data-start="el-1" data-duration="8" data-track-index="0" src="..."></video>
+  <img id="el-3" data-start="5" data-duration="4" data-track-index="1" src="..." />
+  <audio id="el-4" data-start="0" data-duration="30" data-track-index="2" src="..." />
+
+  <!-- Load composition from external file -->
+  <div
+    id="el-5"
+    data-composition-id="intro-anim"
+    data-composition-src="compositions/intro-anim.html"
+    data-start="0"
+    data-track-index="3"
+  ></div>
+
+  <!-- Another loaded composition -->
+  <div
+    id="el-6"
+    data-composition-id="captions"
+    data-composition-src="compositions/caption-overlay.html"
+    data-start="0"
+    data-track-index="4"
+  ></div>
+</div>
+```
+
+The framework will:
+
+1. Fetch the HTML file specified in `data-composition-src`
+2. Extract the `<template>` content
+3. Clone and mount it into the composition element
+4. Execute any `<script>` tags within the template
+5. Register the timeline in `window.__timelines`
+
+### Best Practices for Composition Files
+
+**Use separate HTML files when:**
+
+- The composition is reusable across multiple projects or scenes
+- The composition has complex logic, styling, or structure (>20 lines)
+- You want to keep the main `index.html` clean and focused on orchestration
+- The composition represents a distinct functional unit (captions, titles, animations)
+
+**Use inline compositions when:**
+
+- The composition is truly one-off and project-specific
+- The composition is very simple (<10 lines total)
+- You're prototyping and iterating quickly
+
+**File naming conventions:**
+
+- Use kebab-case: `intro-anim.html`, `caption-overlay.html`, `emoji-burst.html`
+- Name files descriptively based on their purpose or visual function
+- Group related compositions in subdirectories if you have many: `compositions/titles/`, `compositions/overlays/`
+
+## Clip Types
+
+A clip is any discrete block on the timeline. We represent clips as HTML elements and apply data-attributes to describe them.
+
+- `<video>` — Video clips, B-roll, A-roll
+- `<img>` — Static images, overlays
+- `<audio>` — Music, sound effects
+- `<div data-composition-id="...">` — Nested compositions (animations, grouped sequences)
+
+## HTML Attributes
+
+### All Clips
+
+- `id` — Unique identifier (e.g., "el-1")
+- `data-start` — Start time in seconds, or a clip `id` reference. See [Relative Timing](#relative-timing).
+- `data-duration` — Duration in seconds. Required for `<img>` clips. Optional for `<video>` and `<audio>` (defaults to the source media's full duration). Not used on compositions.
+- `data-track-index` — Timeline track number. Tracks serve two purposes: they determine visual layering (higher tracks render in front) and they group clips into rows on the timeline. Clips on the same track **cannot overlap in time**.
+
+### Media Clips (video, audio)
+
+- `data-media-start` — (optional) Playback begins at this time in the source file, in seconds. Defaults to `0`.
+
+### Composition Clips
+
+- `data-composition-id` — Unique composition ID
+- `data-composition-src` — (optional) Path to external HTML file containing the composition template. The framework will fetch, instantiate, and mount the template automatically.
+
+> Compositions do **not** use `data-duration`. Their duration is determined by their GSAP timeline.
+
+## Relative Timing
+
+Instead of calculating absolute start times, a clip can reference another clip's `id` in its `data-start` attribute. This means "start when that clip ends." The referenced clip must be in the same composition and must have a known duration (either an explicit `data-duration` or an inferred duration from the source media).
+
+### Basic Sequential Clips
+
+```html
+<video id="intro" data-start="0" data-duration="10" data-track-index="0" src="..."></video>
+<video id="main" data-start="intro" data-duration="20" data-track-index="0" src="..."></video>
+<video id="outro" data-start="main" data-duration="5" data-track-index="0" src="..."></video>
+```
+
+`main` resolves to second 10, `outro` resolves to second 30. If `intro`'s duration changes to 15, `main` and `outro` shift automatically.
+
+### Offsets (gaps and overlaps)
+
+Add `+ N` or `- N` after the ID to offset from the end of the referenced clip:
+
+```html
+<!-- intro ends at 10. "intro + 2" = 10 + 2 = starts at second 12 (2s gap) -->
+<video
+  id="scene-a"
+  data-start="intro + 2"
+  data-duration="20"
+  data-track-index="0"
+  src="..."
+></video>
+
+<!-- intro ends at 10. "intro - 0.5" = 10 - 0.5 = starts at second 9.5 (0.5s overlap for crossfade) -->
+<!-- Different track because clips on the same track cannot overlap -->
+<video
+  id="scene-b"
+  data-start="intro - 0.5"
+  data-duration="20"
+  data-track-index="1"
+  src="..."
+></video>
+```
+
+### Rules
+
+- **Same composition only** — references resolve within the clip's parent composition
+- **No circular references** — A cannot start after B if B starts after A
+- **Referenced clip must have a known duration** — the system needs a known end time to resolve the reference (either explicit `data-duration` or inferred from source media)
+- **Parsing** — if the value is a valid number, it is absolute seconds; otherwise it is parsed as `<id>`, `<id> + <number>`, or `<id> - <number>`
+
+## Video Clips
+
+Full-screen or positioned video clips. Videos sync their playback to the timeline position.
+
+```html
+<video
+  id="el-1"
+  data-start="0"
+  data-duration="15"
+  data-track-index="0"
+  src="./assets/video.mp4"
+></video>
+```
+
+- `data-media-start` — Playback begins at this time in the source video file (seconds). Default: `0`.
+- `data-duration` — (optional) How long the clip occupies on the timeline, in seconds. Playback runs from `data-media-start` for up to `data-duration` seconds. If the source media runs out before `data-duration` elapses, playback naturally stops (the clip remains mounted showing the last frame). If omitted, defaults to the remaining duration of the source file from `data-media-start`.
+
+## Image Clips
+
+Static images that appear for a duration.
+
+```html
+<img id="el-2" data-start="5" data-duration="4" data-track-index="1" src="./assets/video.mp4" />
+```
+
+## Audio Clips
+
+Background music or sound effects. Audio clips are invisible.
+
+```html
+<audio
+  id="el-4"
+  data-start="0"
+  data-duration="30"
+  data-track-index="2"
+  src="./assets/music.mp3"
+></audio>
+```
+
+- `data-media-start` — Playback begins at this time in the source audio file (seconds). Default: `0`.
+- `data-duration` — (optional) How long the clip occupies on the timeline, in seconds. Playback runs from `data-media-start` for up to `data-duration` seconds. If the source media runs out before `data-duration` elapses, playback naturally stops (the clip remains mounted but silent). If omitted, defaults to the remaining duration of the source file from `data-media-start`.
+
+## Two Layers: Primitives and Scripts
+
+Every composition — master or sub — has the same two layers:
+
+- **HTML** — primitive clips (`video`, `img`, `audio`, nested `div[data-composition-id]`). This is the declarative structure: what plays, when, and on which track.
+- **Script** — effects, transitions, dynamic DOM, canvas, SVG — creative animation and visuals via GSAP. Scripts do **not** control media playback or clip visibility; the framework handles those via data attributes.
+
+Both layers are available to every composition. The schema defines the primitives and the timeline contract; scripts handle visual creativity on top of that.
+
+> **Warning:** Never use scripts to play/pause/seek media elements or to show/hide clips based on timing. The framework does this automatically from `data-start`, `data-duration`, and `data-media-start`. Scripts that duplicate this behavior will conflict with the framework.
+
+### Script Isolation
+
+Each composition's script is scoped to that composition. When a composition is loaded from an external HTML file via `data-composition-src`, its inline `<script>` and `<style>` tags are automatically included and scoped to that composition.
+
+**Preferred approach** — Composition in separate HTML file:
+
+```html
+<!-- In index.html -->
+<div
+  id="el-5"
+  data-composition-id="intro-anim"
+  data-composition-src="compositions/intro-anim.html"
+  data-start="0"
+  data-track-index="2"
+></div>
+```
+
+**Alternative approach** — External JS file:
+
+```html
+<!-- In index.html -->
+<div
+  id="el-5"
+  data-composition-id="intro-anim"
+  data-start="0"
+  data-track-index="2"
+  data-width="1920"
+  data-height="1080"
+>
+  <script src="intro-anim.js"></script>
+</div>
+```
+
+The separate HTML file approach is preferred because it keeps all composition code (structure, style, script) in one self-contained, reusable file.
+
+The only required file is `index.html`. Every composition must have at least a script to create and register its GSAP timeline.
+
+### Top-Level Composition
+
+The top-level composition is the `index.html` entry point. It acts as the conductor — sequencing clips and placing sub-composition timelines into an overall master timeline. It can technically do anything in its script, but its primary purpose is high-level orchestration. Any composition can serve as a top-level composition or be nested into another — there is no structural difference.
+
+```html
+<div id="comp-1" data-composition-id="my-video" data-start="0" data-width="1920" data-height="1080">
+  <!-- Primitive clips -->
+  <video id="el-1" data-start="0" data-duration="10" data-track-index="0" src="..."></video>
+  <video id="el-2" data-start="el-1" data-duration="8" data-track-index="0" src="..."></video>
+  <img id="el-3" data-start="5" data-duration="4" data-track-index="1" src="..." />
+  <audio id="el-4" data-start="0" data-duration="30" data-track-index="2" src="..." />
+
+  <!-- Load sub-compositions from external files -->
+  <div
+    id="el-5"
+    data-composition-id="intro-anim"
+    data-composition-src="compositions/intro-anim.html"
+    data-start="0"
+    data-track-index="3"
+  ></div>
+
+  <div
+    id="el-6"
+    data-composition-id="captions"
+    data-composition-src="compositions/caption-overlay.html"
+    data-start="0"
+    data-track-index="4"
+  ></div>
+
+  <script>
+    // Just register the timeline - framework auto-nests sub-compositions
+    const tl = gsap.timeline({ paused: true });
+    window.__timelines["my-video"] = tl;
+  </script>
+</div>
+```
+
+### Sub-Compositions
+
+Sub-compositions are a spectrum. One might simply group a few primitive clips. Another might be a fully custom program with its own HTML, CSS, and JavaScript — creating, animating, and destroying DOM however it sees fit. There are no categories or constraints on what a sub-composition does internally. The only rule: it must be driven by a GSAP timeline and export it. If children use `position: absolute`, always set explicit `left`/`top`/`bottom`/`right` — the composition root is a positioned container, so omitting coordinates produces unpredictable placement.
+
+## Wrapping Dynamic Content in Compositions
+
+**Critical Rule: All visual content must live inside a composition with data attributes to appear in the timeline.**
+
+When you have dynamic or script-animated content (captions, emojis, overlays, text animations), wrap them in a composition element with `data-start`, `data-duration` (or let the timeline determine duration), and `data-track-index`. The children inside can be freely created and animated via JavaScript—they don't need individual data attributes.
+
+### Wrong: Dynamic content outside a composition
+
+```html
+<!-- BAD: captions-container is not a composition - won't appear in timeline -->
+<div id="ui-layer">
+  <div id="captions-container">
+    <!-- Dynamically created caption groups via JS -->
+  </div>
+  <div class="emoji" id="emoji-1">🤩</div>
+</div>
+
+<script>
+  // These animations work visually but elements don't appear in timeline
+  tl.to(".caption-group", { opacity: 1 }, 0.5);
+  tl.to("#emoji-1", { scale: 1.2 }, 2);
+</script>
+```
+
+### Correct: Dynamic content wrapped in compositions
+
+**Preferred approach** — Load from external HTML files:
+
+```html
+<!-- GOOD: Each logical group is a composition loaded from its own file -->
+<div
+  id="captions-comp"
+  data-composition-id="captions"
+  data-composition-src="compositions/captions.html"
+  data-start="0"
+  data-track-index="5"
+></div>
+
+<div
+  id="emojis-comp"
+  data-composition-id="emojis"
+  data-composition-src="compositions/emojis.html"
+  data-start="0"
+  data-track-index="6"
+></div>
+```
+
+**Alternative approach** — Inline composition (useful for one-off custom compositions):
+
+```html
+<div
+  id="captions-comp"
+  data-composition-id="captions"
+  data-start="0"
+  data-track-index="5"
+  data-width="1080"
+  data-height="1920"
+>
+  <!-- Children created/animated by script - no data attributes needed -->
+  <div id="captions-container"></div>
+  <script>
+    const captionTL = gsap.timeline({ paused: true });
+    // Dynamically create and animate caption groups...
+    window.__timelines["captions"] = captionTL;
+  </script>
+</div>
+
+<div
+  id="emojis-comp"
+  data-composition-id="emojis"
+  data-start="0"
+  data-track-index="6"
+  data-width="1080"
+  data-height="1920"
+>
+  <div class="emoji" id="emoji-1">🤩</div>
+  <div class="emoji" id="emoji-2">🏔️</div>
+  <script>
+    const emojiTL = gsap.timeline({ paused: true });
+    emojiTL.to("#emoji-1", { opacity: 1, scale: 1.2 }, 2);
+    emojiTL.to("#emoji-2", { opacity: 1, scale: 1.2 }, 4);
+    window.__timelines["emojis"] = emojiTL;
+  </script>
+</div>
+```
+
+### When to create separate compositions
+
+- **Captions**: One composition for all captions, script manages word groups
+- **Emojis/Stickers**: One composition for the emoji layer
+- **Hooks/Titles**: One composition per distinct title sequence
+- **Overlays**: Group related overlays into compositions by purpose
+
+The composition appears in the timeline with its start time and duration (determined by its GSAP timeline). Everything inside is managed by the script but inherits the composition's timeline position.
+
+### Caption discoverability contract
+
+To make caption previews deterministic and easy to isolate in downstream UIs (timeline, mini-player, editor overlays), caption compositions should expose a stable root selector.
+
+Use these attributes on the caption root node:
+
+- `data-timeline-role="captions"` (required)
+- `data-caption-root="true"` (recommended)
+
+Example:
+
+```html
+<div
+  id="captions-comp"
+  data-composition-id="captions"
+  data-start="0"
+  data-track-index="5"
+  data-width="1080"
+  data-height="1920"
+  data-timeline-role="captions"
+  data-caption-root="true"
+>
+  <div id="caption-container"></div>
+  <script>
+    const captionTL = gsap.timeline({ paused: true });
+    window.__timelines["captions"] = captionTL;
+  </script>
+</div>
+```
+
+## Timeline Contract
+
+The framework initializes `window.__timelines = {}` before any scripts run. Every composition **must** have a script that creates a GSAP timeline and registers it:
+
+```js
+const tl = gsap.timeline({ paused: true });
+// ... add tweens, nested timelines, etc.
+window.__timelines["<data-composition-id>"] = tl;
+```
+
+### Rules
+
+- **Every composition needs a script** — at minimum, to create and register its timeline. A composition without a script has no timeline and cannot participate in the hierarchy.
+- **All timelines start paused** — create timelines with `{ paused: true }`. The top-level timeline is controlled externally by the frontend player or renderer.
+- **Framework auto-nests sub-timelines** — you do **not** need to manually add sub-composition timelines to the master timeline. The framework automatically nests any timeline registered in `window.__timelines` into its parent based on the composition's `data-start` attribute.
+- **Duration comes from the timeline** — a composition's duration is `tl.duration()`. The timeline is the sole source of truth; there is no `data-duration` on compositions.
+- **Timelines must be finite** — every timeline must have a finite duration. Infinite or indefinite timelines are not supported.
+
+### What NOT to do
+
+```js
+// UNNECESSARY - the framework does this automatically
+if (window.__timelines["captions"]) {
+  masterTL.add(window.__timelines["captions"], 0);
+}
+```
+
+Just register your timeline in `window.__timelines` and the framework handles the rest.
+
+## Output Checklist
+
+- [ ] Every composition has `data-width` and `data-height` attributes
+- [ ] Each reusable composition is in its own HTML file (in `compositions/` directory)
+- [ ] Compositions loaded via `data-composition-src` attribute
+- [ ] Each composition file uses `<template>` tag to wrap its content
+- [ ] `window.__timelines` given all compositions' timelines
+- [ ] Complex/dynamic animations are handled by scripts in compositions
@@ -0,0 +1,61 @@
+# Core Notes
+
+Extended design rationale and context for decisions in `core.md`. This file is for humans reviewing the design — it is not consumed by the LLM agent.
+
+## Interactive Compositions
+
+### Why `data-start="interactive"`
+
+We considered three alternatives:
+
+1. **`data-interactive` boolean attribute** — Keeps `data-start` clean, but then `data-start` must either be omitted (breaking the convention that every clip has `data-start`) or present but meaningless. Two attributes to express what one value handles.
+
+2. **Event-driven `data-start="on:click:#element"`** — More expressive and extensible to other events (`on:hover`, `on:end:#video`), but complex to parse, harder for an LLM to author correctly, and the trigger/target relationship is declared on the target which reads backwards.
+
+3. **`data-start="interactive"` (chosen)** — Reuses the existing `data-start` attribute with a new keyword value. Reads naturally: "when does this start?" → "interactively." One attribute, no ambiguity, easy to parse (`=== "interactive"` check).
+
+The tradeoff is that `data-start` is now overloaded (was purely numeric/reference, now has a keyword). We accepted this because the parsing is trivial and the readability gain is significant.
+
+### Why `window.__navigate()` instead of `data-goto`
+
+Navigation is behavior, not structure. The framework's philosophy separates these: HTML declares structure and timing, scripts handle behavior.
+
+A `data-goto="composition-id"` attribute on trigger elements would be declarative and concise, but limits what authors can do. With a runtime API, scripts can:
+
+- Add conditional logic (`if (score > 50) navigate('win') else navigate('lose')`)
+- Animate a transition before navigating
+- Add delays or timeouts
+- Chain multiple actions on a single click
+- Use any DOM event, not just clicks
+
+The trigger element is just normal HTML with a normal `addEventListener`. The LLM only needs to know `window.__navigate(id)` — plain JS.
+
+### Navigation Behavior: Replace
+
+When `__navigate()` is called:
+
+1. The currently active composition's timeline pauses
+2. The current composition hides (visibility/display)
+3. The target interactive composition shows
+4. The target's timeline seeks to 0 and plays
+
+We chose Replace (parent hides entirely) over Overlay (target plays on top) or Pause-and-branch (parent pauses, resumes when target ends) because it's the simplest mental model and matches how most interactive video works (YouTube branching, Netflix Bandersnatch).
+
+### Ownership Model
+
+Interactive compositions are children of the composition that branches to them in the DOM tree. The parent composition is the "root" of its branching experience — it owns its branches.
+
+However, `window.__navigate()` resolves composition IDs globally across the full tree, not scoped to the current parent. This means:
+
+- A deeply nested interactive composition can navigate to any other interactive composition by ID
+- A shared "game over" or "credits" composition can be reached from anywhere
+- Circular navigation is possible (A → B → A) — the framework does not prevent loops
+
+### Future Extensions
+
+These are not implemented but the design accommodates them without breaking changes:
+
+- **Overlay mode**: `window.__navigate(id, { mode: 'overlay' })` — target plays on top, parent pauses or continues
+- **History / back**: `window.__navigate('$back')` to return to the previous composition, `window.__navigateHistory` to read the stack
+- **Auto-advance / timeout**: Scripts can implement this today with `setTimeout(() => window.__navigate('default'), 10000)`. A declarative shorthand could be added later.
+- **Pause-and-branch**: `window.__navigate(id, { mode: 'branch' })` — parent pauses, resumes when target ends
@@ -0,0 +1,189 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <!--
+    REQUIRED: viewport must match your composition dimensions.
+    Landscape: width=1920, height=1080
+    Portrait:  width=1080, height=1920
+  -->
+    <meta name="viewport" content="width=1920, height=1080" />
+
+    <!--
+    REQUIRED: data-composition-id identifies this composition.
+    Must match the key you register in window.__timelines below.
+    Optional: data-width/data-height help the producer determine resolution.
+  -->
+    <meta data-composition-id="my-video" data-width="1920" data-height="1080" />
+
+    <title>My Video</title>
+
+    <!-- REQUIRED: GSAP for timeline animations -->
+    <script src="https://cdn.jsdelivr.net/npm/gsap@3.12.5/dist/gsap.min.js"></script>
+
+    <style>
+      * {
+        margin: 0;
+        padding: 0;
+        box-sizing: border-box;
+      }
+      html,
+      body {
+        width: 1920px;
+        height: 1080px;
+        overflow: hidden;
+        background: #000;
+      }
+
+      #stage {
+        position: relative;
+        width: 1920px;
+        height: 1080px;
+        overflow: hidden;
+      }
+
+      /*
+      REQUIRED: All timed clips need visibility: hidden.
+      The framework toggles visibility based on data-start/data-duration.
+      Never toggle visibility yourself in scripts.
+    */
+      .clip {
+        position: absolute;
+        top: 0;
+        left: 0;
+        width: 100%;
+        height: 100%;
+        visibility: hidden;
+        object-fit: cover;
+      }
+    </style>
+  </head>
+  <body>
+    <div id="stage">
+      <!--
+      ════════════════════════════════════════════════════
+      VIDEO CLIPS
+      ════════════════════════════════════════════════════
+
+      REQUIRED attributes:
+        data-start       — when the clip appears (seconds)
+        data-duration    — how long the clip stays on screen (seconds)
+                           (optional for video/audio — defaults to source duration)
+        data-track-index — z-stacking order + timeline track grouping
+                           (higher = in front, clips on same track cannot overlap)
+
+      REQUIRED on <video>:
+        muted            — framework manages audio via separate <audio> element
+        playsinline      — prevents fullscreen on mobile
+
+      REQUIRED:
+        class="clip"     — sets visibility: hidden so framework can manage lifecycle
+
+      DO NOT:
+        - Call video.play(), video.pause(), or set video.currentTime in scripts
+        - Toggle visibility in scripts
+        - Use the video element for audio (use a separate <audio> element)
+    -->
+      <video
+        id="el-video"
+        class="clip"
+        data-start="0"
+        data-duration="10"
+        data-track-index="0"
+        src="my-video.mp4"
+        muted
+        playsinline
+      ></video>
+
+      <!--
+      ════════════════════════════════════════════════════
+      IMAGE CLIPS
+      ════════════════════════════════════════════════════
+
+      data-duration is REQUIRED for images (they have no intrinsic duration).
+    -->
+      <img
+        id="el-image"
+        class="clip"
+        data-start="10"
+        data-duration="5"
+        data-track-index="0"
+        src="my-image.png"
+      />
+
+      <!--
+      ════════════════════════════════════════════════════
+      AUDIO
+      ════════════════════════════════════════════════════
+
+      Audio is invisible — no class="clip" needed (no visual to hide).
+      Use a separate <audio> element even if the source is the same .mp4 file.
+      The framework syncs audio playback to the timeline position.
+
+      Optional: data-volume="0.5" (0-1, default 1)
+    -->
+      <audio
+        id="el-audio"
+        data-start="0"
+        data-duration="15"
+        data-track-index="1"
+        data-volume="0.8"
+        src="my-video.mp4"
+      ></audio>
+
+      <!--
+      ════════════════════════════════════════════════════
+      TEXT / HTML OVERLAYS
+      ════════════════════════════════════════════════════
+
+      Any div with data-start becomes a timed clip.
+      Content inside can be freely styled with CSS.
+      Animate visual properties (opacity, transform) with GSAP — fine.
+      Do NOT animate visibility — the framework owns that.
+    -->
+      <div
+        id="el-title"
+        class="clip"
+        data-start="1"
+        data-duration="4"
+        data-track-index="2"
+        style="display: flex; align-items: center; justify-content: center; z-index: 10"
+      >
+        <h1 style="font-family: sans-serif; font-size: 72px; color: white; opacity: 0">
+          Hello World
+        </h1>
+      </div>
+    </div>
+
+    <script>
+      /*
+      ════════════════════════════════════════════════════
+      GSAP TIMELINE
+      ════════════════════════════════════════════════════
+
+      REQUIRED: Create a paused GSAP timeline.
+      REQUIRED: Register it in window.__timelines with the SAME ID
+                as data-composition-id on the <meta> tag.
+
+      Without this registration, NOTHING works — no playback,
+      no seeking, no rendering. This is the #1 mistake.
+    */
+      var tl = gsap.timeline({ paused: true });
+
+      // Animate visual properties only — opacity, transform, color, etc.
+      // The framework handles clip mounting/unmounting and media playback.
+      tl.to("#el-video", { opacity: 1, duration: 0.5 }, 0);
+      tl.to("#el-video", { opacity: 0, duration: 0.5 }, 9.5);
+
+      tl.to("#el-image", { opacity: 1, duration: 0.5 }, 10);
+      tl.to("#el-image", { opacity: 0, duration: 0.5 }, 14.5);
+
+      tl.to("#el-title h1", { opacity: 1, y: 0, duration: 0.8, ease: "power2.out" }, 1);
+      tl.to("#el-title h1", { opacity: 0, duration: 0.5 }, 4);
+
+      // REQUIRED: Register the timeline. The key MUST match data-composition-id.
+      window.__timelines = window.__timelines || {};
+      window.__timelines["my-video"] = tl;
+    </script>
+  </body>
+</html>
@@ -0,0 +1,5 @@
+# HyperFrames Core Changelog
+
+## v0.1
+
+Initial schema specification. Defines the foundational HTML-as-video format: compositions, clip types (video, image, audio, nested compositions), data attributes for timing and tracks, relative timing with offsets, the two-layer primitives + scripts model, and the GSAP timeline contract.
@@ -0,0 +1,332 @@
+# HyperFrames Schema
+
+Reference for generating and editing HyperFrames HTML compositions. This is your source of truth for how to author compositions.
+
+## Overview
+
+HyperFrames uses HTML as the source of truth for describing a video:
+
+- **HTML clips** = video, image, audio, composition
+- **Data attributes** = timing, metadata, styling
+- **CSS** = positioning and appearance
+- **GSAP timeline** = animations and playback sync
+
+### Framework-Managed Behavior
+
+The framework reads data attributes and automatically manages:
+
+- **Primitive clip timeline entries** — the framework reads `data-start`, `data-duration`, and `data-track-index` from primitive clips and adds them to the composition's GSAP timeline. You do not manually add primitive clips to the timeline in scripts.
+- **Media playback** (play, pause, seek) for `<video>` and `<audio>`
+- **Clip lifecycle** — clips are **mounted** (made visible on screen) and **unmounted** (removed from screen) based on `data-start` and `data-duration`
+- **Timeline synchronization** (keeping media in sync with the GSAP master timeline)
+- **Media loading** — the framework waits for all media elements to load before resolving timing and starting playback
+
+Mounting and unmounting controls **presence**, not appearance. A clip that is mounted is on screen; a clip that is unmounted is not. Transitions (fade in, slide in, etc.) are separate — they are animated in scripts and happen _after_ a clip is mounted or _before_ it is unmounted.
+
+The framework does **not** handle transitions, effects, or visual animation — those are driven by GSAP in JavaScript.
+
+Do not manually call `video.play()`, `video.pause()`, set `audio.currentTime`, or mount/unmount clips in scripts. The framework owns media playback and clip lifecycle. Animating visual properties like `opacity` or `transform` for transitions is fine — that's what scripts are for.
+
+## Viewport
+
+The root composition must declare its intended render dimensions using `data-width` and `data-height` attributes. Common sizes:
+
+- **Landscape**: `data-width="1920" data-height="1080"`
+- **Portrait**: `data-width="1080" data-height="1920"`
+
+e.g.,
+
+```html
+<div id="main" data-composition-id="my-video" data-start="0" data-width="1920" data-height="1080">
+  <!-- clips -->
+</div>
+```
+
+Every composition's container is full-screen within the viewport by default. The framework applies full-screen sizing to composition containers automatically.
+
+To position or size individual clips (e.g., picture-in-picture, overlay placement), use standard CSS on the element.
+
+## Compositions
+
+A composition is the fundamental grouping unit in HyperFrames. Every clip — video, image, audio — must live inside a composition. The `index.html` file is itself a composition (the top-level one), and it can contain nested compositions within it. Any composition can be imported into another composition as a sub-composition — there is no special "root" type.
+
+A composition carries the same core attributes as any other clip (`id`, `data-start`, `data-track-index`), so it can be placed and timed on a timeline just like a video or image. A composition's length is determined by its GSAP timeline — there is no `data-duration` on compositions. This means compositions can be nested: a composition clip inside another composition behaves like a self-contained video within the parent timeline.
+
+```html
+<div id="comp-1" data-composition-id="my-video" data-start="0" data-width="1920" data-height="1080">
+  <!-- Clips live inside the composition -->
+  <video id="el-1" data-start="0" data-duration="10" data-track-index="0" src="..."></video>
+  <video id="el-2" data-start="el-1" data-duration="8" data-track-index="0" src="..."></video>
+  <img id="el-3" data-start="5" data-duration="4" data-track-index="1" src="..." />
+  <audio id="el-4" data-start="0" data-duration="30" data-track-index="2" src="..." />
+
+  <!-- Nested composition (e.g. a motion graphic or title sequence) -->
+  <div id="el-5" data-composition-id="intro-anim" data-start="0" data-track-index="3">
+    <!-- its own clips, timeline, and script -->
+    <script src="intro-anim.js"></script>
+  </div>
+</div>
+```
+
+> Note: `data-width` and `data-height` are only required on the **root** composition. Nested compositions inherit the viewport dimensions.
+
+## Clip Types
+
+A clip is any discrete block on the timeline. We represent clips as HTML elements and apply data-attributes to describe them.
+
+- `<video>` — Video clips, B-roll, A-roll
+- `<img>` — Static images, overlays
+- `<audio>` — Music, sound effects
+- `<div data-composition-id="...">` — Nested compositions (animations, grouped sequences)
+
+## HTML Attributes
+
+### All Clips
+
+- `id` — Unique identifier (e.g., "el-1")
+- `data-start` — Start time in seconds, or a clip `id` reference. See [Relative Timing](#relative-timing).
+- `data-duration` — Duration in seconds. Required for `<img>` clips. Optional for `<video>` and `<audio>` (defaults to the source media's full duration). Not used on compositions.
+- `data-track-index` — Timeline track number. Tracks serve two purposes: they determine visual layering (higher tracks render in front) and they group clips into rows on the timeline. Clips on the same track **cannot overlap in time**.
+
+### Media Clips (video, audio)
+
+- `data-media-start` — (optional) Playback begins at this time in the source file, in seconds. Defaults to `0`.
+
+### Composition Clips
+
+- `data-composition-id` — Unique composition ID
+
+> Compositions do **not** use `data-duration`. Their duration is determined by their GSAP timeline.
+
+## Relative Timing
+
+Instead of calculating absolute start times, a clip can reference another clip's `id` in its `data-start` attribute. This means "start when that clip ends." The referenced clip must be in the same composition and must have a known duration (either an explicit `data-duration` or an inferred duration from the source media).
+
+### Basic Sequential Clips
+
+```html
+<video id="intro" data-start="0" data-duration="10" data-track-index="0" src="..."></video>
+<video id="main" data-start="intro" data-duration="20" data-track-index="0" src="..."></video>
+<video id="outro" data-start="main" data-duration="5" data-track-index="0" src="..."></video>
+```
+
+`main` resolves to second 10, `outro` resolves to second 30. If `intro`'s duration changes to 15, `main` and `outro` shift automatically.
+
+### Offsets (gaps and overlaps)
+
+Add `+ N` or `- N` after the ID to offset from the end of the referenced clip:
+
+```html
+<!-- intro ends at 10. "intro + 2" = 10 + 2 = starts at second 12 (2s gap) -->
+<video
+  id="scene-a"
+  data-start="intro + 2"
+  data-duration="20"
+  data-track-index="0"
+  src="..."
+></video>
+
+<!-- intro ends at 10. "intro - 0.5" = 10 - 0.5 = starts at second 9.5 (0.5s overlap for crossfade) -->
+<!-- Different track because clips on the same track cannot overlap -->
+<video
+  id="scene-b"
+  data-start="intro - 0.5"
+  data-duration="20"
+  data-track-index="1"
+  src="..."
+></video>
+```
+
+### Rules
+
+- **Same composition only** — references resolve within the clip's parent composition
+- **No circular references** — A cannot start after B if B starts after A
+- **Referenced clip must have a known duration** — the system needs a known end time to resolve the reference (either explicit `data-duration` or inferred from source media)
+- **Parsing** — if the value is a valid number, it is absolute seconds; otherwise it is parsed as `<id>`, `<id> + <number>`, or `<id> - <number>`
+
+## Video Clips
+
+Full-screen or positioned video clips. Videos sync their playback to the timeline position.
+
+```html
+<video
+  id="el-1"
+  data-start="0"
+  data-duration="15"
+  data-track-index="0"
+  src="./assets/video.mp4"
+></video>
+```
+
+- `data-media-start` — Playback begins at this time in the source video file (seconds). Default: `0`.
+- `data-duration` — (optional) How long the clip occupies on the timeline, in seconds. Playback runs from `data-media-start` for up to `data-duration` seconds. If the source media runs out before `data-duration` elapses, playback naturally stops (the clip remains mounted showing the last frame). If omitted, defaults to the remaining duration of the source file from `data-media-start`.
+
+## Image Clips
+
+Static images that appear for a duration.
+
+```html
+<img id="el-2" data-start="5" data-duration="4" data-track-index="1" src="./assets/video.mp4" />
+```
+
+## Audio Clips
+
+Background music or sound effects. Audio clips are invisible.
+
+```html
+<audio
+  id="el-4"
+  data-start="0"
+  data-duration="30"
+  data-track-index="2"
+  src="./assets/music.mp3"
+></audio>
+```
+
+- `data-media-start` — Playback begins at this time in the source audio file (seconds). Default: `0`.
+- `data-duration` — (optional) How long the clip occupies on the timeline, in seconds. Playback runs from `data-media-start` for up to `data-duration` seconds. If the source media runs out before `data-duration` elapses, playback naturally stops (the clip remains mounted but silent). If omitted, defaults to the remaining duration of the source file from `data-media-start`.
+
+## Two Layers: Primitives and Scripts
+
+Every composition — master or sub — has the same two layers:
+
+- **HTML** — primitive clips (`video`, `img`, `audio`, nested `div[data-composition-id]`). This is the declarative structure: what plays, when, and on which track.
+- **Script** — effects, transitions, dynamic DOM, canvas, SVG — creative animation and visuals via GSAP. Scripts do **not** control media playback or clip visibility; the framework handles those via data attributes.
+
+Both layers are available to every composition. The schema defines the primitives and the timeline contract; scripts handle visual creativity on top of that.
+
+> **Warning:** Never use scripts to play/pause/seek media elements or to show/hide clips based on timing. The framework does this automatically from `data-start`, `data-duration`, and `data-media-start`. Scripts that duplicate this behavior will conflict with the framework.
+
+### Script Isolation
+
+Each composition's script is scoped to that composition. For sub-compositions, external JS files are the natural way to keep them self-contained and reusable:
+
+```html
+<div id="el-5" data-composition-id="intro-anim" data-start="0" data-track-index="2">
+  <script src="intro-anim.js"></script>
+</div>
+```
+
+Inline scripts are fine when the script belongs to that composition. The JS file itself is the documentation for what a composition does — the schema doesn't need to describe its internals.
+
+The only required file is `index.html`. Every composition must have at least a script to create and register its GSAP timeline.
+
+### Top-Level Composition
+
+The top-level composition is the `index.html` entry point. It acts as the conductor — sequencing clips and placing sub-composition timelines into an overall master timeline. It can technically do anything in its script, but its primary purpose is high-level orchestration. Any composition can serve as a top-level composition or be nested into another — there is no structural difference.
+
+```html
+<div id="comp-1" data-composition-id="my-video" data-start="0" data-width="1920" data-height="1080">
+  <video id="el-1" data-start="0" data-duration="10" data-track-index="0" src="..."></video>
+  <video id="el-2" data-start="el-1" data-duration="8" data-track-index="0" src="..."></video>
+  <img id="el-3" data-start="5" data-duration="4" data-track-index="1" src="..." />
+  <audio id="el-4" data-start="0" data-duration="30" data-track-index="2" src="..." />
+
+  <div id="el-5" data-composition-id="intro-anim" data-start="0" data-track-index="3">
+    <script src="intro-anim.js"></script>
+  </div>
+
+  <script>
+    // Just register the timeline - framework auto-nests sub-compositions
+    const tl = gsap.timeline({ paused: true });
+    window.__timelines["my-video"] = tl;
+  </script>
+</div>
+```
+
+### Sub-Compositions
+
+Sub-compositions are a spectrum. One might simply group a few primitive clips. Another might be a fully custom program with its own HTML, CSS, and JavaScript — creating, animating, and destroying DOM however it sees fit. There are no categories or constraints on what a sub-composition does internally. All compositions default to full-screen within the viewport. The only rule: it must be driven by a GSAP timeline and export it.
+
+## Wrapping Dynamic Content in Compositions
+
+**Critical Rule: All visual content must live inside a composition with data attributes to appear in the timeline.**
+
+When you have dynamic or script-animated content (captions, emojis, overlays, text animations), wrap them in a composition element with `data-start`, `data-duration` (or let the timeline determine duration), and `data-track-index`. The children inside can be freely created and animated via JavaScript—they don't need individual data attributes.
+
+### Wrong: Dynamic content outside a composition
+
+```html
+<!-- BAD: captions-container is not a composition - won't appear in timeline -->
+<div id="ui-layer">
+  <div id="captions-container">
+    <!-- Dynamically created caption groups via JS -->
+  </div>
+  <div class="emoji" id="emoji-1">🤩</div>
+</div>
+
+<script>
+  // These animations work visually but elements don't appear in timeline
+  tl.to(".caption-group", { opacity: 1 }, 0.5);
+  tl.to("#emoji-1", { scale: 1.2 }, 2);
+</script>
+```
+
+### Correct: Dynamic content wrapped in compositions
+
+```html
+<!-- GOOD: Each logical group is a composition that appears in timeline -->
+<div id="captions-comp" data-composition-id="captions" data-start="0" data-track-index="5">
+  <!-- Children created/animated by script - no data attributes needed -->
+  <div id="captions-container"></div>
+  <script>
+    const captionTL = gsap.timeline({ paused: true });
+    // Dynamically create and animate caption groups...
+    window.__timelines["captions"] = captionTL;
+  </script>
+</div>
+
+<div id="emojis-comp" data-composition-id="emojis" data-start="0" data-track-index="6">
+  <div class="emoji" id="emoji-1">🤩</div>
+  <div class="emoji" id="emoji-2">🏔️</div>
+  <script>
+    const emojiTL = gsap.timeline({ paused: true });
+    emojiTL.to("#emoji-1", { opacity: 1, scale: 1.2 }, 2);
+    emojiTL.to("#emoji-2", { opacity: 1, scale: 1.2 }, 4);
+    window.__timelines["emojis"] = emojiTL;
+  </script>
+</div>
+```
+
+### When to create separate compositions
+
+- **Captions**: One composition for all captions, script manages word groups
+- **Emojis/Stickers**: One composition for the emoji layer
+- **Hooks/Titles**: One composition per distinct title sequence
+- **Overlays**: Group related overlays into compositions by purpose
+
+The composition appears in the timeline with its start time and duration (determined by its GSAP timeline). Everything inside is managed by the script but inherits the composition's timeline position.
+
+## Timeline Contract
+
+The framework initializes `window.__timelines = {}` before any scripts run. Every composition **must** have a script that creates a GSAP timeline and registers it:
+
+```js
+const tl = gsap.timeline({ paused: true });
+// ... add tweens, nested timelines, etc.
+window.__timelines["<data-composition-id>"] = tl;
+```
+
+### Rules
+
+- **Every composition needs a script** — at minimum, to create and register its timeline. A composition without a script has no timeline and cannot participate in the hierarchy.
+- **All timelines start paused** — create timelines with `{ paused: true }`. The top-level timeline is controlled externally by the frontend player or renderer.
+- **Framework auto-nests sub-timelines** — you do **not** need to manually add sub-composition timelines to the master timeline. The framework automatically nests any timeline registered in `window.__timelines` into its parent based on the composition's `data-start` attribute.
+- **Duration comes from the timeline** — a composition's duration is `tl.duration()`. The timeline is the sole source of truth; there is no `data-duration` on compositions.
+
+### What NOT to do
+
+```js
+// UNNECESSARY - the framework does this automatically
+if (window.__timelines["captions"]) {
+  masterTL.add(window.__timelines["captions"], 0);
+}
+```
+
+Just register your timeline in `window.__timelines` and the framework handles the rest.
+
+## Output Checklist
+
+- [ ] Root composition has `data-width` and `data-height` attributes
+- [ ] `window.__timelines` given all compositions' timelines
+- [ ] Complex/dynamic animations are handled by scripts in compositions