chore: import upstream snapshot with attribution
regression / regression-shards (style-16-prod style-9-prod style-17-prod iframe-render-compat variables-prod mp4-h265-sdr, shard-4) (push) Has been cancelled
regression / regression-shards (style-4-prod style-11-prod style-2-prod animejs-adapter typegpu-adapter parallel-capture-regression, shard-5) (push) Has been cancelled
regression / regression-shards (style-7-prod style-8-prod style-10-prod css-spinner-render-compat webm-transparency mp4-h264-sdr webm-vp9, shard-3) (push) Has been cancelled
regression / regression-shards (sub-composition-video style-18-prod raf-ball-render-compat font-variant-numeric sub-comp-t0 sub-comp-id-selector, shard-7) (push) Has been cancelled
Windows render verification / Detect changes (push) Has been cancelled
Windows render verification / Preflight (lint + format) (push) Has been cancelled
Windows render verification / Render on windows-latest (push) Has been cancelled
Windows render verification / Tests on windows-latest (push) Has been cancelled
CI / Detect changes (push) Has been cancelled
CI / Build (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Fallow audit (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Typecheck (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Producer: integration tests (push) Has been cancelled
CI / Producer: unit tests (push) Has been cancelled
CI / File size check (push) Has been cancelled
CI / Test: skills (push) Has been cancelled
CI / Skills: manifest in sync (push) Has been cancelled
CI / CLI: npx shim (macos-latest) (push) Has been cancelled
CI / CLI: npx shim (ubuntu-latest) (push) Has been cancelled
CI / CLI: npx shim (windows-latest) (push) Has been cancelled
CI / SDK: unit + contract + smoke (push) Has been cancelled
CI / Test: runtime contract (push) Has been cancelled
CI / Studio: load smoke (push) Has been cancelled
CI / Smoke: global install (push) Has been cancelled
CI / CLI smoke (required) (push) Has been cancelled
CI / Semantic PR title (push) Has been cancelled
Player perf / Detect changes (push) Has been cancelled
Player perf / Preflight (lint + format) (push) Has been cancelled
Player perf / player-perf (push) Has been cancelled
Player perf / Perf: drift (push) Has been cancelled
Player perf / Perf: fps (push) Has been cancelled
Player perf / Perf: parity (push) Has been cancelled
Player perf / Perf: scrub (push) Has been cancelled
Player perf / Perf: load (push) Has been cancelled
preview-regression / Detect changes (push) Has been cancelled
preview-regression / Preflight (lint + format) (push) Has been cancelled
preview-regression / Preview parity (push) Has been cancelled
preview-regression / preview-regression (push) Has been cancelled
regression / regression (push) Has been cancelled
regression / Detect changes (push) Has been cancelled
regression / Preflight (lint + format) (push) Has been cancelled
regression / regression-shards (hdr-regression style-5-prod style-3-prod mov-prores, shard-1) (push) Has been cancelled
regression / regression-shards (overlay-montage-prod style-12-prod chat missing-host-comp-id png-sequence portrait-edge-bleed, shard-6) (push) Has been cancelled
regression / regression-shards (style-13-prod style-6-prod vignelli-stacking gsap-letters-render-compat audio-mux-parity, shard-8) (push) Has been cancelled
regression / regression-shards (style-15-prod hdr-hlg-regression style-1-prod many-cuts vfr-screen-recording render-symlinked-assets, shard-2) (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Docs / Validate docs (push) Has been cancelled
Sync skills to ClawHub / Publish changed skills (push) Has been cancelled
regression / regression-shards (style-16-prod style-9-prod style-17-prod iframe-render-compat variables-prod mp4-h265-sdr, shard-4) (push) Has been cancelled
regression / regression-shards (style-4-prod style-11-prod style-2-prod animejs-adapter typegpu-adapter parallel-capture-regression, shard-5) (push) Has been cancelled
regression / regression-shards (style-7-prod style-8-prod style-10-prod css-spinner-render-compat webm-transparency mp4-h264-sdr webm-vp9, shard-3) (push) Has been cancelled
regression / regression-shards (sub-composition-video style-18-prod raf-ball-render-compat font-variant-numeric sub-comp-t0 sub-comp-id-selector, shard-7) (push) Has been cancelled
Windows render verification / Detect changes (push) Has been cancelled
Windows render verification / Preflight (lint + format) (push) Has been cancelled
Windows render verification / Render on windows-latest (push) Has been cancelled
Windows render verification / Tests on windows-latest (push) Has been cancelled
CI / Detect changes (push) Has been cancelled
CI / Build (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Fallow audit (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Typecheck (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Producer: integration tests (push) Has been cancelled
CI / Producer: unit tests (push) Has been cancelled
CI / File size check (push) Has been cancelled
CI / Test: skills (push) Has been cancelled
CI / Skills: manifest in sync (push) Has been cancelled
CI / CLI: npx shim (macos-latest) (push) Has been cancelled
CI / CLI: npx shim (ubuntu-latest) (push) Has been cancelled
CI / CLI: npx shim (windows-latest) (push) Has been cancelled
CI / SDK: unit + contract + smoke (push) Has been cancelled
CI / Test: runtime contract (push) Has been cancelled
CI / Studio: load smoke (push) Has been cancelled
CI / Smoke: global install (push) Has been cancelled
CI / CLI smoke (required) (push) Has been cancelled
CI / Semantic PR title (push) Has been cancelled
Player perf / Detect changes (push) Has been cancelled
Player perf / Preflight (lint + format) (push) Has been cancelled
Player perf / player-perf (push) Has been cancelled
Player perf / Perf: drift (push) Has been cancelled
Player perf / Perf: fps (push) Has been cancelled
Player perf / Perf: parity (push) Has been cancelled
Player perf / Perf: scrub (push) Has been cancelled
Player perf / Perf: load (push) Has been cancelled
preview-regression / Detect changes (push) Has been cancelled
preview-regression / Preflight (lint + format) (push) Has been cancelled
preview-regression / Preview parity (push) Has been cancelled
preview-regression / preview-regression (push) Has been cancelled
regression / regression (push) Has been cancelled
regression / Detect changes (push) Has been cancelled
regression / Preflight (lint + format) (push) Has been cancelled
regression / regression-shards (hdr-regression style-5-prod style-3-prod mov-prores, shard-1) (push) Has been cancelled
regression / regression-shards (overlay-montage-prod style-12-prod chat missing-host-comp-id png-sequence portrait-edge-bleed, shard-6) (push) Has been cancelled
regression / regression-shards (style-13-prod style-6-prod vignelli-stacking gsap-letters-render-compat audio-mux-parity, shard-8) (push) Has been cancelled
regression / regression-shards (style-15-prod hdr-hlg-regression style-1-prod many-cuts vfr-screen-recording render-symlinked-assets, shard-2) (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Docs / Validate docs (push) Has been cancelled
Sync skills to ClawHub / Publish changed skills (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,142 @@
|
||||
# Remotion → HyperFrames API Map
|
||||
|
||||
Authoritative translation table. Load this reference when starting a translation
|
||||
to know the high-level mapping; load the per-topic references for fragile
|
||||
details (timing, transitions, etc.).
|
||||
|
||||
## Reading this table
|
||||
|
||||
- **`drop`** = remove from output entirely. The HF runtime handles it.
|
||||
- **`see references/X.md`** = the mapping is non-trivial; read the linked file.
|
||||
- **`refuse + interop`** = the skill bows out and recommends the runtime adapter
|
||||
pattern from [PR #214](https://github.com/heygen-com/hyperframes/pull/214).
|
||||
|
||||
## Composition root
|
||||
|
||||
| Remotion | HyperFrames |
|
||||
| ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
|
||||
| `<Composition id durationInFrames fps width height>` | root `<div id="stage" data-composition-id data-start="0" data-duration="<dur/fps>" data-fps data-width data-height>` |
|
||||
| `defaultProps={...}` | `data-*` attributes on `#stage` (one per scalar prop). Nested objects/arrays — see [parameters.md](parameters.md) |
|
||||
| `schema={z.object(...)}` | not represented in HTML; the schema lives in the agent's translation step only |
|
||||
| `calculateMetadata` (sync) | resolve at translation time, write concrete values into `data-*` |
|
||||
| `calculateMetadata` (async) | **refuse + interop** — see [escape-hatch.md](escape-hatch.md) |
|
||||
| `registerRoot(RemotionRoot)` | drop |
|
||||
| `<AbsoluteFill style>` | `<div style="position:absolute;inset:0;{style}">` |
|
||||
|
||||
## Sequencing
|
||||
|
||||
See [sequencing.md](sequencing.md) for nesting and stagger details.
|
||||
|
||||
| Remotion | HyperFrames |
|
||||
| ------------------------------------------ | --------------------------------------------------------------------------------------------------------- |
|
||||
| `<Sequence from={F} durationInFrames={D}>` | `<div data-start="<F/fps>" data-duration="<D/fps>" data-track-index="N">` |
|
||||
| `<Series>` + `<Series.Sequence>` | siblings with sequential `data-start` values |
|
||||
| `<Loop durationInFrames={D}>` | not a primitive — emit a custom GSAP `repeat: -1` loop with manual offset math |
|
||||
| `<Freeze frame={F}>` | drop the wrapper; HF doesn't have running animation outside the seek-driven timeline so freeze is a no-op |
|
||||
|
||||
## Timing
|
||||
|
||||
See [timing.md](timing.md) — this is the highest-leverage section.
|
||||
|
||||
| Remotion | HyperFrames |
|
||||
| ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
|
||||
| `useCurrentFrame()` | drop — HF seeks the timeline. The math derived from `frame` becomes an animatable property of a paused GSAP tween. |
|
||||
| `useVideoConfig()` for `fps` / `durationInFrames` | drop — read from `data-fps` / `data-duration` on `#stage` |
|
||||
| `interpolate(frame, [a,b], [x,y])` (linear) | `gsap.fromTo(t, {p:x}, {p:y, duration:(b-a)/fps, ease:"none"})` at offset `a/fps` |
|
||||
| `interpolate(frame, [a,b,c,d], [x,y,y,z])` (multi-segment) | three `gsap.to` calls at offsets `a/fps`, `b/fps`, `c/fps` |
|
||||
| `interpolate(..., {easing: Easing.bezier})` | GSAP `CustomEase.create("c", "M0,0 C${a},${b} ${c},${d} 1,1")` |
|
||||
| `spring({frame, fps, config: {damping, stiffness, mass}})` | GSAP `back.out(N)` — see [timing.md](timing.md) for damping → overshoot table |
|
||||
| `interpolateColors(frame, range, colors)` | `gsap.to({...}, { backgroundColor, color, duration, ease })` — GSAP handles color tweens natively |
|
||||
| `Easing.in / .out / .inOut(power)` | GSAP `power<N>.in` / `power<N>.out` / `power<N>.inOut` |
|
||||
|
||||
## Media
|
||||
|
||||
See [media.md](media.md) for trim, volume ramps, and decoder notes.
|
||||
|
||||
| Remotion | HyperFrames |
|
||||
| -------------------------------------- | --------------------------------------------------------------------------- |
|
||||
| `<Audio src volume>` | `<audio data-start data-duration data-track-index data-volume src>` |
|
||||
| `<Audio playbackRate startFrom endAt>` | `data-playback-rate`, `data-trim-start`, `data-trim-end` |
|
||||
| `<Video src>` | `<video muted playsinline data-start data-duration data-track-index src>` |
|
||||
| `<OffthreadVideo>` | `<video>` — HF doesn't need the off-thread variant (uses headless Chrome) |
|
||||
| `<Img src>` | `<img>` |
|
||||
| `<IFrame src>` | `<iframe>` — HF auto-falls back to screenshot mode for nested iframes |
|
||||
| `staticFile("x.png")` | `"assets/x.png"` — copy the file into `hf-src/assets/` next to `index.html` |
|
||||
| `delayRender()` / `continueRender()` | drop — HF waits on asset readiness via the Frame Adapter pattern |
|
||||
|
||||
## Transitions
|
||||
|
||||
See [transitions.md](transitions.md).
|
||||
|
||||
| Remotion | HyperFrames |
|
||||
| ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------- |
|
||||
| `<TransitionSeries>` + `<TransitionSeries.Transition presentation={fade()} />` | manual `gsap.to(scene, {opacity: 0/1, duration})` crossfade at the boundary |
|
||||
| `slide()`, `wipe()`, `clockWipe()`, `fade()` | HF [shader-transitions](https://hyperframes.heygen.com/catalog/blocks) package presets — pick the closest |
|
||||
| `linearTiming({durationInFrames})` | duration in seconds (`/fps`) |
|
||||
| `springTiming({config})` | duration in seconds, ease `back.out` — see [timing.md](timing.md) |
|
||||
|
||||
## Lottie
|
||||
|
||||
See [lottie.md](lottie.md).
|
||||
|
||||
| Remotion | HyperFrames |
|
||||
| ------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
|
||||
| `<Lottie animationData={data}>` | `<div id="lottie-N">` + `<script>const anim = lottie.loadAnimation({...}); window.__hfLottie.push(anim)</script>` |
|
||||
| `loop` / `playbackRate` props | translate only after checking player seek behavior; HF adapter seeks absolute time via `goToAndStop` |
|
||||
| `@remotion/lottie` runtime | `lottie-web` from CDN — drop the React wrapper |
|
||||
|
||||
## Fonts
|
||||
|
||||
See [fonts.md](fonts.md).
|
||||
|
||||
| Remotion | HyperFrames |
|
||||
| --------------------------------------------------- | ------------------------------------------------------------------------------------------- |
|
||||
| `loadFont()` from `@remotion/google-fonts/<Family>` | `@font-face` rule referencing the Google Fonts CSS, OR `<link>` to Google Fonts in `<head>` |
|
||||
| Local font via `@font-face` | same — paste the rule into `<style>` |
|
||||
| System font fallback | document the font-fallback divergence cost (see [eval.md](eval.md)) |
|
||||
|
||||
## Parameters
|
||||
|
||||
See [parameters.md](parameters.md).
|
||||
|
||||
| Remotion | HyperFrames |
|
||||
| ----------------------------- | --------------------------------------------------------------------------------------------- |
|
||||
| `z.object({foo: z.string()})` | `data-foo` on `#stage` (the schema is implicit in HTML structure) |
|
||||
| nested array prop (`stats[]`) | repeated HTML markup with per-instance `data-*` attrs |
|
||||
| Zod default values | bake defaults into the HTML directly |
|
||||
| Zod runtime validation | not represented; if validation matters, validate in the translation step before emitting HTML |
|
||||
|
||||
## React patterns
|
||||
|
||||
| Remotion | HyperFrames |
|
||||
| -------------------------------------------------- | ---------------------------------------------------------------- |
|
||||
| Custom React subcomponent (pure, prop-driven) | inline as repeated HTML using the prop interface as the template |
|
||||
| `useState` driving animation | **refuse + interop** |
|
||||
| `useReducer` driving animation | **refuse + interop** |
|
||||
| `useEffect(fn, [deps])` (non-empty deps) | **refuse + interop** |
|
||||
| `useEffect(fn, [])` (mount-once side effect) | drop the effect; use `queueMicrotask` if startup work is needed |
|
||||
| `useCallback`, `useMemo` | drop the wrappers — decorative |
|
||||
| Custom hook (pure derivation of `useCurrentFrame`) | inline the body |
|
||||
| Custom hook with state/effects | refuse + interop |
|
||||
|
||||
## Distributed rendering
|
||||
|
||||
`@remotion/lambda` and `@remotion/cloudrun` are deployment configuration —
|
||||
orthogonal to the rendered composition itself. The skill emits these as
|
||||
**warnings** (not blockers) and drops them in step 3 (Generate) with a note
|
||||
in `TRANSLATION_NOTES.md`. HF is single-machine today; document the gap.
|
||||
|
||||
| Remotion | HyperFrames |
|
||||
| -------------------------- | ------------------------------------------------------ |
|
||||
| `@remotion/lambda` import | drop the import (warning `r2hf/lambda-import`) |
|
||||
| `renderMediaOnLambda(...)` | drop the call; note in `TRANSLATION_NOTES.md` |
|
||||
| `@remotion/cloudrun` | drop the import + call; note in `TRANSLATION_NOTES.md` |
|
||||
|
||||
## When to bow out entirely
|
||||
|
||||
If any blocker pattern is present, recommend the runtime interop pattern from
|
||||
[PR #214](https://github.com/heygen-com/hyperframes/pull/214) instead of
|
||||
attempting translation. See [escape-hatch.md](escape-hatch.md).
|
||||
|
||||
The blockers are documented in [`scripts/lint_source.py`](../scripts/lint_source.py)
|
||||
and tested by [tier-4-escape-hatch](../assets/test-corpus/tier-4-escape-hatch/).
|
||||
@@ -0,0 +1,115 @@
|
||||
# When to bow out: the runtime interop pattern
|
||||
|
||||
Some Remotion compositions can't be translated cleanly. The skill should
|
||||
recognize them upfront and recommend the **runtime interop pattern** from
|
||||
[PR #214](https://github.com/heygen-com/hyperframes/pull/214) instead of
|
||||
producing broken HTML.
|
||||
|
||||
## When to recommend interop
|
||||
|
||||
Run `scripts/lint_source.py` first. If it returns any blocker, recommend
|
||||
interop. The blockers are:
|
||||
|
||||
| Rule | What it catches |
|
||||
| --------------------------- | -------------------------------------------------------------- |
|
||||
| `r2hf/use-state` | useState driving animation |
|
||||
| `r2hf/use-reducer` | useReducer driving animation |
|
||||
| `r2hf/use-effect-deps` | useEffect/useLayoutEffect with non-empty deps (side effects) |
|
||||
| `r2hf/async-metadata` | calculateMetadata returns a Promise |
|
||||
| `r2hf/third-party-react-ui` | Imports from MUI, Chakra, Mantine, antd, shadcn, Radix, NextUI |
|
||||
|
||||
Each of these breaks the seek-driven, deterministic-frame model that HF
|
||||
relies on. Translating them produces silently-wrong output.
|
||||
|
||||
## What the interop pattern actually does
|
||||
|
||||
Per [PR #214](https://github.com/heygen-com/hyperframes/pull/214), the
|
||||
runtime adapter:
|
||||
|
||||
1. Bundles the user's Remotion code with React + `@remotion/player` via esbuild.
|
||||
2. Mounts a Remotion `<Player>` inside an HF composition's HTML.
|
||||
3. Pauses the player on mount.
|
||||
4. Registers the player on `window.__hfRemotion` with `seekTo(frame)`,
|
||||
`pause()`, `durationInFrames`, `fps`.
|
||||
5. HF's render loop seeks the player frame-by-frame via `seekTo(frame)`.
|
||||
|
||||
Result: Remotion's React tree renders at HF's deterministic frame ticks.
|
||||
Custom hooks, useState, useEffect, MUI components — all work because
|
||||
Remotion's React reconciler is doing the rendering.
|
||||
|
||||
## The recommendation message
|
||||
|
||||
When the skill detects a blocker, output something like:
|
||||
|
||||
> The Remotion source uses `useState` (and others), which can't be
|
||||
> translated to HF's seek-driven HTML model. The recommended path is the
|
||||
> **runtime interop pattern**: bundle your Remotion code with `@remotion/player`
|
||||
> and let HF drive it frame-by-frame.
|
||||
>
|
||||
> See https://github.com/heygen-com/hyperframes/pull/214 for the full
|
||||
> implementation. Quick summary:
|
||||
>
|
||||
> 1. Bundle `entry.tsx` with esbuild: `npx esbuild entry.tsx --bundle --outfile=dist/bundle.js --format=iife --jsx=automatic`
|
||||
> 2. Mount the Player and register on `window.__hfRemotion`:
|
||||
>
|
||||
> ```tsx
|
||||
> const playerRef = useRef<PlayerRef>(null);
|
||||
> useEffect(() => {
|
||||
> playerRef.current?.pause();
|
||||
> window.__hfRemotion = window.__hfRemotion || [];
|
||||
> window.__hfRemotion.push({
|
||||
> seekTo: (f) => playerRef.current?.seekTo(f),
|
||||
> pause: () => playerRef.current?.pause(),
|
||||
> durationInFrames,
|
||||
> fps,
|
||||
> });
|
||||
> }, []);
|
||||
> ```
|
||||
>
|
||||
> 3. Reference the bundle from your HF `index.html` and render normally:
|
||||
> `<script src="dist/bundle.js"></script>`
|
||||
|
||||
## The lint output already includes recommendations
|
||||
|
||||
`lint_source.py` emits a `recommendation` field per finding. Surface those
|
||||
verbatim — they're tuned per blocker rule:
|
||||
|
||||
```json
|
||||
{
|
||||
"rule": "r2hf/use-state",
|
||||
"message": "useState detected — Remotion compositions that drive animation via React state are not deterministic frame-capture targets in HyperFrames",
|
||||
"recommendation": "Use the runtime interop pattern from PR #214 instead of attempting a translation"
|
||||
}
|
||||
```
|
||||
|
||||
## When NOT to bow out: warnings only
|
||||
|
||||
Some patterns produce warnings, not blockers — translate after dropping
|
||||
the wrappers:
|
||||
|
||||
| Rule | Action |
|
||||
| ------------------------- | ------------------------------------------------------------------- |
|
||||
| `r2hf/lambda-import` | drop the `@remotion/lambda` config; HF runs single-machine, log gap |
|
||||
| `r2hf/delay-render` | drop the call; HF handles asset readiness |
|
||||
| `r2hf/use-callback` | drop the wrapper, inline the function |
|
||||
| `r2hf/use-memo` | drop the wrapper, compute inline |
|
||||
| `r2hf/custom-hook` (pure) | inline the hook body if it's a derivation of `useCurrentFrame` |
|
||||
| `r2hf/static-file` | replace `staticFile("x")` with `"assets/x"` |
|
||||
| `r2hf/interpolate-colors` | translate to GSAP color tween (see [timing.md](timing.md)) |
|
||||
|
||||
These are documented in T4 cases 05–07.
|
||||
|
||||
`r2hf/lambda-import` is a warning — not a blocker — because Lambda
|
||||
configuration is orthogonal to the rendered composition. Translating an
|
||||
otherwise-clean Remotion comp shouldn't fail just because the author also
|
||||
configured AWS Lambda for distributed rendering. The skill drops the
|
||||
`@remotion/lambda` imports and `renderMediaOnLambda(...)` calls in step 3
|
||||
(Generate) and writes a `TRANSLATION_NOTES.md` entry so the user knows to
|
||||
set up HF rendering separately.
|
||||
|
||||
## When the source has BOTH blockers AND warnings
|
||||
|
||||
Bow out. The presence of a single blocker means the skill shouldn't
|
||||
attempt translation — even if the rest of the composition is clean.
|
||||
The user should use interop for the whole thing OR refactor the
|
||||
blocker patterns out of their Remotion source first.
|
||||
@@ -0,0 +1,140 @@
|
||||
# Eval: how to validate a translation end-to-end
|
||||
|
||||
Every translation should be measured. The skill ships three scripts and
|
||||
a tiered test corpus that, together, gate translation quality.
|
||||
|
||||
## The three scripts
|
||||
|
||||
| Script | Input | Output |
|
||||
| ------------------------ | --------------------------- | ------------------------------------------------------------------- |
|
||||
| `scripts/lint_source.py` | Remotion source dir or file | JSON findings + exit code (0 clean, 1 has blockers) |
|
||||
| `scripts/render_diff.sh` | two MP4 paths | per-frame SSIM + JSON summary (`mean`, `min`, `p05`, `p95`, `pass`) |
|
||||
| `scripts/frame_strip.sh` | two MP4 paths | side-by-side comparison strip PNG for visual debugging |
|
||||
|
||||
Run them in this order: **lint → render → diff → (if fail) strip**.
|
||||
|
||||
## Per-fixture flow
|
||||
|
||||
```bash
|
||||
# 1. Lint the source — blockers mean stop
|
||||
python3 ../../scripts/lint_source.py ./remotion-src/src/
|
||||
|
||||
# 2. Generate any binary assets (T2+T3 only)
|
||||
[ -f setup.sh ] && ./setup.sh
|
||||
|
||||
# 3. Render Remotion baseline
|
||||
cd remotion-src && npm install && npm run render
|
||||
# -> remotion-src/out/baseline.mp4
|
||||
|
||||
# 4. Render HF translation
|
||||
cd .. && node ../../../packages/cli/dist/cli.js render hf-src/ --output hf.mp4
|
||||
# -> hf.mp4
|
||||
|
||||
# 5. SSIM diff
|
||||
../../scripts/render_diff.sh ./remotion-src/out/baseline.mp4 ./hf.mp4 ./diff
|
||||
# -> diff/summary.json
|
||||
|
||||
# 6. If diff fails, generate frame strip for visual inspection
|
||||
../../scripts/frame_strip.sh ./remotion-src/out/baseline.mp4 ./hf.mp4 ./strip 8
|
||||
# -> strip/strip.png
|
||||
```
|
||||
|
||||
## Reading `diff/summary.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"frame_count": 90,
|
||||
"mean": 0.974,
|
||||
"min": 0.972,
|
||||
"max": 0.999,
|
||||
"p05": 0.972,
|
||||
"p95": 0.983,
|
||||
"threshold": 0.95,
|
||||
"pass": true
|
||||
}
|
||||
```
|
||||
|
||||
| Field | What it tells you |
|
||||
| ------------- | --------------------------------------------------------------------------- |
|
||||
| `mean` | average SSIM across all frames; the headline number |
|
||||
| `min` | worst frame; below threshold means at least one frame is structurally wrong |
|
||||
| `p05` / `p95` | 5th / 95th percentile — most frames sit between these |
|
||||
| `threshold` | from `R2HF_SSIM_THRESHOLD` env var (default 0.85) |
|
||||
| `pass` | whether `mean >= threshold` |
|
||||
|
||||
## Validated tier thresholds
|
||||
|
||||
Calibrated against actual Remotion + HF renders:
|
||||
|
||||
| Tier | Composition shape | Mean | Threshold | Margin |
|
||||
| ---- | ------------------------------------------- | ----- | --------- | ------ |
|
||||
| T1 | single-element fade-in | 0.974 | 0.95 | +0.022 |
|
||||
| T2 | multi-scene + spring + audio + image | 0.985 | 0.95 | +0.016 |
|
||||
| T3 | data-driven, custom subcomponents, count-up | 0.953 | 0.90 | +0.038 |
|
||||
|
||||
Each fixture's `expected.json` carries:
|
||||
|
||||
- `ssim_threshold` — the gate for `pass`
|
||||
- `validation` — the actual measured numbers from the calibration run
|
||||
- `translation_notes` — what's lossy and why
|
||||
|
||||
## Critical: encoder config
|
||||
|
||||
Both Remotion and HF must output the same pixel format for SSIM to be
|
||||
meaningful. Remotion's default JPEG output writes `yuvj420p` (full-range);
|
||||
HF outputs `yuv420p` (limited-range). The mismatch costs ~0.05 SSIM.
|
||||
|
||||
Every fixture's `remotion.config.ts` sets:
|
||||
|
||||
```ts
|
||||
Config.setVideoImageFormat("png");
|
||||
Config.setColorSpace("bt709");
|
||||
```
|
||||
|
||||
If the user's source doesn't have these, add them in the translation
|
||||
step — otherwise the diff measures encoder differences, not translation
|
||||
fidelity.
|
||||
|
||||
## What the noise floor looks like
|
||||
|
||||
The dominant non-translation noise is **system font fallback divergence**.
|
||||
Remotion's bundled Chromium and HF's `chrome-headless-shell` interpret
|
||||
`font-weight: 800` differently when there's no real font installed:
|
||||
|
||||
- Remotion HELLO at 160px: medium-weight stroke
|
||||
- HF HELLO at 160px: heavy-weight stroke
|
||||
|
||||
This costs ~0.025 mean SSIM. Visible in T1's frame strip.
|
||||
[fonts.md](fonts.md) covers how to mitigate (use Inter, load explicit
|
||||
Google Fonts).
|
||||
|
||||
## Threshold rule of thumb
|
||||
|
||||
Set the threshold ~0.02 below measured `p05`:
|
||||
|
||||
- Real translation regressions drop mean by 0.05+ — caught.
|
||||
- Encoder/font drift between CI runs is bounded at ~0.01 — not caught.
|
||||
|
||||
If a calibration run's measured mean is far above your initial threshold
|
||||
guess, _don't_ tighten the threshold to fit. Leave headroom — fixtures
|
||||
re-rendered on different hardware will drift.
|
||||
|
||||
## When the diff fails
|
||||
|
||||
1. **Look at `frame_strip.sh` output first.** A side-by-side strip at 6–10
|
||||
evenly-spaced timestamps shows whether the failure is structural
|
||||
(wrong scene durations, missing element) or cosmetic (different font
|
||||
weight, slight timing skew).
|
||||
2. **Check `diff/ssim.log`.** Per-frame SSIM tells you _which_ frames
|
||||
failed. Cluster of bad frames in the middle of a scene = animation
|
||||
problem; bad frames at scene boundaries = sequencing problem.
|
||||
3. **Re-read the relevant reference.** [timing.md](timing.md) for
|
||||
spring/easing issues, [sequencing.md](sequencing.md) for scene
|
||||
boundary issues, [media.md](media.md) for asset loading issues.
|
||||
|
||||
## CI integration
|
||||
|
||||
The fixtures are not yet wired into CI (`packages/producer/tests/` runs
|
||||
inside Docker; the skill corpus needs the same). PR 7 of the stack adds
|
||||
the orchestrator that runs all four tiers and emits an aggregated pass
|
||||
report. For now, evaluate by hand per fixture.
|
||||
@@ -0,0 +1,112 @@
|
||||
# Font translation
|
||||
|
||||
Fonts are the dominant non-translation noise floor. Same `font-weight: 800`
|
||||
renders perceptibly bolder on HF's `chrome-headless-shell` than on
|
||||
Remotion's bundled Chromium when there's no real font installed. Validation
|
||||
showed this costs ~0.025 mean SSIM at the noise floor.
|
||||
|
||||
## Pattern: `@remotion/google-fonts/<Family>`
|
||||
|
||||
```tsx
|
||||
import { loadFont } from "@remotion/google-fonts/Inter";
|
||||
loadFont("normal", { weights: ["400", "800"] });
|
||||
```
|
||||
|
||||
Translate to a `<link>` tag in `<head>`:
|
||||
|
||||
```html
|
||||
<head>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
|
||||
<link
|
||||
href="https://fonts.googleapis.com/css2?family=Inter:wght@400;800&display=swap"
|
||||
rel="stylesheet"
|
||||
/>
|
||||
<style>
|
||||
body {
|
||||
font-family: Inter, sans-serif;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
```
|
||||
|
||||
Pull the family name and weights from the import path and `loadFont`
|
||||
arguments. HF's compiler inlines the Google Fonts CSS at render time, so
|
||||
you don't pay a network round-trip per render.
|
||||
|
||||
## Pattern: local fonts via `@font-face`
|
||||
|
||||
```tsx
|
||||
import { Font } from "remotion";
|
||||
|
||||
Font.loadFont("/MyFont.woff2", "MyFont");
|
||||
```
|
||||
|
||||
Translate to a `@font-face` rule:
|
||||
|
||||
```html
|
||||
<style>
|
||||
@font-face {
|
||||
font-family: "MyFont";
|
||||
src: url("assets/MyFont.woff2") format("woff2");
|
||||
font-weight: 400;
|
||||
font-style: normal;
|
||||
}
|
||||
</style>
|
||||
```
|
||||
|
||||
Copy the font file into `hf-src/assets/` next to the HTML.
|
||||
|
||||
## Pattern: system font fallback (no font load)
|
||||
|
||||
```tsx
|
||||
<div style={{ fontFamily: "Helvetica, Arial, sans-serif" }}>...</div>
|
||||
```
|
||||
|
||||
Same string in HF — but be aware: on Linux without a real Helvetica
|
||||
installed (typical CI environment), Remotion and HF fall back to
|
||||
_different_ sans-serif system fonts because they bundle different
|
||||
Chromium versions. This is the noise floor: ~0.025 mean SSIM cost,
|
||||
visible as different stroke widths at large font weights (800+).
|
||||
|
||||
If matching the Remotion render exactly matters for a specific
|
||||
fixture, load the same font explicitly — don't rely on system
|
||||
fallback.
|
||||
|
||||
## When in doubt: use Inter
|
||||
|
||||
Inter renders identically across Chromium versions and is free.
|
||||
Translate any "system sans-serif" Remotion comp to Inter when you
|
||||
need to minimize font drift in the validation harness.
|
||||
|
||||
## Font loading and `delayRender`
|
||||
|
||||
Remotion uses `delayRender()` to defer the first frame until fonts
|
||||
load. HF's compiler inlines Google Fonts at compile time and waits
|
||||
on `@font-face` readiness via the Frame Adapter pattern — the
|
||||
`delayRender` call drops in translation. See [media.md](media.md).
|
||||
|
||||
## Multi-weight loading
|
||||
|
||||
When Remotion loads multiple weights:
|
||||
|
||||
```tsx
|
||||
loadFont("normal", { weights: ["400", "500", "700", "800"] });
|
||||
```
|
||||
|
||||
Inline all weights in the Google Fonts URL:
|
||||
|
||||
```
|
||||
?family=Inter:wght@400;500;700;800&display=swap
|
||||
```
|
||||
|
||||
Translation rule: enumerate every distinct `font-weight` value
|
||||
that appears in the composition's CSS (`font-weight: 800` →
|
||||
weight 800 must be loaded). If the Remotion source loads weights
|
||||
that aren't actually used, drop them.
|
||||
|
||||
## Font subsetting
|
||||
|
||||
Remotion's `loadFont` doesn't subset; HF's compiler doesn't either
|
||||
(yet). Don't try to optimize this in translation — it's lossless to
|
||||
keep the same weight set as the Remotion source.
|
||||
@@ -0,0 +1,136 @@
|
||||
# Translation limitations
|
||||
|
||||
What the skill explicitly cannot translate, separated from the
|
||||
blocker-list (which is enforced by `lint_source.py`). These are
|
||||
_known_ gaps — surface them to the user as translation notes
|
||||
when translating the surrounding composition.
|
||||
|
||||
## React patterns the skill refuses
|
||||
|
||||
See [escape-hatch.md](escape-hatch.md). Any of these triggers
|
||||
a bow-out:
|
||||
|
||||
- `useState`, `useReducer` driving animation
|
||||
- `useEffect` / `useLayoutEffect` with non-empty deps (side effects)
|
||||
- async `calculateMetadata`
|
||||
- Third-party React UI libraries (MUI, Chakra, Mantine, antd, shadcn, Radix, NextUI)
|
||||
|
||||
`@remotion/lambda` is no longer in this list — it's a warning, not a
|
||||
blocker, because Lambda config is orthogonal to composition rendering.
|
||||
The skill drops the imports and `renderMediaOnLambda(...)` calls and
|
||||
writes a `TRANSLATION_NOTES.md` entry. See
|
||||
[escape-hatch.md](escape-hatch.md).
|
||||
|
||||
## Patterns that work with caveats
|
||||
|
||||
### Volume ramps on `<Audio>`
|
||||
|
||||
Remotion accepts a function for `volume`:
|
||||
|
||||
```tsx
|
||||
<Audio src={...} volume={(f) => interpolate(f, [0, 30], [0, 1])} />
|
||||
```
|
||||
|
||||
HF supports static `data-volume` only. Translation: bake the ramp into
|
||||
the audio file at translation time using `ffmpeg afade`, OR drop the
|
||||
ramp with a note. The dropped-ramp path produces audibly different
|
||||
output but visually-identical video, so SSIM passes — just flag it.
|
||||
|
||||
### `<Loop>` with stateful children
|
||||
|
||||
```tsx
|
||||
<Loop durationInFrames={30}>
|
||||
<CounterThatIncrementsViaUseRef />
|
||||
</Loop>
|
||||
```
|
||||
|
||||
Loop with `repeat: -1` works for _visual_ repetition. If the looped
|
||||
child has cross-iteration state (a counter, a randomness seed), HF
|
||||
won't reproduce it identically per iteration. Bow out unless the
|
||||
child is fully deterministic per-iteration.
|
||||
|
||||
### Remotion's `<Img>` with crossOrigin
|
||||
|
||||
```tsx
|
||||
<Img src="https://other-domain.com/x.png" crossOrigin="anonymous" />
|
||||
```
|
||||
|
||||
HF's renderer doesn't enforce CORS the same way Remotion does. Most
|
||||
public images work; private images served with auth headers won't.
|
||||
If the source uses `crossOrigin="use-credentials"`, the asset needs
|
||||
to be downloaded and inlined at translation time.
|
||||
|
||||
### Custom `presentation` in `<TransitionSeries>`
|
||||
|
||||
```tsx
|
||||
const customPresentation: PresentationComponent = ({ children, presentationProgress }) => {
|
||||
return <div style={{ filter: `blur(${(1 - presentationProgress) * 20}px)` }}>{children}</div>;
|
||||
};
|
||||
```
|
||||
|
||||
Pure presentations (transform/filter/opacity computed from progress)
|
||||
translate to GSAP tweens cleanly. Presentations that read
|
||||
`useCurrentFrame()` internally or have stateful children don't —
|
||||
bow out.
|
||||
|
||||
### Code-split components (`React.lazy`)
|
||||
|
||||
```tsx
|
||||
const HeavyChart = React.lazy(() => import("./HeavyChart"));
|
||||
```
|
||||
|
||||
`React.lazy` is async and doesn't fit the deterministic-render model.
|
||||
Translate to a regular import; the resulting HF composition will
|
||||
just include all the code upfront.
|
||||
|
||||
## Patterns that always work
|
||||
|
||||
- `<AbsoluteFill>` and `<Sequence>` (any nesting)
|
||||
- `useCurrentFrame()` derivations: `interpolate`, `spring`, `Easing`,
|
||||
`interpolateColors`, manual math
|
||||
- `<Audio>`, `<Video>`, `<Img>`, `<IFrame>` with simple props
|
||||
- `staticFile()` references
|
||||
- Custom React subcomponents that are pure functions of props
|
||||
- Custom hooks that are pure derivations of `useCurrentFrame`
|
||||
- `@remotion/lottie` (translates to HF's Lottie adapter)
|
||||
- `@remotion/google-fonts/<Family>` (translates to `<link>` or `@font-face`)
|
||||
- Sync `calculateMetadata` (resolved at translation time)
|
||||
- `<TransitionSeries>` with built-in presentations (`fade`, `slide`,
|
||||
`wipe`, `clockWipe`, `flip`, `iris`)
|
||||
|
||||
## What the skill never tries to translate
|
||||
|
||||
These are out-of-scope by design:
|
||||
|
||||
- **HDR rendering** — HF supports HDR but Remotion doesn't, so there's
|
||||
nothing to translate from.
|
||||
- **Variable frame rate** — both tools assume constant fps.
|
||||
- **Multi-composition `<Composition>` lists** — translate one at a
|
||||
time. The skill prompts the user to choose which composition.
|
||||
- **Remotion Studio props panel** — visual prop editing in HF Studio
|
||||
needs different infrastructure; out of scope.
|
||||
|
||||
## Reporting gaps to the user
|
||||
|
||||
When translation produces _something_ but the something has gaps, write
|
||||
a `TRANSLATION_NOTES.md` next to the output:
|
||||
|
||||
```markdown
|
||||
# Translation notes
|
||||
|
||||
The following Remotion patterns were translated with caveats:
|
||||
|
||||
- `<Audio volume={(f) => ...}>` (line 15): volume ramp dropped — added
|
||||
static `data-volume="0.5"`. To preserve the ramp, run
|
||||
`ffmpeg -i music.wav -af "afade=t=in:st=0:d=1" music.faded.wav` and
|
||||
swap the source file.
|
||||
- `<HeavyChart>` (line 30): translated as inline HTML. The original
|
||||
React.lazy boundary was dropped — bundle size unchanged because HF
|
||||
serves a single HTML file.
|
||||
|
||||
If any of these caveats matter, consider the runtime interop pattern
|
||||
instead.
|
||||
```
|
||||
|
||||
This file is also generated by the skill alongside the HF output, not
|
||||
held in the corpus.
|
||||
@@ -0,0 +1,121 @@
|
||||
# Lottie translation: @remotion/lottie → HF lottie adapter
|
||||
|
||||
Lottie animations are a clean translation case — HF has a built-in
|
||||
[Lottie adapter](https://github.com/heygen-com/hyperframes/blob/main/packages/core/src/runtime/adapters/lottie.ts)
|
||||
that supports both `lottie-web` and `@lottiefiles/dotlottie-web`. The
|
||||
adapter auto-discovers animations registered on `window.__hfLottie`
|
||||
and seeks them per-frame via `goToAndStop`.
|
||||
|
||||
## Pattern
|
||||
|
||||
```tsx
|
||||
import { Lottie } from "@remotion/lottie";
|
||||
import animationData from "./hello.json";
|
||||
|
||||
export const MyComp = () => (
|
||||
<AbsoluteFill>
|
||||
<Lottie animationData={animationData} loop={false} />
|
||||
</AbsoluteFill>
|
||||
);
|
||||
```
|
||||
|
||||
Translates to:
|
||||
|
||||
```html
|
||||
<div id="stage" ...>
|
||||
<div id="lottie-anim" style="width:100%;height:100%"></div>
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/bodymovin/5.12.2/lottie.min.js"></script>
|
||||
<script>
|
||||
const anim = lottie.loadAnimation({
|
||||
container: document.getElementById("lottie-anim"),
|
||||
renderer: "svg",
|
||||
loop: false,
|
||||
autoplay: false,
|
||||
path: "assets/hello.json",
|
||||
});
|
||||
window.__hfLottie = window.__hfLottie || [];
|
||||
window.__hfLottie.push(anim);
|
||||
</script>
|
||||
</div>
|
||||
```
|
||||
|
||||
Key differences from a typical Lottie embed:
|
||||
|
||||
- `autoplay: false` — HF drives playback by seeking
|
||||
- `loop: false` typically (unless Remotion's `loop={true}`)
|
||||
- `window.__hfLottie.push(anim)` is what hooks the animation into HF's
|
||||
per-frame seek
|
||||
|
||||
## Asset handling
|
||||
|
||||
Remotion bundles the animation JSON via webpack import. HF needs the JSON
|
||||
on disk under `assets/` and references it via path:
|
||||
|
||||
1. Copy `hello.json` from the Remotion project into `hf-src/assets/`.
|
||||
2. Reference as `path: "assets/hello.json"` in `loadAnimation`.
|
||||
|
||||
For dotlottie (binary) format, swap in `@lottiefiles/dotlottie-web`:
|
||||
|
||||
```html
|
||||
<script src="https://unpkg.com/@lottiefiles/dotlottie-web"></script>
|
||||
<canvas id="anim" style="width:100%;height:100%"></canvas>
|
||||
<script>
|
||||
const player = new DotLottie({
|
||||
canvas: document.getElementById("anim"),
|
||||
src: "assets/hello.lottie",
|
||||
autoplay: false,
|
||||
});
|
||||
window.__hfLottie = window.__hfLottie || [];
|
||||
window.__hfLottie.push(player);
|
||||
</script>
|
||||
```
|
||||
|
||||
The HF adapter handles both player APIs (it duck-types `goToAndStop`
|
||||
vs `setCurrentRawFrameValue` / `seek`).
|
||||
|
||||
## Multiple Lottie animations
|
||||
|
||||
Multiple `<Lottie>` instances in one composition work — push each one
|
||||
onto `window.__hfLottie` and the adapter will seek all of them in sync:
|
||||
|
||||
```js
|
||||
window.__hfLottie.push(anim1);
|
||||
window.__hfLottie.push(anim2);
|
||||
window.__hfLottie.push(anim3);
|
||||
```
|
||||
|
||||
## Lottie source isn't actually translation-blocking
|
||||
|
||||
Lottie animations encode their own deterministic timeline. They're the
|
||||
_easiest_ part of a Remotion composition to translate because the
|
||||
animation logic is already self-contained — neither Remotion nor HF
|
||||
"animate" them, both just seek them. Translation cost is near-zero.
|
||||
|
||||
## After Effects → Lottie limitations
|
||||
|
||||
Lottie supports a subset of After Effects features. Expressions, most
|
||||
Effects (drop shadow, color overlay), all blend modes beyond Normal/Add/
|
||||
Multiply, luma mattes, and most 3D parameters are not supported. If the
|
||||
Remotion composition uses a Lottie file that depends on these, the
|
||||
animation will break in BOTH Remotion and HF — this isn't a translation
|
||||
problem, it's a Lottie limitation. See
|
||||
[airbnb/lottie/after-effects.md](https://github.com/airbnb/lottie/blob/master/after-effects.md)
|
||||
for the full supported feature list.
|
||||
|
||||
## Loop behavior
|
||||
|
||||
Remotion's `loop={true}` plays the animation continuously. Translate to
|
||||
the player option only after checking the generated frames. The HF
|
||||
adapter seeks absolute composition time; it does not add modulo looping
|
||||
or playback-rate scaling on top of the player. For exact repeating
|
||||
cycles or non-default playback rates, bake the timing into the Lottie
|
||||
asset or author an explicit timeline around the Lottie layer and verify
|
||||
the rendered output.
|
||||
|
||||
## Performance note
|
||||
|
||||
Per the [Lottie adapter](https://github.com/heygen-com/hyperframes/blob/main/packages/core/src/runtime/adapters/lottie.ts)
|
||||
docs: lottie-web's `goToAndStop(time, isFrame=false)` takes time in ms;
|
||||
the adapter passes `time * 1000` for precision. This is more accurate
|
||||
than passing frame numbers (especially for animations whose internal
|
||||
fps doesn't match the HF render fps).
|
||||
@@ -0,0 +1,149 @@
|
||||
# Media translation: Audio, Video, Img, IFrame, staticFile
|
||||
|
||||
## Asset paths
|
||||
|
||||
Remotion's `staticFile("x.png")` resolves to the project's `public/` directory.
|
||||
HF uses relative paths from the composition's `index.html`, conventionally
|
||||
`assets/`:
|
||||
|
||||
```tsx
|
||||
<Img src={staticFile("logo.png")} />
|
||||
```
|
||||
|
||||
```html
|
||||
<img src="assets/logo.png" />
|
||||
```
|
||||
|
||||
When translating, copy the asset from `remotion-src/public/x` to
|
||||
`hf-src/assets/x`. Multiple files can be batched with a setup script;
|
||||
see T2's `setup.sh` for an example pattern.
|
||||
|
||||
## `<Audio>`
|
||||
|
||||
```tsx
|
||||
<Audio src={staticFile("music.wav")} volume={0.5} />
|
||||
```
|
||||
|
||||
```html
|
||||
<audio
|
||||
data-start="0"
|
||||
data-duration="6"
|
||||
data-track-index="2"
|
||||
data-volume="0.5"
|
||||
src="assets/music.wav"
|
||||
></audio>
|
||||
```
|
||||
|
||||
`data-start` and `data-duration` are required — the runtime needs them to
|
||||
schedule the audio. Default to the composition's full duration if Remotion
|
||||
didn't specify trim.
|
||||
|
||||
### Volume ramps
|
||||
|
||||
```tsx
|
||||
<Audio src={staticFile("music.wav")} volume={(f) => interpolate(f, [0, 30], [0, 1])} />
|
||||
```
|
||||
|
||||
HF supports static `data-volume` only for now. Volume ramps need to be
|
||||
applied to the audio file at translation time (with ffmpeg `afade`) or the
|
||||
ramp is dropped with a translation note.
|
||||
|
||||
### Trim / playbackRate
|
||||
|
||||
```tsx
|
||||
<Audio src={staticFile("music.wav")} startFrom={60} endAt={180} playbackRate={1.5} />
|
||||
```
|
||||
|
||||
```html
|
||||
<audio
|
||||
data-start="0"
|
||||
data-duration="<resolved from trim>"
|
||||
data-trim-start="2"
|
||||
data-trim-end="6"
|
||||
data-playback-rate="1.5"
|
||||
src="assets/music.wav"
|
||||
></audio>
|
||||
```
|
||||
|
||||
`startFrom` / `endAt` are frame indexes; convert to seconds.
|
||||
|
||||
## `<Video>` and `<OffthreadVideo>`
|
||||
|
||||
```tsx
|
||||
<Video src={staticFile("intro.mp4")} muted playsInline />
|
||||
<OffthreadVideo src={staticFile("intro.mp4")} muted />
|
||||
```
|
||||
|
||||
```html
|
||||
<video
|
||||
muted
|
||||
playsinline
|
||||
data-start="0"
|
||||
data-duration="5"
|
||||
data-track-index="0"
|
||||
src="assets/intro.mp4"
|
||||
></video>
|
||||
```
|
||||
|
||||
`<OffthreadVideo>` is a Remotion-specific optimization for headless
|
||||
rendering. HF runs in headless Chrome already, so the off-thread variant
|
||||
collapses to a regular `<video>`.
|
||||
|
||||
`muted` and `playsinline` are required for the runtime to autoplay
|
||||
(browser policy). Always emit them.
|
||||
|
||||
## `<Img>`
|
||||
|
||||
```tsx
|
||||
<Img src={staticFile("logo.png")} style={{ width: 200, height: 200 }} />
|
||||
```
|
||||
|
||||
```html
|
||||
<img src="assets/logo.png" style="width: 200px; height: 200px;" />
|
||||
```
|
||||
|
||||
Width/height get rounded to integer px. If the original style has
|
||||
animated dimensions, the GSAP tween animates them — see [timing.md](timing.md).
|
||||
|
||||
## `<IFrame>`
|
||||
|
||||
```tsx
|
||||
<IFrame src="https://example.com" />
|
||||
```
|
||||
|
||||
```html
|
||||
<iframe src="https://example.com"></iframe>
|
||||
```
|
||||
|
||||
When HF detects a nested iframe in a composition, it auto-falls back to
|
||||
**screenshot mode** rather than the deterministic BeginFrame mode. This
|
||||
costs render performance but produces visibly-correct output. See
|
||||
[hyperframes-vs-remotion.mdx](https://github.com/heygen-com/hyperframes/blob/main/docs/guides/hyperframes-vs-remotion.mdx)
|
||||
for details.
|
||||
|
||||
## `delayRender()` / `continueRender()`
|
||||
|
||||
```tsx
|
||||
const handle = delayRender();
|
||||
useEffect(() => {
|
||||
loadAsset().then(() => continueRender(handle));
|
||||
}, []);
|
||||
```
|
||||
|
||||
Drop. HF waits on asset readiness via the [Frame Adapter pattern](https://hyperframes.heygen.com/concepts/frame-adapters)
|
||||
— images, videos, fonts, and Lottie animations all signal load
|
||||
completion natively. There's nothing to do at the application level.
|
||||
|
||||
## When the asset isn't a file
|
||||
|
||||
If Remotion's media source is a Buffer, dataURL, or URL.createObjectURL,
|
||||
the asset doesn't exist on disk and can't be copied via setup.sh. Two
|
||||
options:
|
||||
|
||||
1. Materialize the asset at translation time — write the buffer to a file
|
||||
in `hf-src/assets/`.
|
||||
2. Embed as a data URL directly in the HTML (`src="data:image/png;base64,..."`)
|
||||
for small assets (< 100 KB).
|
||||
|
||||
For audio/video Buffers, option 1 is preferred — base64-encoded media
|
||||
bloats the HTML and slows the renderer.
|
||||
@@ -0,0 +1,167 @@
|
||||
# Parameter translation: Zod schemas, defaultProps, calculateMetadata
|
||||
|
||||
How a typed Remotion `<Composition schema={...} defaultProps={...} />`
|
||||
turns into a parameterized HF composition.
|
||||
|
||||
## Sync calculateMetadata (translatable)
|
||||
|
||||
```tsx
|
||||
<Composition
|
||||
id="MyVideo"
|
||||
component={MyVideo}
|
||||
schema={z.object({ title: z.string(), duration: z.number() })}
|
||||
defaultProps={{ title: "Hello", duration: 90 }}
|
||||
calculateMetadata={({ props }) => ({
|
||||
durationInFrames: props.duration,
|
||||
fps: 30,
|
||||
})}
|
||||
/>
|
||||
```
|
||||
|
||||
When `calculateMetadata` is synchronous and only uses `props`, **resolve
|
||||
it at translation time** — call it with `defaultProps` (or whatever the
|
||||
caller specifies) and write the concrete result into the HTML:
|
||||
|
||||
```html
|
||||
<div
|
||||
id="stage"
|
||||
data-composition-id="MyVideo"
|
||||
data-start="0"
|
||||
data-duration="3" <!-- 90/30 -->
|
||||
data-fps="30"
|
||||
data-title="Hello"
|
||||
></div>
|
||||
```
|
||||
|
||||
The `data-title` attribute carries the value through. Code that originally
|
||||
read `props.title` reads `document.getElementById("stage").dataset.title`
|
||||
in HF.
|
||||
|
||||
## Async calculateMetadata (NOT translatable)
|
||||
|
||||
```tsx
|
||||
<Composition
|
||||
calculateMetadata={async ({ props }) => {
|
||||
const res = await fetch(...);
|
||||
return { durationInFrames: res.duration };
|
||||
}}
|
||||
/>
|
||||
```
|
||||
|
||||
**Refuse + interop**. HF needs composition metadata up-front to seed the
|
||||
HTML. Resolving network calls at translation time defeats the purpose
|
||||
of having dynamic metadata. See [escape-hatch.md](escape-hatch.md).
|
||||
|
||||
The lint rule `r2hf/async-metadata` catches this. T4 case 03 tests it.
|
||||
|
||||
## Default props
|
||||
|
||||
```tsx
|
||||
defaultProps={{
|
||||
title: "Hello",
|
||||
subtitle: "World",
|
||||
count: 42,
|
||||
}}
|
||||
```
|
||||
|
||||
Translate to `data-*` attributes on the root `#stage` div:
|
||||
|
||||
```html
|
||||
<div id="stage" data-title="Hello" data-subtitle="World" data-count="42">...</div>
|
||||
```
|
||||
|
||||
Convention: `propName` → `data-prop-name` (kebab-case). Inside the GSAP
|
||||
script, read via `document.getElementById("stage").dataset.propName`.
|
||||
|
||||
## Nested object / array props
|
||||
|
||||
```tsx
|
||||
defaultProps={{
|
||||
stats: [
|
||||
{ label: "Stars", value: 1247, color: "#fbbf24" },
|
||||
{ label: "Forks", value: 312, color: "#60a5fa" },
|
||||
],
|
||||
}}
|
||||
```
|
||||
|
||||
Don't try to encode the array as a JSON `data-` attribute — HF's runtime
|
||||
doesn't parse those. Materialize the array as **repeated HTML markup**:
|
||||
|
||||
```html
|
||||
<div id="scene-stats">
|
||||
<div class="stat-card" data-stat-index="0" data-stat-value="1247" style="--card-color:#fbbf24">
|
||||
<div class="number">0</div>
|
||||
<div class="label">Stars</div>
|
||||
</div>
|
||||
<div class="stat-card" data-stat-index="1" data-stat-value="312" style="--card-color:#60a5fa">
|
||||
<div class="number">0</div>
|
||||
<div class="label">Forks</div>
|
||||
</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
The component template (`StatCard.tsx`) becomes the markup template;
|
||||
each instance gets its scalar props rendered as `data-*` and CSS
|
||||
custom properties.
|
||||
|
||||
Validated in T3 — three StatCards reused with different props,
|
||||
mean SSIM 0.953.
|
||||
|
||||
## Numeric props that need typed parsing
|
||||
|
||||
`document.getElementById("stage").dataset.count` is a string. Convert at
|
||||
read time:
|
||||
|
||||
```js
|
||||
const count = Number(stage.dataset.count);
|
||||
```
|
||||
|
||||
Or inline values directly into the GSAP script when the data is known
|
||||
at translation time and doesn't need to vary per render.
|
||||
|
||||
## Boolean props
|
||||
|
||||
```tsx
|
||||
defaultProps={{ darkMode: true }}
|
||||
```
|
||||
|
||||
Two conventions:
|
||||
|
||||
- `data-dark-mode="true"` — read as string, compare `=== "true"`
|
||||
- `data-dark-mode` (presence/absence) — `<div data-dark-mode>` for true, omit for false
|
||||
|
||||
Pick one and be consistent. The presence/absence form is HTML-idiomatic
|
||||
and pairs well with CSS attribute selectors:
|
||||
|
||||
```css
|
||||
[data-dark-mode] .scene {
|
||||
background: #000;
|
||||
}
|
||||
```
|
||||
|
||||
## Zod runtime validation
|
||||
|
||||
Remotion's `schema` validates props at composition load. HF doesn't have
|
||||
an equivalent — by the time the HTML is in the renderer, the schema is
|
||||
already gone.
|
||||
|
||||
Validate at translation time instead. If the user passes invalid data,
|
||||
fail with a translation error before emitting HTML. This matches Zod's
|
||||
"fail loud" intent without requiring the runtime dependency.
|
||||
|
||||
## When the composition uses props for computed prop derivation
|
||||
|
||||
```tsx
|
||||
const Composition: React.FC<Props> = ({ stats }) => {
|
||||
const total = stats.reduce((acc, s) => acc + s.value, 0);
|
||||
return <div>{total}</div>;
|
||||
};
|
||||
```
|
||||
|
||||
Compute the derived value at translation time and bake it into the HTML
|
||||
or a `data-` attribute. Don't try to express the computation in JS in the
|
||||
HF composition — that adds runtime overhead and makes the HTML stateful
|
||||
in ways that complicate human editing.
|
||||
|
||||
If the derivation is non-trivial (involves the array itself, not just
|
||||
scalars), materialize it as static text in the HTML.
|
||||
@@ -0,0 +1,195 @@
|
||||
# Sequencing translation: Sequence, Series, Composition root
|
||||
|
||||
How Remotion's nested `Sequence` tree maps to HF's flat `data-start` /
|
||||
`data-duration` markup with a single paused GSAP timeline.
|
||||
|
||||
## The core idea
|
||||
|
||||
Remotion's `<Sequence from={F} durationInFrames={D}>` is a coordinate
|
||||
transform: it shifts `useCurrentFrame()` by `F` and clips the child
|
||||
component to the window `[F, F+D]`. HF doesn't have a per-element
|
||||
"current frame" — there's a single composition seek time and the
|
||||
runtime hides/shows elements based on their `data-start` / `data-duration`.
|
||||
|
||||
Result: the nested tree flattens into a list of siblings on the same
|
||||
parent, each with their own time window.
|
||||
|
||||
## `<Composition>` → root `#stage`
|
||||
|
||||
```tsx
|
||||
<Composition
|
||||
id="MyVideo"
|
||||
component={MyVideo}
|
||||
durationInFrames={300}
|
||||
fps={30}
|
||||
width={1280}
|
||||
height={720}
|
||||
/>
|
||||
```
|
||||
|
||||
```html
|
||||
<div
|
||||
id="stage"
|
||||
data-composition-id="MyVideo"
|
||||
data-start="0"
|
||||
data-duration="10" <!-- 300/30 -->
|
||||
data-fps="30"
|
||||
data-width="1280"
|
||||
data-height="720"
|
||||
>
|
||||
<!-- composition content -->
|
||||
</div>
|
||||
```
|
||||
|
||||
`data-start="0"` is required on `#stage` (the runtime needs it to anchor
|
||||
playback; missing it triggers a lint warning).
|
||||
|
||||
## `<AbsoluteFill>` → positioned div
|
||||
|
||||
```tsx
|
||||
<AbsoluteFill style={{ backgroundColor: "#0a0a0a" }}>...children...</AbsoluteFill>
|
||||
```
|
||||
|
||||
```html
|
||||
<div style="position:absolute;inset:0;background-color:#0a0a0a;">...children...</div>
|
||||
```
|
||||
|
||||
`AbsoluteFill` is just a styled div in Remotion. Translate to a div with
|
||||
`position:absolute; inset:0` and copy through any other style props.
|
||||
|
||||
## `<Sequence>` → time-windowed div
|
||||
|
||||
```tsx
|
||||
<Sequence from={0} durationInFrames={90}>
|
||||
<TitleCard />
|
||||
</Sequence>
|
||||
```
|
||||
|
||||
```html
|
||||
<div data-start="0" data-duration="3" data-track-index="0">
|
||||
<!-- TitleCard children inlined -->
|
||||
</div>
|
||||
```
|
||||
|
||||
Convert frames to seconds: `from/fps`, `durationInFrames/fps`. Pick a
|
||||
`data-track-index` per parallel rendering layer (background = 0,
|
||||
overlays = 1, audio = 2, etc.). Sequential scenes can share an index.
|
||||
|
||||
## Nested `<Sequence>` flattens
|
||||
|
||||
Remotion adds offsets when sequences nest:
|
||||
|
||||
```tsx
|
||||
<Sequence from={60} durationInFrames={120}>
|
||||
<Sequence from={30} durationInFrames={60}>
|
||||
<ImageScene />
|
||||
</Sequence>
|
||||
</Sequence>
|
||||
```
|
||||
|
||||
The inner sequence's effective window is `[60+30, 60+30+60] = [90, 150]`.
|
||||
|
||||
Translate by computing the sum and emitting one HF div with the resolved
|
||||
window:
|
||||
|
||||
```html
|
||||
<div data-start="3" data-duration="2" data-track-index="0">
|
||||
<!-- ImageScene children -->
|
||||
</div>
|
||||
```
|
||||
|
||||
## `<Series>` → siblings with sequential offsets
|
||||
|
||||
```tsx
|
||||
<Series>
|
||||
<Series.Sequence durationInFrames={60}>
|
||||
<A />
|
||||
</Series.Sequence>
|
||||
<Series.Sequence durationInFrames={120}>
|
||||
<B />
|
||||
</Series.Sequence>
|
||||
<Series.Sequence durationInFrames={90}>
|
||||
<C />
|
||||
</Series.Sequence>
|
||||
</Series>
|
||||
```
|
||||
|
||||
Each `Sequence.Sequence` lives in the next time slot. Emit siblings
|
||||
with `data-start` accumulating:
|
||||
|
||||
```html
|
||||
<div data-start="0" data-duration="2" data-track-index="0">A</div>
|
||||
<div data-start="2" data-duration="4" data-track-index="0">B</div>
|
||||
<div data-start="6" data-duration="3" data-track-index="0">C</div>
|
||||
```
|
||||
|
||||
## Crossfading scene boundaries
|
||||
|
||||
Remotion `<Sequence>` shows/hides at hard boundaries by default. HF does
|
||||
the same — but if your composition needs a smooth fade between scenes,
|
||||
you have to drive opacity explicitly with GSAP at the boundary:
|
||||
|
||||
```js
|
||||
const tl = gsap.timeline({ paused: true });
|
||||
tl.set(scene1, { opacity: 1 }, 0);
|
||||
tl.set(scene1, { opacity: 0 }, 2); // hard cut at 2s
|
||||
tl.set(scene2, { opacity: 1 }, 2);
|
||||
```
|
||||
|
||||
For a 0.5 s crossfade:
|
||||
|
||||
```js
|
||||
tl.to(scene1, { opacity: 0, duration: 0.5 }, 1.5);
|
||||
tl.to(scene2, { opacity: 1, duration: 0.5 }, 1.5);
|
||||
```
|
||||
|
||||
For Remotion `<TransitionSeries>` translations see [transitions.md](transitions.md).
|
||||
|
||||
## `<Loop>`
|
||||
|
||||
```tsx
|
||||
<Loop durationInFrames={30}>
|
||||
<Spinner />
|
||||
</Loop>
|
||||
```
|
||||
|
||||
HF doesn't have a `<Loop>` primitive. Translate to a GSAP timeline with
|
||||
`repeat: -1`:
|
||||
|
||||
```js
|
||||
const spinTl = gsap.timeline({ paused: true, repeat: -1, repeatRefresh: false });
|
||||
spinTl.to(spinner, { rotate: 360, duration: 1.0, ease: "none" });
|
||||
// Embed in the main composition timeline at the right offset:
|
||||
mainTl.add(spinTl, 3);
|
||||
```
|
||||
|
||||
This is fragile — Remotion's `<Loop>` resets internal state every iteration,
|
||||
which GSAP repeat does too, but if the looped child has its own animation,
|
||||
you need to be careful that GSAP's `repeatRefresh` is on or off as needed.
|
||||
For most simple "spin forever" cases this is fine.
|
||||
|
||||
## `<Freeze>`
|
||||
|
||||
```tsx
|
||||
<Freeze frame={30}>
|
||||
<Animated />
|
||||
</Freeze>
|
||||
```
|
||||
|
||||
Drop the wrapper. `<Freeze>` pins `useCurrentFrame()` at a constant for
|
||||
the children — but in HF, the children's animation is already driven by
|
||||
explicit GSAP tweens, so freeze translates to "don't tween this element".
|
||||
|
||||
## Multiple parallel tracks
|
||||
|
||||
When you have a background video + overlay text + audio playing
|
||||
simultaneously, use distinct `data-track-index` values:
|
||||
|
||||
```html
|
||||
<div data-track-index="0">background video</div>
|
||||
<div data-track-index="1">overlay text</div>
|
||||
<audio data-track-index="2" ...></audio>
|
||||
```
|
||||
|
||||
The runtime picks track ordering from the index. See [media.md](media.md)
|
||||
for media-specific track conventions.
|
||||
@@ -0,0 +1,165 @@
|
||||
# Timing translation: interpolate, spring, easing
|
||||
|
||||
The single highest-leverage reference. Easings and timings are what readers
|
||||
notice; getting them wrong costs more SSIM than any other translation choice.
|
||||
Empirically validated against tiers T1–T3.
|
||||
|
||||
## Conversion: frames → seconds
|
||||
|
||||
HF's timeline is in seconds. Remotion is frame-based. Always:
|
||||
|
||||
```
|
||||
time_seconds = frame / fps
|
||||
```
|
||||
|
||||
So at fps=30:
|
||||
|
||||
- frame 15 → 0.5 s
|
||||
- frame 30 → 1.0 s
|
||||
- frame 90 → 3.0 s
|
||||
|
||||
Do this conversion once when translating, not at runtime.
|
||||
|
||||
## interpolate — linear
|
||||
|
||||
```tsx
|
||||
const opacity = interpolate(frame, [0, 30], [0, 1], { extrapolateRight: "clamp" });
|
||||
```
|
||||
|
||||
Translates to:
|
||||
|
||||
```js
|
||||
gsap.to(target, { opacity: 1, duration: 1.0, ease: "none" }, 0);
|
||||
// fromTo if the property starts at 0 and CSS doesn't already set it
|
||||
gsap.fromTo(target, { opacity: 0 }, { opacity: 1, duration: 1.0, ease: "none" }, 0);
|
||||
```
|
||||
|
||||
`ease: "none"` matches Remotion's default linear interpolation. CSS sets the
|
||||
`from` value if your initial state is in CSS; otherwise use `fromTo`.
|
||||
|
||||
`extrapolateLeft`/`extrapolateRight` defaults to `"extend"` in Remotion but
|
||||
`"clamp"` is what the agent will see most often. GSAP doesn't extend — values
|
||||
hold at the start and end of the tween. So for `clamp`, GSAP matches; for
|
||||
`extend`, you'd need to extend the input range manually before emitting.
|
||||
|
||||
## interpolate — multi-segment
|
||||
|
||||
```tsx
|
||||
const opacity = interpolate(frame, [0, 15, 75, 90], [0, 1, 1, 0]);
|
||||
```
|
||||
|
||||
Three keyframed tweens at offsets `[0]/fps`, `[1]/fps`, `[2]/fps`:
|
||||
|
||||
```js
|
||||
const tl = gsap.timeline({ paused: true });
|
||||
tl.to(target, { opacity: 1, duration: 0.5, ease: "none" }, 0);
|
||||
tl.to(target, { opacity: 1, duration: 2.0, ease: "none" }, 0.5);
|
||||
tl.to(target, { opacity: 0, duration: 0.5, ease: "none" }, 2.5);
|
||||
```
|
||||
|
||||
Validated in T1 — mean SSIM 0.974 against Remotion baseline.
|
||||
|
||||
## spring → GSAP back.out
|
||||
|
||||
Remotion's `spring()` is the most lossy translation. The mapping is approximate
|
||||
but close enough that real-world compositions hold ≥ 0.92 SSIM (T2: 0.985, T3: 0.953).
|
||||
|
||||
| Remotion `spring` config | GSAP equivalent | Validated in |
|
||||
| ------------------------------------------------- | ---------------------------------------------------- | -------------------------------- |
|
||||
| `{damping: 12, stiffness: 100, mass: 1}` (snappy) | `back.out(1.4)` over ~0.7 s | T2, T3 (TitleScene) |
|
||||
| `{damping: 14, stiffness: 90, mass: 1}` (calmer) | `back.out(1.2)` over ~0.7 s | T3 (StatCard) |
|
||||
| `{damping: 8, stiffness: 200}` (very bouncy) | `back.out(2.0)` or `elastic.out(1, 0.5)` over ~0.6 s | not validated; budget ~0.05 SSIM |
|
||||
| `{overshootClamping: true}` | `power3.out` over ~0.6 s (no overshoot) | not validated |
|
||||
|
||||
**Rule of thumb**: `back.out(N)` overshoot ratio ≈ `(stiffness / damping^2) * 1.4`. For
|
||||
`damping:12, stiffness:100` that gives `1.4 * 100/144 = 0.97`, which is close to
|
||||
the validated 1.4 (the formula is rough; tune by visual). Default duration is
|
||||
~0.7 s for the typical config.
|
||||
|
||||
When the spring's `delay`/`from`/`to` are non-default, scale the duration
|
||||
proportionally.
|
||||
|
||||
## interpolate with custom easing
|
||||
|
||||
```tsx
|
||||
import { Easing } from "remotion";
|
||||
interpolate(frame, [0, 30], [0, 1], { easing: Easing.out(Easing.cubic) });
|
||||
```
|
||||
|
||||
| Remotion | GSAP |
|
||||
| ---------------------------- | -------------------------------------------------------------------------------------- |
|
||||
| `Easing.in(Easing.linear)` | `ease: "none"` |
|
||||
| `Easing.out(Easing.cubic)` | `ease: "power3.out"` |
|
||||
| `Easing.inOut(Easing.cubic)` | `ease: "power3.inOut"` |
|
||||
| `Easing.out(Easing.poly(N))` | `ease: "power<N>.out"` (N=2 quad, 3 cubic, 4 quart, 5 quint) |
|
||||
| `Easing.bezier(a,b,c,d)` | `CustomEase.create("c", "M0,0 C${a},${b} ${c},${d} 1,1")` (requires CustomEase plugin) |
|
||||
| `Easing.elastic(bounciness)` | `ease: "elastic.out(${bounciness}, 0.3)"` |
|
||||
| `Easing.bounce` | `ease: "bounce.out"` |
|
||||
| `Easing.back(overshoot)` | `ease: "back.out(${overshoot * 1.7})"` (Remotion's overshoot scale differs) |
|
||||
|
||||
## interpolate driving non-numeric properties
|
||||
|
||||
```tsx
|
||||
const color = interpolateColors(frame, [0, 30], ["#ff0000", "#0000ff"]);
|
||||
```
|
||||
|
||||
GSAP does color tweens natively:
|
||||
|
||||
```js
|
||||
gsap.to(target, { color: "#0000ff", duration: 1.0, ease: "none" }, 0);
|
||||
```
|
||||
|
||||
Same for `backgroundColor`, `borderColor`. The `from` value is read from CSS
|
||||
or the inline style.
|
||||
|
||||
## Custom count-up / number tweens
|
||||
|
||||
When Remotion uses a frame-driven number ramp (`Math.round(value * eased)`):
|
||||
|
||||
```tsx
|
||||
const t = interpolate(frame, [0, 45], [0, 1]);
|
||||
const eased = 1 - (1 - t) ** 3; // cubic ease-out
|
||||
const value = Math.round(target * eased);
|
||||
return <div>{value.toLocaleString()}</div>;
|
||||
```
|
||||
|
||||
GSAP equivalent — tween a counter object, write `textContent` on update:
|
||||
|
||||
```js
|
||||
const counter = { v: 0 };
|
||||
tl.to(
|
||||
counter,
|
||||
{
|
||||
v: target,
|
||||
duration: 1.5,
|
||||
ease: "power3.out",
|
||||
onUpdate: () => {
|
||||
el.textContent = Math.round(counter.v).toLocaleString();
|
||||
},
|
||||
},
|
||||
0,
|
||||
);
|
||||
```
|
||||
|
||||
`power3.out` matches `1 - (1-t)^3` exactly. Validated in T3 (mean SSIM 0.953).
|
||||
Per-frame digit mismatches occur on sub-frame timing offsets but final values
|
||||
converge — no SSIM impact above the noise floor.
|
||||
|
||||
## Stagger via per-instance prop
|
||||
|
||||
When custom subcomponents take a `delayInFrames` prop:
|
||||
|
||||
```tsx
|
||||
<StatCard delayInFrames={i * 12} value={...} />
|
||||
```
|
||||
|
||||
Translate to GSAP timeline offsets:
|
||||
|
||||
```js
|
||||
cards.forEach((card, i) => {
|
||||
const start = base + i * (12 / fps); // i * 0.4s at fps=30
|
||||
tl.to(card, { ... }, start);
|
||||
});
|
||||
```
|
||||
|
||||
Validated in T3 — three StatCards staggered at 0.0/0.4/0.8 s.
|
||||
@@ -0,0 +1,114 @@
|
||||
# Transitions translation: @remotion/transitions → HF crossfades / shader-transitions
|
||||
|
||||
The `@remotion/transitions` package is Remotion's library of pre-built
|
||||
scene-to-scene transitions. HF has two paths to translate them:
|
||||
|
||||
1. **Manual GSAP crossfade** — for simple opacity/transform transitions. Free, no extra package.
|
||||
2. **HF shader-transitions package** — for visually-rich transitions that match the @remotion/transitions presets.
|
||||
|
||||
## Pattern: `<TransitionSeries>` is `<Series>` with overlap
|
||||
|
||||
```tsx
|
||||
<TransitionSeries>
|
||||
<TransitionSeries.Sequence durationInFrames={60}>
|
||||
<SceneA />
|
||||
</TransitionSeries.Sequence>
|
||||
<TransitionSeries.Transition
|
||||
presentation={fade()}
|
||||
timing={linearTiming({ durationInFrames: 15 })}
|
||||
/>
|
||||
<TransitionSeries.Sequence durationInFrames={60}>
|
||||
<SceneB />
|
||||
</TransitionSeries.Sequence>
|
||||
</TransitionSeries>
|
||||
```
|
||||
|
||||
Translates to scenes that overlap by the transition duration:
|
||||
|
||||
- SceneA: [0, 60] = `data-start="0" data-duration="2"`
|
||||
- SceneB: [60-15, 60-15+60] = `data-start="1.5" data-duration="2"` (the transition window overlaps the end of A and start of B)
|
||||
|
||||
Then drive the transition with GSAP:
|
||||
|
||||
```js
|
||||
// Manual fade (presentation={fade()})
|
||||
tl.to(sceneA, { opacity: 0, duration: 0.5, ease: "none" }, 1.5);
|
||||
tl.fromTo(sceneB, { opacity: 0 }, { opacity: 1, duration: 0.5, ease: "none" }, 1.5);
|
||||
```
|
||||
|
||||
## Presentation table
|
||||
|
||||
| Remotion `presentation` | HF translation |
|
||||
| ---------------------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| `fade()` | manual `gsap.to(opacity)` crossfade |
|
||||
| `slide({direction: "from-right"})` | `gsap.fromTo(translateX: "100%" → 0)` on incoming + `to(translateX: "-100%")` on outgoing |
|
||||
| `wipe({direction: "from-left"})` | `gsap.fromTo(clip-path: inset(0 100% 0 0) → inset(0 0 0 0))` on incoming |
|
||||
| `clockWipe()` | use HF's `sdf-iris` shader-transition (`npx hyperframes add sdf-iris`) |
|
||||
| `flip()` | `gsap.to(rotateY)` 180° split between scenes |
|
||||
| `cube()` | use HF's `cinematic-zoom` or build manually with `rotateY` + `transform-origin` |
|
||||
| `iris()` | use HF's `sdf-iris` shader-transition |
|
||||
| `none()` | no transition; hard cut at the boundary |
|
||||
|
||||
## Timing translations
|
||||
|
||||
```tsx
|
||||
linearTiming({durationInFrames: 15}) → ease: "none"
|
||||
linearTiming({durationInFrames: 15, easing: ...}) → ease per the easing table in timing.md
|
||||
springTiming({config: {damping: 12}}) → ease: "back.out(1.4)" (~0.7 s)
|
||||
```
|
||||
|
||||
Convert `durationInFrames` to seconds (`/fps`).
|
||||
|
||||
## When to use HF shader-transitions
|
||||
|
||||
For transitions Remotion presets that have visually-rich GLSL equivalents
|
||||
(iris, ripple, zoom, glitch), use HF's [shader-transitions](https://hyperframes.heygen.com/catalog/blocks)
|
||||
package. They produce richer output than manual GSAP transforms.
|
||||
|
||||
```bash
|
||||
npx hyperframes add sdf-iris
|
||||
```
|
||||
|
||||
Then in the composition:
|
||||
|
||||
```html
|
||||
<div id="iris-transition" class="hf-shader-transition" data-start="1.5" data-duration="0.5">
|
||||
<!-- bound scenes via the shader-transition's data-from / data-to -->
|
||||
</div>
|
||||
```
|
||||
|
||||
Each shader-transition has its own data attributes; see the catalog page
|
||||
for the specific block.
|
||||
|
||||
## When the source uses a custom Presentation
|
||||
|
||||
Remotion supports custom `presentation` implementations:
|
||||
|
||||
```tsx
|
||||
const customPresentation: PresentationComponent = ({
|
||||
children,
|
||||
presentationProgress,
|
||||
presentationDirection,
|
||||
}) => {
|
||||
return (
|
||||
<div
|
||||
style={
|
||||
{
|
||||
/* compute transform from progress */
|
||||
}
|
||||
}
|
||||
>
|
||||
{children}
|
||||
</div>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
Translation: extract the math from the `style={...}` block and emit
|
||||
equivalent GSAP tweens. Specifically the transform formula maps directly
|
||||
to a `gsap.to(target, { transform: ... })` parameterized by `progress`.
|
||||
|
||||
If the custom presentation uses `useCurrentFrame()` internally to
|
||||
animate something _outside_ the simple progress curve, treat the source
|
||||
as untranslatable and bow out to the runtime interop pattern (see
|
||||
[escape-hatch.md](escape-hatch.md)).
|
||||
Reference in New Issue
Block a user