wehub/nexu-io--html-video
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 12:30:22 +08:00
commit 36fb403ae1
281 changed files with 80074 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+67
View File
@@ -0,0 +1,67 @@
# macOS
.DS_Store
._*
# Editor
.vscode/
.idea/
*.swp
*.swo
# Node
node_modules/
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
.pnpm-debug.log*
.npm
.yarn-integrity
package-lock.json
yarn.lock
pnpm-lock.yaml
.pnp.*
# Python
__pycache__/
*.py[cod]
*$py.class
.Python
.venv/
venv/
*.egg-info/
# Build / dist
dist/
build/
out/
.next/
.cache/
.parcel-cache/
*.tsbuildinfo
# Env / secrets
.env
.env.*
!.env.example
*.pem
# Wrangler / Cloudflare
.wrangler/
.dev.vars
# Local-only working files
scratch/
tmp/
# Per-user runtime state (projects, assets, preview, MP4 outputs)
# This dir is created by the CLI / studio at runtime and is per-developer.
.html-video/
# Generated video output (large binaries)
*.mp4
*.webm
*.mov
*.gif
output/
renders/
.npmrc
+4
View File
@@ -0,0 +1,4 @@
auto-install-peers=true
strict-peer-dependencies=false
shamefully-hoist=false
prefer-workspace-packages=true
ATTRIBUTIONS.md
+65
View File
@@ -0,0 +1,65 @@
# Attributions
html-video is licensed under Apache-2.0. Some of its bundled video **templates** are
derived from open-source design skills under permissive licenses. This file records every
upstream we built on, its real copyright holder, and its license, as required by the
upstream licenses and by [RFC-07](research/2026-06-04-spec-07-ppt-to-template.md) (our
PPT-skill → template conversion standard).
> **What "derived" means here**: we transformed static presentation/slide designs into
> animated [hyperframes](https://github.com/heygen-com/hyperframes) video templates —
> rebuilt as original CSS/SVG keyframe timelines, re-coloured/re-typed where noted, with
> our own sample data. No upstream source code is copied verbatim. The upstream licenses
> (MIT) permit commercial use and derivative works; we attribute them regardless.
Many of these design styles are themselves homages to real-world design studios. Where a
template is inspired by a specific studio, we record that as the **L1 origin** in the
template's `provenance` block — as a factual statement of inspiration, never as a claim of
affiliation or endorsement. Studio names are **not** used as template names.
---
## Upstream design skills
### huashu-design
- **Copyright holder**: alchaincyf (花叔 · 花生) © 2026
- **License**: MIT
- **Source**: https://github.com/alchaincyf/huashu-design
- **Used by templates**:
- `frame-pentagram-stat` — via `assets/showcases/ppt/ppt-pentagram.html`
(L1 inspiration: Pentagram / Michael Bierut)
- `frame-build-minimal` — via `assets/showcases/ppt/ppt-build.html`
(L1 inspiration: Build, London design studio)
- `frame-takram-organic` — via `assets/showcases/ppt/ppt-takram.html`
(L1 inspiration: Takram, Japanese design firm)
### frontend-slides
- **Copyright holder**: Zara Zhang © 2025
- **License**: MIT
- **Source**: https://github.com/zarazhangrui/frontend-slides
- **Used by templates**:
- `frame-bold-signal` — via `STYLE_PRESETS.md` (preset "Bold Signal")
- `frame-creative-voltage` — via `STYLE_PRESETS.md` (preset "Creative Voltage")
- `frame-electric-studio` — via `STYLE_PRESETS.md` (preset "Electric Studio")
(These three are original presets composed by the skill author — no specific real-world
studio is the L1 origin.)
---
## Studio names referenced as L1 inspiration
The following real-world design studios / designers are named in template `provenance.origin`
purely as a record of stylistic inspiration. html-video is **not affiliated with, endorsed by,
or sponsored by** any of them.
- **Pentagram** (and Michael Bierut) — pentagram.com
- **Build** — London design studio
- **Takram** — takram.com
---
*Per-template details (the full three-layer `provenance` block: origin / via_skill /
transformation) live in each template's `template.html-video.yaml`.*
CLAUDE.md
+223
View File
@@ -0,0 +1,223 @@
# html-video 项目工作区
> Open-source HTML→Video meta-layer。让本地 coding agent 跨多个渲染 engine（Hyperframes / Remotion / Motion Canvas / Revideo）一站式做 HTML 视频。
> 启动时间：2026-05-26（继 OD / HA / OD-pitch / OD-landing 之后的下一代 nexu-io 开源产品）
## 角色与边界
- **角色**：T9（端口段 **3071-3079**）—— 现有 T1-T8 已分配
- **职责**：html-video 项目的产品定位 / 架构 / engine 适配器 spec / 模板生态 / agent skill 设计 / 文档 / 公开 launch 物料
- **不做**：
- 跨项目协调（T1 主控的活）
- OD 主线开发（T5 `open-design` 工作树）
- 增长策略 / OKR（T2 `opendesign-growth`）
## 产品定位（决策时间线）
- **2026-05-26 启动思路**：Joey 最初以为 Hyperframes 是闭源 SaaS，想做 "HTML 视频开源整合工具"。调研发现 Hyperframes 已经是 Apache-2.0 开源 + 21K★ + agent-native，跟 Joey 设想的产品 1:1 重合。
- **2026-05-26 定位拍板**：放弃"做 HF 杀手"的正面竞争路线，改走 **Meta-aggregator** —— 把 HF / Remotion / Motion Canvas / Revideo 都包成可选 backend，agent 选 engine + 模板 + 一键出片。差异化明确，跟 HF 不正面撞。
### 关键差异化 vs Hyperframes
| 维度 | Hyperframes | html-video |
|---|---|---|
| 渲染引擎 | 单一（GSAP+Puppeteer） | 多引擎 pluggable |
| Authoring 范式 | 单一（HTML+CSS+GSAP） | 跟随 backend（HTML/React/TS-generator 都行） |
| Agent 能力 | 自家 skill | 跨引擎 agent 决策 + skill 注入 |
| 模板生态 | 自家曲库 | 跨生态 curated + 社区贡献 |
## 目录结构
```
html-video/
├── README.md 公开门面（对外，已立）
├── CLAUDE.md 本文件，内部 working notes
├── LICENSE Apache-2.0
├── .gitignore 通用
├── notes/ 迭代笔记 / RFC 草稿 / 决策记录
├── research/ 竞品调研 / engine spec 草案 / API 探索
└── assets/ 素材（截图 / 草图 / launch 物料）
```
代码骨架（adapter spec、CLI、studio 等）等架构定型后另起 `core/` `adapters/` `cli/` `studio/` 等目录。
## 当前状态（2026-05-28 v0.8 content-graph + multi-frame）
v0.7 之前的进展见 git log。本节记 v0.8 落地的事 + 还没接的事。
- ✅ `@html-video/content-graph` 新 package：ContentGraph schema（entity/data/text 节点 + sequence/dependency/contrast 边）+ `validate` + `topoSort`（dependency 硬约束 / sequence 软排 / contrast 不参与排序）+ `totalDurationSec`
- ✅ core：`Project` 加 `contentGraphPath` + `frames[]`；`orchestrator` 加 `writeContentGraph` / `readContentGraph` / `writeFrameHtml`；`exportMp4` 多帧路径走 ffmpeg concat（缺 ffmpeg 时报友好错）
- ✅ studio agent：prompt 同时教单帧 fast path + 多帧（\`\`\`json#content-graph + \`\`\`html#<nodeId> 双块协议）；server 解析后自动调 writeContentGraph + writeFrameHtml；无 graph 时回落 v0.7
- ✅ studio UI：frames-strip 切帧 + graph viewer modal（JSON 只读 + 下载）；单帧 fast path 时整条隐藏
- ✅ 已 push `nexu-io/html-video` main（commit `149ace9`）
- ⏳ HF upstream 真实 render（v0.9?）：adapter-hyperframes 仍是 stub，单帧/多帧 export MP4 都是空文件
- ⏳ ffmpeg concat 真实跑通：依赖 ffmpeg 装好；adapter render 出真 MP4 之后才 end-to-end 有用
- ⏳ RFC-06 正式文档稿（落 `research/2026-05-28-spec-06-content-graph.md`）；目前规范散在 content-graph package README + smoke + agent prompt 三处
- ⏳ studio 的 graph viewer 现在只读，未来可加可视化编辑（拖节点 / 加边）
## 会话进度（2026-06-07）— Remotion 原生数据模板 + 按帧增强（Phase A+B 跑通，**git 待 Joey 定**）
> 起点：Joey 修正定位——**Remotion 是用户主动按帧挂的"动效增强插件"，hyperframes 是基座，AI 退到建议**（推翻 RFC-09 v0.1 的"AI 自动选引擎/引擎名隐形"）。本轮做 RFC-08 **Phase 2 原生 tsx 数据动画** 的第一刀：CLI 层跑通混流 MP4。在 `feat/remotion-adapter` 分支（HEAD 原 `633a3dd` = Phase 1 桥接）。
**做完的（全部端到端真实渲染 + 量像素验证，非 mock）：**
- ✅ 类型层（`packages/core/src/types/index.ts`）：`TemplateRef` 加 `mode?:'bridge'|'native'` + `nativeCompositionId?`；`FrameRecord` 加 `engine?`/`nativeTemplateId?`/`data?`；`TemplateMetadata` 加 `native?:{compositionId}`。
- ✅ 首个**原生 Remotion 模板** `templates/frame-data-rollup/`：`DataRollup.tsx`（柱子 `spring()` 生长 + 数字 `interpolate()` 0→目标滚动）+ `Root.tsx`（honor inputProps.width/height）+ `entry.ts` + yaml（`engine:remotion` / `native.compositionId:DataRollup` / provenance origin=none 原创）+ SKILL/example/package/preview.png。纯系统字体栈避黑屏。
- ✅ `adapter-remotion/src/render.ts`：抽 `renderComposition()` helper；加 `mode==='native'` 分支（bundle 模板自己 entry / select compositionId / inputProps={data,width,height}、不跑 neutralize）；`bundleCache` 单条 → `Map<entryPath,serveUrl>`（bridge+native 共存、原生模板一次 bundle 跨多帧复用）。
- ✅ `core/src/project.ts`：export loop 按帧解析引擎（`resolveFrameTemplateRef`）；`enhanceFrameNative(projectId,nodeId,nativeTemplateId)`（断言 kind=data、`normalizeRollupData` 归一 node.data、设三字段）+ `unenhanceFrame`（清三字段，htmlPath 不动=非破坏性）；`concatFramesWithFfmpeg` 加 `reencode` 参数。
- ✅ 文档纠正：RFC-09 升 v0.2（定位改"基座+主动增强+AI 建议"）；旧 notes `2026-06-06-remotion-user-experience-design.md` 顶部加作废声明指向 v0.2。
**Phase A 验收（单原生帧 vs 桥接对照，量像素）**：原生 native.mp4 非黑像素 **1.5%→8.8%→16.9%**（柱子在长）vs 桥接静态图 **22.6%×3 恒定**；同时刻 t=0.5 两者差 **56.7万像素**。铁证"原生数据动画 ≠ 桥接静态图"。6.2s 出 120 帧 1080p。
**Phase B 验收（真实 orchestrator 3 帧混流）**：intro(hyperframes 蓝)→ middle(`enhanceFrameNative`→原生 Remotion 数据柱)→ outro(hyperframes 酒红)，`exportMp4`→一条 **8.000s** MP4，全解码无错、单流、三段视觉各异。
**关键教训（新会话必读）：**
- **混流 concat 不能用 concat demuxer + 重编码**：hyperframes 与 Remotion 段的 timebase/PTS 不兼容，concat demuxer 会**错误累加时间戳→8s 片变 35s**（`-vsync cfr` 也救不了，因为坏 PTS 已被喂进来）。**正解 = concat FILTER**：每段独立 `-i` + `-filter_complex "[0:v][1:v]...concat=n=N:v=1[v]"` 重建时间轴 → 精确 8s。单引擎仍走 demuxer `-c copy`（快、无损）。判混流类输出**必须 ffprobe 量时长 + `ffmpeg -f null -` 全解码**，不能只看文件生成。
- **Remotion 能 bundle 模板自己的 `source/entry.ts`**（templates/ 下无独立 node_modules），webpack 能 resolve 到 workspace 根的 react/remotion——最大未知，已验证可行。原生模板做成 `templates/` 一等模板（不塞进 adapter 内部）这条路通。
- **本机仍无 timeout 命令**（macOS）；PIL 仍装不上（brew Python expat bug），量像素继续用 ImageMagick `magick ... -threshold -format '%[fx:100*mean]'`。
**待 Joey 定（git 写操作，未执行）**：① 本轮改动开 PR（按惯例对外可见/新模板走 PR review）还是直推；② 上一会话(6/6)遗留的 CLAUDE.md 39 行未提交记录怎么处理（CLAUDE.md 是追踪文件但属内部 notes，不宜混进代码 PR）。临时 driver 在 `~/Desktop/claude-code/scratch/hv-remotion-native/`（不进库）。
## 会话进度（2026-06-06）— 渲染 + studio 一连串修复，**4 个 PR 全部 merged 进 main**
> 任务起点："继续修字体跳闪"。顺着 Joey 的逐轮实测，从渲染引擎挖到 studio agent 交互/生成层，
> 修了 4 个 PR 全部合并。**全部端到端实调 Anthropic API 验证（非 mock）。** main 顶 = `1a83279`。
| PR | 修了什么 | merge commit |
|---|---|---|
| [#22](https://github.com/nexu-io/html-video/pull/22) | 字体跳闪（FOUT）+ 多 composition 完整播放 | `0029840` |
| [#23](https://github.com/nexu-io/html-video/pull/23) | 多帧改内容重生成整片 graph + 主题纠偏 + confirm 软校验 | `bf4b215`(merge `e340af1`) |
| [#24](https://github.com/nexu-io/html-video/pull/24) | 主题从开场原话贯穿到生成（"随机"不跑题） | `a9c5325` |
| [#25](https://github.com/nexu-io/html-video/pull/25) | 生成后迭代子状态机 + 每帧时长硬约束 + 自由发挥推进 | `1a83279` |
**改动文件**（都在 main）：`packages/adapter-hyperframes/src/render.ts`（字体冻结、时长 explicit）、
`packages/cli/src/studio-server.ts`（主题锁定、生成后迭代）、`packages/core/src/types/index.ts`+`project.ts`（durationMode）。
**#22 字体跳闪真根因（两轮才修对）**：① 第一版只加 `document.fonts.ready` 等待——不够，因为纯 CSS
`@keyframes` 动画在 `goto` 后**不等字体就自动跑**，等字体的 ~3s 连同 FOUT 一起被 playwright 录进开头，
单文件模板还不裁死区。② 正解：`addInitScript` 注入 `* {animation-play-state:paused}` 在文档解析前**冻结所有动画**
→ 等 stylesheet link + `fonts.load()` 每个 face + `fonts.ready` → 解冻并把冻结期当 leadInMs 裁掉。统一了单文件 + 多 composition 时序。
对 CSS/rAF/setTimeout 三类驱动都验证首帧字体正确（`fonts.check()=true`）。
**#25 生成后迭代（Joey 反复反馈"后面指令全失效"的核心）**：根因 = `detectPhase` L1793 `hadGenerationYet()=true`
把**生成后任何消息硬判成 iterate**，iterate 只做单帧 preview.html 重写 → 假"✓ updated"。
正解 = 卡片驱动子状态机：pin 帧→单帧编辑；明确说改风格/模板/时长/内容→直达对应卡；**其余一切（含模糊"改一下"/
"换个模板重新生成"）→ 默认弹 edit-menu 问清楚**。换风格走 restyle（复用现有 graph 文案、跳过重规划、仅按新风格逐帧重渲）。
### 关键教训（新会话必读）
- **字体跳闪只在导出 mp4 复现**（后端冷 chromium 录屏）；**studio iframe 实时预览有字体缓存，看不出**。验证载体只能是导出的 mp4，量早期帧（t≈0.1s）字体形态。本机 PIL 装不上（brew Python expat bug），量像素用 ImageMagick；颜色验证用 computed-style 而非截图（Playwright 截图在动画站超时）。
- **studio agent 这种"判用户意图"的逻辑，别用白名单正则判触发**——"换个模板重新生成一下"就因不在白名单掉进单帧重写黑洞。正确思路是**反转默认**：默认走交互/重生成，只有明确的少数情况（pin 帧）才单帧。判 render/交互类修复**必须端到端实跑**，不能信"tsc 过 + 逻辑看着对"。
- **`project.intent` 字段前端创建时不传**（只发 name），恢复开场主题靠 `history.find(role==='user')` 第一条。
- **per_frame 时长**：render.ts 探测动画长度会无条件 extend 覆盖用户设定（4s→29.7s）。修法 = `RenderConfig.durationMode='explicit'`（多帧 export 传），explicit 时不 extend，ffmpeg `tpad=stop_mode=clone` 补尾帧到精确时长；单帧快速预览仍 'auto' 照常 extend。
- **PR 合并流程坑（2026-06-06 踩过）**：PR #23 在我追加第二个 commit **之前**就被合并了（merge commit，非 squash，疑 Joey 手点或自动合），导致那个 commit 落空 → 只能 cherry-pick 到新分支另开 #24 补。教训：push 追加 commit 前先确认 PR 还 OPEN。Joey 偏好 **squash 合并**。
**会话尾已清理**：4 个 `fix/*` 分支（本地+远端）已删；本地 studio 3071/3075 + 画廊 3072 已关。
诊断备份在 `notes/2026-06-06-studio-session-verify-bold/`（未追踪，含原始 messages/graph + DIAGNOSIS.md）。
**仍未追踪不进版本库**（Joey 未定）：`research/render-all-previews.sh` / `repick-posters.sh` / `preview-renders/` / `preview-renders.old-*/` / spec-08/09 md / 上述 notes 目录。
## 会话进度（2026-06-05）— 新模板 frame-bold-poster，**已做完 + studio 验证，git 待定**
> 上一会话在 `feat/template-bold-poster` 分支做了一个新模板 `frame-bold-poster`（暖白纸底 +
> 番红 + 巨型倾斜 Shrikhand 大字的 1970s 欧洲社论海报风），未提交即中断。本轮接续：核对 + 验证。
- ✅ 接续盘点：当前在 `feat/template-bold-poster` 分支（HEAD = main = origin/main = `cecbe0a`，三者同步）。
唯一在制品 = 未追踪目录 `templates/frame-bold-poster/`（template.html-video.yaml + SKILL.md +
example.md + package.json + preview.png + source/index.html）。**6/4 那轮 provenance 整改其实早已
merge 进 main（commit `db31210`），下面 6/4 段里"待 commit"的说法已过时，勿再据此重复提交。**
- ✅ 核对新模板严格遵循 RFC-07：三层署名齐全 —— origin = "1970s 欧洲社论海报传统"（标 `kind: movement`，
诚实地不伪造具体工作室）/ via_skill = frontend-slides · Zara Zhang · MIT · 指向具体上游
`bold-template-pack/templates/bold-poster/design.md` / transformation 声明为原创 CSS keyframes、
调色与字体栈跟随上游 spec、**未照搬上游 HTML**。
- ✅ 查重：与已有 `frame-bold-signal`（同样 frontend-slides 派生）**不撞车** —— bold-signal 是深色底
`#1a1a1a` + Archivo Black + 橙红 `#FF5722`、源自 `STYLE_PRESETS.md` preset、origin=none；
bold-poster 是暖白 `#F5F2EF` + Shrikhand/Libre Baskerville 衬线 + 番红 `#D8000F`、源自具体上游模板。
- ✅ 验证：① `pnpm -r build` 全过；② CLI `search-templates` 重读磁盘 → 27 模板齐全，bold-poster
排第一（score 0.8）engine_installed=true license OK；③ 起 studio 到 **3071**（旧会话的 3074 还占着、
跑旧状态），`/api/templates` 确认 bold-poster present=true。`prefers-reduced-motion` 终态也处理了。
- ⏳ **git 待定（Joey 6/5 决定先在本地 studio 看效果再说）**：本轮**不动 git**，未 commit / 未 push /
未开 PR。待 Joey 在 studio 看完动效后，再决定 commit + 开 PR（按约定模板/公开内容走 PR review）。
studio 运行中：`http://localhost:3071`（pid 见 `/tmp/hv-studio-3071.log`）。
## 会话进度（2026-06-04）— 模板来源/署名整改，**已完成并 merge 进 main（`db31210`）**
> 起因：Joey 发现新加的模板与已有模板视觉撞车，且"原创"声明未经核实。
> 做了一次全量来源审计 + 定转换规范 + 按规范整改了 6 个模板的署名。
**审计与规范（前半段）：**
- ✅ 回退了今天误加的 3 个重复模板 commit（`abd8168`，曾是 build-metrics / pentagram-benchmark /
takram-radar）。`git reset --hard origin/main` 回到 26 个模板的干净状态，HEAD = `166fdc1`。
→ **结论：这 3 个是已有 huashu 系模板的"+3指标数据"换皮变体，不要再加回来。**
- ✅ 真实 clone 两个上游核实 license（都真是 MIT）+ 记下**真实版权人**：
- huashu-design → `alchaincyf（花叔 · 花生）` © 2026
- frontend-slides → `Zara Zhang` © 2025
- ✅ 关键发现：Pentagram / Build / Takram **不是 huashu 原创**，是 huashu 在致敬真实设计工作室
（Pentagram=Michael Bierut / Build=伦敦工作室 / Takram=日本公司）。署名是**三层**不是两层。
且我们模板示例数据（95.7/73.8/AIME/SWE）是从上游 ppt 页照搬的。
- ✅ 写了审计报告 `notes/2026-06-04-provenance-audit.md`（未提交）
- ✅ 写了转换规范 **`research/2026-06-04-spec-07-ppt-to-template.md`（RFC-07，未提交）** —— 含三层
署名 schema（origin/via_skill/transformation）+ 命名规范 + 转换质量门槛 + 查重 + 交付清单。
Joey 已认可此规范，后续按它走。
- ✅ 决定：**"默认渲染效果"不另做 loop.mp4**。studio 预览弹窗已用 `mode:'iframe'` 实时跑动画，
用户在 studio 能直接看动效；preview.png 只做 studio 之外（README/官网）的静态兜底。
**整改（后半段，本轮做完）—— 6 个模板按 RFC-07 补三层署名，只补署名不改名：**
- ✅ 6 个模板的扁平 `provenance.inspired_by` 全部换成三层结构
`origin`（L1 真实工作室）/ `via_skill`（L2 skill+真实作者全名+license+具体 source_file）/ `transformation`：
- huashu 系 source_file = `assets/showcases/ppt/ppt-{pentagram,build,takram}.html`，
origin 分别 = Pentagram(Michael Bierut) / Build(伦敦工作室) / Takram(日本公司)。
- frontend-slides 系 source_file = `STYLE_PRESETS.md`（preset "Bold Signal"/"Creative Voltage"/
"Electric Studio"），**origin 诚实标 `none`** —— 这三个是 skill 作者自组的原创 preset，没特定 L1 工作室。
- via_skill.author 填真实全名：`alchaincyf (花叔 · 花生)` / `Zara Zhang`（上游 LICENSE 核实过）。
- ✅ 遵守 Joey 决定（6/4）：本轮**只补 provenance 三层 + 真实作者名，未改 id/显示名/示例数据**
（命名挪用工作室名的问题留下一轮）。
- ✅ 新建根 `ATTRIBUTIONS.md` 汇总两个上游 + license + 每个模板的 L1/L2 映射 + "not affiliated" 声明。
- ✅ 验证：① 用项目 `yaml@2.9.0` 直接解析 6 文件 → 三层字段齐全、无 `inspired_by` 残留、语法 OK；
② 全新 CLI 进程 `search-templates` 重读磁盘 → 6 个模板全部正常加载（license/best_for/duration 都对）；
③ studio(localhost:3074) `/api/templates` 24 模板齐全。注：provenance **不进** API / UI 渲染路径
（studio 只用 source HTML + inputs schema + poster），本轮没碰这些，渲染逻辑上不受影响。
**待办（下次接）：**
- ~~commit + push~~ → **已于 6/4 完成并 merge 进 main（`db31210`）**，此条作废。
- ⏳ 下一轮（独立）：按 RFC-07 ③ 给 6 个模板**重命名**（去掉挪用的工作室名，改设计特征中性名，
如 `frame-editorial-anchor` / `frame-luxe-minimal` / `frame-soft-radar`），涉及改 id 要同步 source 目录名 + 引用。
> 笔误备忘：本项目正确写法是 **hyperframes**（不是 hiframes/HiFrames），文档里如再见到要改。
> 上游 clone 仍在 `~/Desktop/claude-code/scratch/upstream-check/`（huashu-design + frontend-slides，可能被定期清理）。
## 跑起来
```bash
cd ~/Desktop/claude-code/projects/html-video
pnpm install
pnpm -r build
pnpm --filter @html-video/cli smoke
# CLI 直接跑
./packages/cli/dist/bin.js doctor
./packages/cli/dist/bin.js search-templates --intent "github stars" --top 3
```
## 与姊妹项目的关系
| 项目 | 角色 | 关系 |
|---|---|---|
| `open-design` (T5) | OD 主线 maintainer 工作树 | 设计资产侧；本项目是视频侧；同 nexu-io org，可以共享部分 skill 基建 |
| `html-anything` | OD 的 sister 开源（HTML 模板/页面） | 命名同 family（"HTML X"），定位互补：HA 做 HTML 静态页、本项目做 HTML 视频 |
| `od-pitch` (T6) | 路演 deck | 启动时如果做 launch deck，可以复用 od-pitch 的 magazine/recruiting 同款风格 |
| `growth-dashboard` (T7) | 数据罗盘 | 上线后增加 html-video 的 stars/forks/issues 监控 |
## 写操作守则
- 任何对外 publish 动作（CF Pages / GitHub repo create / 推 main / launch tweet / 公众号）**每次都先告诉 Joey**
- 改动 README.md 这种"公开门面"前先 review，避免误更新对外定位
- 第一次 push 到 nexu-io/html-video 是高敏感动作，要 Joey 明确点头 + 选好 launch 时机
## 命名 & 标语备忘
- 项目名：`html-video`
- 公司名 / org: `nexu-io`
- 推荐 tagline 候选：
- "HTML→Video meta-layer for coding agents"（直白）
- "Bring your agent. Pick any engine. Render any video."（口号化）
- "One agent, every HTML video engine."（最短）
- 选哪个等公开发布前定
CONTRIBUTING.md
+502
View File
@@ -0,0 +1,502 @@
# Contributing to html-video
> **English** · [中文](#中文)
Thank you for your interest in contributing! html-video is an Apache-2.0 project by the [Open Design](https://github.com/nexu-io/open-design) team. We welcome contributions of all kinds — code, docs, templates, bug reports, and ideas.
---
## Community
- **Discord**: [Join the Open Design Discord](https://github.com/nexu-io/open-design#community) — the main hub for questions, design discussions, and real-time help.
- **X (Twitter)**: Follow [@nexudotio](https://x.com/nexudotio) for project updates.
- **GitHub Issues**: Bug reports, feature requests, and template proposals all go here.
---
## Development Setup
### Prerequisites
| Requirement | Minimum Version | How to check |
|---|---|---|
| **Node.js** | 20+ | `node --version` |
| **pnpm** | 9+ | `pnpm --version` |
| **ffmpeg** | Any recent | `ffmpeg -version` |
| **Chromium** (or Playwright browsers) | See below | `npx playwright install chromium` |
**Why Chromium?** The default [Hyperframes](https://github.com/heygen-com/hyperframes) engine renders videos by recording animated HTML in a headless Chromium browser. You need either:
- A system Chromium/Chrome install (auto-detected), or
- Playwright's bundled Chromium: `npx playwright install chromium`
**Why ffmpeg?** After recording each frame as WebM, ffmpeg encodes them to MP4 (libx264) and concatenates them into the final video. It's also used for the optional AI soundtrack mixing.
### Clone, Install, Build
```bash
git clone https://github.com/nexu-io/html-video.git
cd html-video
pnpm install
pnpm -r build
```
This is a **pnpm workspace monorepo**. All packages live under `packages/` and templates under `templates/`. The `pnpm -r build` command builds every package in dependency order.
### Run Locally
**Studio (browser UI):**
```bash
node packages/cli/dist/bin.js studio
# Opens at http://127.0.0.1:3071
```
**CLI tools:**
```bash
# Check what's installed and ready
node packages/cli/dist/bin.js doctor
# Search templates by intent
node packages/cli/dist/bin.js search-templates --intent "data chart" --top 5
```
---
## Project Structure
```
packages/
├── core/ Types, registries, orchestrator, MiniMax + ffmpeg audio
├── content-graph/ Multi-frame storyboard IR (nodes + edges, topo-sort)
├── runtime/ Agent runtime — detect / spawn / stream (13 agents)
├── adapter-hyperframes/ Hyperframes engine adapter — Chromium + ffmpeg render
├── cli/ `html-video` command + studio HTTP server + source fetching
└── project-studio/ Browser studio UI (chat, gallery, frames, soundtrack, export)
templates/ 21 curated, license-clean video templates
research/ RFCs (engine adapter / template metadata / agent skill / content-graph)
```
---
## How to Add a New Agent Runtime
Adding support for a new coding agent is the most common contribution. It's a self-contained change in `packages/runtime/`.
**Pattern (from Trae CLI PR #12):**
### 1. Create the agent definition file
`packages/runtime/src/defs/<agent>.ts`:
```ts
import type { AgentDef } from '../types.js';
export const myAgent: AgentDef = {
id: 'my-agent', // kebab-case, stable
name: 'My Agent', // Human-readable
bin: 'my-agent-cli', // CLI binary name (looked up on PATH)
versionArgs: ['--version'], // Args to check version (for `doctor`)
buildArgs(prompt, ctx) { // Build spawn arguments
return ['--print', prompt];
},
streamFormat: 'plain', // 'plain' | 'claude-stream' | 'json-event-stream' | 'acp-json-rpc'
promptViaStdin: false, // true if sending prompt via stdin
installUrl: 'https://example.com/install', // Where users can get it
};
```
See `packages/runtime/src/types.ts` for the full `AgentDef` interface — it supports ACP JSON-RPC agents, HTTP-based agents (like the Anthropic API), binary fallbacks, and extra availability checks.
### 2. Register the agent
In `packages/runtime/src/registry.ts`, import and add your agent to the `AGENT_DEFS` array:
```ts
import { myAgent } from './defs/my-agent.js';
export const AGENT_DEFS: AgentDef[] = [
// ... existing agents
myAgent,
];
```
Order matters: the first available agent is the default selection in the studio.
### 3. Test
```bash
pnpm --filter @html-video/runtime build
node packages/cli/dist/bin.js doctor # Your agent should appear if its binary is on PATH
```
---
## How to Add a New Template
Templates live under `templates/<id>/` and are described by a `template.html-video.yaml` manifest. The studio scans templates at startup and the agent reads the manifest to understand what the template does and what inputs it needs.
### Minimum structure
```
templates/frame-my-cool-animation/
├── template.html-video.yaml # Required — see format below
├── source/index.html # Required — the animated HTML (Hyperframes engine)
├── SKILL.md # Agent-readable instructions for filling in the template
├── example.md # Example input
└── poster.svg / preview.png # Static preview image
```
### Provenance rules (RFC-07)
Every template MUST follow the [RFC-07 provenance rules](research/2026-06-04-spec-07-ppt-to-template.md):
1. **License gate**: Only permissive open-source licenses (MIT, Apache-2.0, BSD, CC-BY, CC-BY-SA). No NC, ND, or unlicensed sources.
2. **Three-layer attribution**: L1 (original design studio/designer) → L2 (skill/upstream author) → L3 (our transformation). All three must be recorded in `provenance`.
3. **Naming**: Use descriptive feature names, not studio/designer names. ❌ `frame-pentagram-stat` → ✅ `frame-editorial-anchor`
4. **Transformation quality**: Must add real animation timeline, use own sample data, and have identifiable redesign from the upstream source.
5. **Deduplication**: Check against existing templates from the same upstream. Don't ship near-identical variants.
### manifest.yaml skeleton
```yaml
spec_version: 1
id: frame-my-cool-animation
name: My Cool Animation
description: A short description for agents and the gallery.
engine: hyperframes
category: title-card
tags: [animation, reveal]
best_for:
- "Product launch teasers"
- "Social media shorts"
inputs:
schema:
type: object
required: [title]
properties:
title: { type: string, description: "Main headline" }
subtitle: { type: string, description: "Subtitle line" }
examples:
- title: "Hello World"
subtitle: "This is an example"
output:
formats: [mp4]
default_format: mp4
duration: { type: variable, min_sec: 3, max_sec: 15 }
license:
spdx: Apache-2.0
attribution_required: false
redistribution_allowed: true
commercial_use: true
provenance:
origin:
name: "Original Designer / Studio"
kind: design-studio
reference: "https://example.com"
via_skill:
name: upstream-skill-name
author: "Author Name"
url: https://github.com/author/upstream
license: MIT
source_file: path/to/source.html
transformation: >
Static design → animated Hyperframes timeline with CSS @keyframes.
Re-colored, original sample data.
```
See [RFC-02](research/2026-05-26-spec-02-template-metadata.md) for the complete `template.html-video.yaml` specification.
---
## How to Add a New Engine Adapter
The engine adapter interface ([RFC-01](research/2026-05-26-spec-01-engine-adapter.md)) lets any video rendering backend plug into html-video. The shipped adapter is `@html-video/adapter-hyperframes` — use it as the reference implementation.
### 1. Create a new package
```
packages/adapter-<engine>/
├── package.json
├── src/
│ ├── index.ts # Export default EngineAdapter instance
│ ├── capabilities.ts # Static capability declaration
│ ├── validate.ts # Validate a template for this engine
│ └── render.ts # Core render implementation
└── tsconfig.json
```
### 2. Implement the EngineAdapter interface
The core contract is defined in `packages/core/src/types.ts`:
```ts
export interface EngineAdapter {
id: EngineId;
name: string;
upstreamVersion: string;
capabilities: EngineCapabilities;
validate(template: Template): ValidationResult;
render(input: RenderInput, ctx: RenderContext): Promise<RenderOutput>;
preview?(template: Template, ctx: PreviewContext): Promise<PreviewHandle>;
listNativeTemplates?(): Promise<NativeTemplateRef[]>;
}
```
Key conventions from RFC-01:
- **Process isolation**: Each `render()` spawns an independent subprocess. Subprocess crashes must reject the promise and leave no partial output files.
- **Progress reporting**: 0-100% based on current frame / total frames. Stage hints: `preparing` (0-10%), `rendering` (10-95%), `muxing` (95-100%).
- **Cancellation**: Respect `ctx.signal.aborted` — kill subprocess, cleanup workDir temp files, reject with `AbortError`.
- **Package naming**: `@html-video/adapter-<name>`, peer-depend on the upstream engine.
### 3. Register in core
The core dynamically loads adapters — once your package is in `packages/` and listed in `pnpm-workspace.yaml`, it will be auto-discovered at runtime.
---
## Code Style
- **Language**: TypeScript (strict mode, `tsconfig.base.json` inheritance)
- **Formatting**: [Biome](https://biomejs.dev/) — 2 spaces, single quotes, trailing commas, semicolons, LF line endings. Run `pnpm format` to auto-format.
- **Linting**: `pnpm lint` runs Biome linter with recommended rules.
- **Monorepo tooling**: pnpm workspaces, `pnpm -r build` builds all packages in order.
- **Imports**: Use `.js` extensions in TypeScript imports (for ESM compatibility).
### Before submitting
```bash
pnpm typecheck # TypeScript across all packages
pnpm lint # Biome linter
pnpm format # Auto-format all files
pnpm test # Run all tests
```
---
## How to Submit a Pull Request
1. **Fork** the repository and create a branch from `main`.
2. **Make your changes** — keep them focused. One PR = one logical change.
3. **Test your changes** — run `pnpm typecheck && pnpm lint && pnpm test`.
4. **Write a clear commit message** — follow conventional commits: `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`.
5. **Open a PR** against `nexu-io/html-video:main`.
6. **Describe** what you changed, why, and how to verify it.
We review PRs regularly. If your PR adds a new feature, link to or include related tests.
---
## License
By contributing, you agree that your contributions will be licensed under the [Apache-2.0 License](LICENSE) — same as the rest of the project. No contributor license agreement (CLA) is required.
---
## Questions?
- **Real-time chat**: Join the [Open Design Discord](https://github.com/nexu-io/open-design#community)
- **Bugs & features**: Open an issue on [GitHub](https://github.com/nexu-io/html-video/issues)
- **Design decisions**: Read the RFCs in [`research/`](research/)
---
## 中文 {#中文}
感谢你有意为 html-video 贡献！html-video 是 [Open Design](https://github.com/nexu-io/open-design) 团队维护的 Apache-2.0 项目。我们欢迎所有形式的贡献 —— 代码、文档、模板、bug 反馈和想法。
### 社区
- **Discord**：[加入 Open Design Discord](https://github.com/nexu-io/open-design#community) — 主要的问答、设计讨论和实时求助渠道。
- **X（Twitter）**：关注 [@nexudotio](https://x.com/nexudotio) 获取项目动态。
- **GitHub Issues**：提 bug、功能建议、模板提案都来这里。
### 开发环境搭建
#### 前置依赖
| 依赖 | 最低版本 | 检查方式 |
|---|---|---|
| **Node.js** | 20+ | `node --version` |
| **pnpm** | 9+ | `pnpm --version` |
| **ffmpeg** | 任意较新版本 | `ffmpeg -version` |
| **Chromium**（或 Playwright 浏览器）| 见下文 | `npx playwright install chromium` |
**为什么需要 Chromium？** 默认的 [Hyperframes](https://github.com/heygen-com/hyperframes) 引擎用无头 Chromium 录制带动画的 HTML 来渲染视频。你需要以下之一：
- 系统安装的 Chromium/Chrome（自动检测），或
- Playwright 内置的 Chromium：`npx playwright install chromium`
**为什么需要 ffmpeg？** 渲出每帧 WebM 后，ffmpeg 将它们编码为 MP4（libx264）再拼接成最终视频。可选的 AI 配乐混音也要用到它。
#### 克隆、安装、构建
```bash
git clone https://github.com/nexu-io/html-video.git
cd html-video
pnpm install
pnpm -r build
```
这是 **pnpm workspace 单体仓库**。所有包在 `packages/` 下，模板在 `templates/` 下。`pnpm -r build` 按依赖顺序构建所有包。
#### 本地运行
**Studio（浏览器界面）：**
```bash
node packages/cli/dist/bin.js studio
# 在 http://127.0.0.1:3071 打开
```
**CLI 工具：**
```bash
# 查看已安装并可用的 agent 和引擎
node packages/cli/dist/bin.js doctor
# 按意图搜索模板
node packages/cli/dist/bin.js search-templates --intent "数据图表" --top 5
```
### 如何添加新的 Agent 运行时
添加对新 coding agent 的支持是最常见的贡献类型。改动集中在 `packages/runtime/` 内。
**模式（参考 Trae CLI PR #12）：**
#### 1. 创建 agent 定义文件
`packages/runtime/src/defs/<agent>.ts`：
```ts
import type { AgentDef } from '../types.js';
export const myAgent: AgentDef = {
id: 'my-agent', // kebab-case，稳定不变
name: 'My Agent', // 可读名称
bin: 'my-agent-cli', // CLI 二进制名（在 PATH 上查找）
versionArgs: ['--version'], // 检查版本的参数（给 `doctor` 用）
buildArgs(prompt, ctx) { // 构建启动参数
return ['--print', prompt];
},
streamFormat: 'plain', // 'plain' | 'claude-stream' | 'json-event-stream' | 'acp-json-rpc'
promptViaStdin: false, // 是否通过 stdin 传递 prompt
installUrl: 'https://example.com/install', // 安装指引链接
};
```
完整 `AgentDef` 接口见 `packages/runtime/src/types.ts` —— 它支持 ACP JSON-RPC agent、基于 HTTP 的 agent（如 Anthropic API）、二进制回退路径和额外的可用性检查。
#### 2. 注册 agent
在 `packages/runtime/src/registry.ts` 中 import 并将你的 agent 加入 `AGENT_DEFS` 数组：
```ts
import { myAgent } from './defs/my-agent.js';
export const AGENT_DEFS: AgentDef[] = [
// ... 已有 agent
myAgent,
];
```
顺序很重要：第一个可用的 agent 是 studio 的默认选项。
#### 3. 测试
```bash
pnpm --filter @html-video/runtime build
node packages/cli/dist/bin.js doctor # 如果二进制在 PATH 上，你的 agent 应该出现
```
### 如何添加新模板
模板放在 `templates/<id>/` 下，由 `template.html-video.yaml` 清单描述。studio 启动时扫描模板，agent 读取清单来了解模板用途和输入需求。
#### 最小目录结构
```
templates/frame-my-cool-animation/
├── template.html-video.yaml # 必选 —— 格式见下文
├── source/index.html # 必选 —— 带动画的 HTML（Hyperframes 引擎）
├── SKILL.md # Agent 可读的填参说明
├── example.md # 示例输入
└── poster.svg / preview.png # 静态预览图
```
#### 来源规范（RFC-07）
每个模板必须遵守 [RFC-07 来源规范](research/2026-06-04-spec-07-ppt-to-template.md)：
1. **许可闸门**：只收明确宽松开源的许可（MIT、Apache-2.0、BSD、CC-BY、CC-BY-SA）。不收 NC、ND 或无许可的来源。
2. **三层署名**：L1（原始设计工作室/设计师）→ L2（skill/上游作者）→ L3（我们的转化）。三层都必须记在 `provenance` 里。
3. **命名**：用描述设计特征的名字，不要挪用工作室/设计师名。❌ `frame-pentagram-stat` → ✅ `frame-editorial-anchor`
4. **转化质量**：必须新增真实的动效时间线、用自有示例数据、相比上游有可辨别的再设计。
5. **查重**：跟同一上游来源的已有模板比对，不提交几乎一样的变体。
### 如何添加新引擎适配器
引擎适配器接口（[RFC-01](research/2026-05-26-spec-01-engine-adapter.md)）让任何视频渲染后端都能接入 html-video。已发布的适配器是 `@html-video/adapter-hyperframes` —— 用它作为参考实现。
#### 1. 创建新包
```
packages/adapter-<engine>/
├── package.json
├── src/
│ ├── index.ts # 导出默认 EngineAdapter 实例
│ ├── capabilities.ts # 静态能力声明
│ ├── validate.ts # 校验模板能否被本引擎渲染
│ └── render.ts # 核心渲染实现
└── tsconfig.json
```
#### 2. 实现 EngineAdapter 接口
核心契约定义在 `packages/core/src/types.ts`，详见 [RFC-01](research/2026-05-26-spec-01-engine-adapter.md)。关键约定：
- **进程隔离**：每次 `render()` 启动独立子进程。子进程崩溃必须 reject promise，不留下不完整的输出文件。
- **进度报告**：按当前帧/总帧数算 0-100%。阶段提示：`preparing`（0-10%）、`rendering`（10-95%）、`muxing`（95-100%）。
- **取消**：响应 `ctx.signal.aborted` —— 杀掉子进程，清理 workDir 临时文件，reject `AbortError`。
- **包命名**：`@html-video/adapter-<name>`，peer-depend 上游引擎。
### 代码风格
- **语言**：TypeScript（strict 模式，继承 `tsconfig.base.json`）
- **格式化**：[Biome](https://biomejs.dev/) —— 2 空格缩进、单引号、尾逗号、分号、LF 换行。运行 `pnpm format` 自动格式化。
- **Lint**：`pnpm lint` 用 Biome linter 的推荐规则。
- **单体仓库工具链**：pnpm workspace，`pnpm -r build` 按顺序构建所有包。
- **Import**：TypeScript import 使用 `.js` 扩展名（ESM 兼容）。
#### 提交前检查
```bash
pnpm typecheck # 全仓库 TypeScript 检查
pnpm lint # Biome linter
pnpm format # 自动格式化所有文件
pnpm test # 运行所有测试
```
### 如何提交 Pull Request
1. **Fork** 仓库，从 `main` 创建分支。
2. **做出改动** —— 保持聚焦。一个 PR = 一个逻辑变更。
3. **测试你的改动** —— 运行 `pnpm typecheck && pnpm lint && pnpm test`。
4. **写清楚的 commit message** —— 遵循 conventional commits：`feat:`、`fix:`、`docs:`、`refactor:`、`test:`、`chore:`。
5. **发起 PR** 到 `nexu-io/html-video:main`。
6. **描述**你改了什么、为什么、怎么验证。
我们会定期 review PR。如果你的 PR 加了新功能，请附带或链接相关测试。
### 许可
贡献即表示你同意你的贡献将按照 [Apache-2.0 许可](LICENSE) 授权 —— 和项目其他部分一致。不需要签署贡献者协议（CLA）。
### 有问题？
- **实时聊天**：加入 [Open Design Discord](https://github.com/nexu-io/open-design#community)
- **Bug 和功能**：在 [GitHub](https://github.com/nexu-io/html-video/issues) 提 issue
- **设计决策**：阅读 [`research/`](research/) 中的 RFC
LICENSE
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
README.md
+289
View File
@@ -0,0 +1,289 @@
# html-video
<p align="center">
<img src="docs/assets/hero.png" alt="html-video — HTML becomes video, on your laptop" width="100%" />
</p>
> **HTML becomes video — on your laptop.** Bring your local coding agent (Open Design · Windsurf CLI · Trae CLI · Claude Code · Cursor · Codex · Gemini · Grok · Qwen · OpenCode · Copilot · Aider · Hermes · or the Anthropic API). Describe a video, or **paste an article link / GitHub repo**, and the agent turns it into a multi-frame, fully animated video — then renders it to a real MP4 right on your machine. One agent loop, pluggable rendering engines, a curated template gallery, optional AI soundtrack. Apache-2.0, no per-render fees, no vendor lock-in.
<p align="center">
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-Apache%202.0-blue.svg?style=flat-square" /></a>
<a href="#supported-agents"><img alt="Agents" src="https://img.shields.io/badge/agents-14%20backends-111?style=flat-square" /></a>
<a href="#showcase"><img alt="Templates" src="https://img.shields.io/badge/templates-21-3ce6ac?style=flat-square" /></a>
<a href="#turn-a-link-into-a-video"><img alt="Sources" src="https://img.shields.io/badge/from-article%20%C2%B7%20repo%20%C2%B7%20prompt-9b59b6?style=flat-square" /></a>
<a href="#soundtrack"><img alt="Soundtrack" src="https://img.shields.io/badge/soundtrack-AI%20music%20%2B%20narration-e67e22?style=flat-square" /></a>
<a href="#quick-start"><img alt="Quickstart" src="https://img.shields.io/badge/quickstart-3%20commands-22a34a?style=flat-square" /></a>
</p>
<!-- Built by the team behind Open Design — these link to its community on purpose. -->
<p align="center">
<a href="https://github.com/nexu-io/open-design#community"><img alt="Discord" src="https://img.shields.io/badge/discord-join-5865f2?style=flat-square&logo=discord&logoColor=white" /></a>
<a href="https://x.com/nexudotio"><img alt="Follow @nexudotio on X" src="https://img.shields.io/badge/follow-%40nexudotio-000000?style=flat-square&logo=x&logoColor=white" /></a>
<a href="https://github.com/nexu-io/open-design"><img alt="By the Open Design team" src="https://img.shields.io/badge/by-nexu--io%2Fopen--design-ff7043?style=flat-square&logo=github&logoColor=white" /></a>
</p>
<p align="center">
<b>An official project by the <a href="https://open-design.ai">Open Design</a> team</b> · <a href="https://open-design.ai">open-design.ai</a>
</p>
<p align="center"><b>English</b> · <a href="README.zh-CN.md">简体中文</a></p>
---
## Showcase
Every template below is a real, animated single-file HTML video — these are live renders, not mockups. Drop one in, let the agent fill it with your content, export to MP4.
<table>
<tr>
<td width="50%"><img src="docs/assets/templates/frame-data-chart-nyt.png" alt="NYT-style data chart" /></td>
<td width="50%"><img src="docs/assets/templates/frame-glitch-title.png" alt="Glitch title" /></td>
</tr>
<tr>
<td><b>frame-data-chart-nyt</b> · data-viz<br/>Editorial NYT-style animated line chart — headline, annotated data points, source line. For "the number went up" stories.</td>
<td><b>frame-glitch-title</b> · title card<br/>Chromatic-aberration glitch title with scanlines. For openers, drops, and "system online" energy.</td>
</tr>
<tr>
<td><img src="docs/assets/templates/frame-liquid-bg-hero.png" alt="Liquid background hero" /></td>
<td><img src="docs/assets/templates/frame-light-leak-cinema.png" alt="Light leak cinema" /></td>
</tr>
<tr>
<td><b>frame-liquid-bg-hero</b> · hero<br/>Aurora liquid-gradient hero with a centered headline. For product reveals and bold statements.</td>
<td><b>frame-light-leak-cinema</b> · cinematic<br/>Warm film-grain + light-leak cinematic frame. For mood, brand films, "a quiet year" storytelling.</td>
</tr>
<tr>
<td><img src="docs/assets/templates/vfx-text-cursor.png" alt="Typewriter cursor VFX" /></td>
<td><img src="docs/assets/templates/frame-logo-outro.png" alt="Logo outro" /></td>
</tr>
<tr>
<td><b>vfx-text-cursor</b> · VFX<br/>Typewriter text with a blinking terminal cursor. For code-style reveals and CLI demos.</td>
<td><b>frame-logo-outro</b> · outro<br/>Clean animated logo end card. For sign-offs and brand stamps at the end of any video.</td>
</tr>
</table>
…and 15 more, including multi-scene product promos, kinetic type, Swiss-grid and Vignelli data cards, decision-tree explainers, Takram-organic motion, and warm-grain editorial. Browse all 21 live in the studio gallery.
---
## Why this exists
HTML→Video is a real category — but every engine is opinionated, and each wants you to learn *its* authoring model:
| Engine | Paradigm | Tradeoff | In html-video |
|---|---|---|---|
| [Hyperframes](https://github.com/heygen-com/hyperframes) | HTML + CSS + GSAP, agent-skill driven | Single rendering paradigm | ✅ **Shipped** — the default engine; renders real MP4 via headless Chromium + ffmpeg |
| [Remotion](https://www.remotion.dev/) | React components | Source-available, paid above 4 devs | 🗺️ Planned |
| [Motion Canvas](https://github.com/motion-canvas/motion-canvas) · [Revideo](https://github.com/redotvideo/revideo) | TypeScript generators on canvas | Best for explainers, code-first | 🗺️ Planned |
| [Manim](https://github.com/3b1b/manim) & friends | Math / 3D first | Niche | 🗺️ Researching |
Picking the right engine per use case, learning each model, and stitching them into one workflow costs real engineering time. Most teams pick one and live with its limits.
**html-video is the meta-layer that sits above all of them.** You talk to your agent; it picks the engine, picks the template, fills in your content, and renders the video. The engine is an implementation detail behind a single adapter interface — one `render(input, ctx)` contract that any backend can satisfy. Add a new engine and every template, every agent, and the whole studio workflow get it for free. No new DSL to learn, no rewrite when you switch engines.
The same idea powers [Open Design](https://github.com/nexu-io/open-design) in the *design* space — an agent meta-layer over many tools. html-video is the *motion* counterpart from the same team.
> **Status:** the pluggable-engine architecture is in place, and the **Hyperframes engine is fully wired up and renders real MP4** — headless Chromium records the animated HTML frame-by-frame and ffmpeg encodes it (libx264). Remotion, Motion Canvas / Revideo, and Manim are on the roadmap: the adapter interface is designed for them, but their adapters aren't built yet. The "In html-video" column above is the single source of truth for what's actually runnable today.
---
## At a glance
| | |
|---|---|
| **Coding agents (14)** | Open Design (Vela) · Windsurf CLI · Trae CLI · Claude Code · Cursor Agent · Codex CLI · Gemini CLI · Grok Build · Qwen Code · OpenCode · GitHub Copilot CLI · Aider · Hermes · Anthropic Messages API — auto-detected on your `PATH`, switchable from the top bar. |
| **Real MP4 render** | Headless Chromium records the animated HTML and ffmpeg encodes it (libx264) — locally, no cloud render, no per-clip fee. |
| **Article / repo → video** | Paste a URL or GitHub repo; the studio fetches it server-side (handles WeChat 公众号 articles) and builds the video from the real content. |
| **21 templates** | Curated, license-clean patterns: data viz, product promos, social shorts, explainers, kinetic type, transitions — previewed live in the gallery. |
| **Multi-frame storyboards** | A content-graph drives multi-scene videos; edit per-frame text inline, reorder, re-render. |
| **AI soundtrack** | Optional background music + narration via MiniMax, mixed into the MP4 at export. |
| **Studio + CLI** | A local browser studio *and* a scriptable `html-video` CLI. |
| **License** | Apache-2.0 — no per-render fees, no seat caps, no contributor agreements. |
---
## How it works
One sentence (or one link) goes in; a real MP4 comes out. The pipeline is the same whether you start from a prompt, an article, or a repo:
```
prompt / link / repo
│
▼
① source fetch studio pulls the URL or repo server-side, flattens it to Markdown
│
▼
② agent loop your agent reads the material + the picked template's style and emits
│ a content-graph (the storyboard) + one HTML block per frame
▼
③ content-graph multi-frame IR — nodes (entity / data / text) + edges (sequence /
│ dependency / contrast); topo-sorted into frame order & timing
▼
④ per-frame HTML each node becomes a self-contained animated HTML frame on disk
│
▼
⑤ Hyperframes render headless Chromium loads each frame, records it (auto-extending to
│ cover the frame's own animation), → webm per frame
▼
⑥ ffmpeg each webm → mp4 (libx264), then concat into one video;
│ optional MiniMax music + narration mixed in
▼
your.mp4
```
Steps ②–④ are where the "meta-layer" lives: the agent decides the storyboard and the engine decides how to draw it, and neither leaks into the other. Step ⑤ is engine-specific — swapping in Remotion or Motion Canvas later replaces only that box, leaving the storyboard and the agent loop untouched. Everything runs on your machine; the only network calls are the optional source fetch and the optional soundtrack.
Single-frame videos take a fast path that skips the content-graph — one template, one HTML, straight to render.
---
## Turn a link into a video
This is what most people reach for: hand your agent a link, get a video back. The agents run as local CLIs with no network access of their own, so the studio fetches the source **server-side** and feeds the real content into the generation prompt — no copy-pasting article bodies, and pages behind a login-free server render (like WeChat 公众号) just work.
```
You: 做一个解读视频 https://mp.weixin.qq.com/s/…
Agent: 好，我读完了《用嘴剪视频的时代来了？…》这篇文章 — 这就基于它生成。下一步选风格。
→ multi-frame explainer, built from the article's actual points
```
- **Web article** → fetched and flattened to Markdown. Server-rendered pages like **WeChat 公众号** articles work out of the box.
- **GitHub repo** → description, top-level structure, and README pulled via the public API — great for "explain this open-source project" videos.
- **Just a prompt** → describe the topic and the agent writes the content from scratch.
Whatever the source, it becomes the material the video is actually built from — not decoration around a canned template. The agent reads the fetched content, decides how many scenes it needs, and writes a **content-graph storyboard**: the key points become frames, the relationships between them (this follows that, this contrasts with that) become edges, and the picked template's visual style is applied per frame. So a 1,500-word article turns into a paced multi-scene explainer whose every line traces back to something in the source, and a repo turns into a structured walkthrough of what the project actually is.
---
## Quick start
### Prerequisites
| Requirement | Minimum | Check |
|---|---|---|
| **Node.js** | 20+ | `node --version` |
| **pnpm** | 9+ | `pnpm --version` |
| **ffmpeg** | Any recent | `ffmpeg -version` |
| **Chromium** (or Playwright browsers) | — | `npx playwright install chromium` |
The default rendering engine records animated HTML in a headless Chromium browser, then uses ffmpeg (libx264) to encode MP4. Install Playwright's Chromium if you don't have a system install:
```bash
npx playwright install chromium
```
### Install & run
```bash
pnpm install
pnpm -r build
node packages/cli/dist/bin.js studio # opens the studio at http://127.0.0.1:3071
```
In the studio: pick a template (or just describe a video / paste a link), chat with your agent, edit per-frame text, add a soundtrack, and export MP4.
CLI utilities:
```bash
node packages/cli/dist/bin.js doctor # detect installed agents + engines
node packages/cli/dist/bin.js search-templates --intent "github stars race" --top 3
```
---
## Supported agents
Auto-detected on your `PATH`; switch the active one from the studio's top bar. The studio leads with **Open Design (Vela)** — one login, many models, lower cost — then falls back to the first *available* agent so a fresh project always has a working backend.
| Agent | Detection | Invocation |
|---|---|---|
| **Open Design (Vela)** | `vela` / bundled in the Open Design app | ACP over stdio — one login in Open Design, pick any model |
| **Windsurf CLI** | `windsurf` | `windsurf --yolo`, ACP over stdio |
| **Trae CLI** | `traecli` | `traecli acp serve --yolo`, ACP over stdio |
| **Claude Code** | `claude` | `claude --print`, prompt via stdin |
| **Cursor Agent** | `cursor-agent` | `cursor-agent --print` |
| **Codex CLI** | `codex` | `codex exec`, prompt via stdin |
| **Hermes** | `hermes` | Hermes ACP CLI |
| **Gemini CLI** | `gemini` | Prompt via stdin |
| **Grok Build** | `grok` | `grok -p <prompt>` |
| **Qwen Code** | `qwen` | Prompt via stdin |
| **OpenCode** | `opencode` | `opencode run`, prompt via stdin |
| **GitHub Copilot CLI** | `copilot` | `copilot --allow-all-tools`, prompt via stdin |
| **Aider** | `aider` | `aider --message <prompt>` |
| **Anthropic API** | BYOK | Direct Messages API — works with no CLI installed |
Nothing installed? Set an Anthropic key and the studio talks to the Messages API directly.
---
## Soundtrack
Give the finished video a voice. In **Settings → Audio**, add a MiniMax API key, then in the per-project **Soundtrack** panel:
- **Background music** — describe a mood (`calm cinematic ambient, slow build`); MiniMax generates an instrumental track.
- **Narration** — type a script; MiniMax reads it (TTS).
Both are mixed into the exported MP4 (music ducked under the voice, optional fade-in/out) via ffmpeg. No key configured? The rest of the studio works unchanged.
---
## Template gallery
The 21 templates aren't a random grab-bag — each one is a self-contained, agent-readable unit described by a `template.html-video.yaml` manifest the studio scans at startup. A manifest carries everything the agent needs to pick and drive the template without opening the HTML:
- **What it's for** — `category`, `tags`, and a `best_for` list (e.g. *"Corporate slide"*, *"Minimal report card"*) that `search-templates` matches your intent against.
- **What it outputs** — supported resolutions, aspect ratios, fps, duration bounds, whether it has an alpha channel or audio.
- **What goes in** — an `inputs` JSON schema, so the agent knows exactly which text/data slots to fill.
- **License provenance** — an SPDX id plus explicit `attribution_required` / `redistribution_allowed` / `commercial_use` flags, and an `assets_attribution` block pointing at the upstream source URL.
That last part is deliberate. Every template is **license-clean by construction**: forks carry their original license, the repo-root [`NOTICE.md`](templates/NOTICE.md) records each source and SPDX, and nothing without a clear permissive license ships. So you can put any of them in commercial work without an audit. Templates span data viz (NYT-style charts, Swiss/Vignelli grids), titles & VFX (glitch, kinetic type, typewriter cursor), heroes & cinematics (liquid gradients, light-leak, warm grain), product promos (15s / 30s multi-scene), and explainer scaffolds (decision trees) — and the format is open, so community templates drop in the same way.
---
## Architecture
```
packages/
├── core/ Project / Asset / ContentGraph types, registries, orchestrator,
│ MiniMax provider + ffmpeg audio mux
├── content-graph/ Multi-frame storyboard IR (nodes + edges, topo-sort)
│ runtime/ Agent runtime — detect / spawn / stream
│ (Open Design/Vela · Windsurf CLI · Trae CLI · Claude · Cursor · Codex · Gemini · Grok · Qwen · OpenCode · Copilot · Aider · Hermes · Anthropic API)
├── adapter-hyperframes/ Hyperframes engine adapter — real render via Chromium + ffmpeg
├── cli/ `html-video` command + the studio HTTP server + source fetching
└── project-studio/ Browser studio UI (chat, template gallery, frames, soundtrack, export)
templates/ 21 curated, license-clean video templates
research/ RFCs (engine adapter / template metadata / agent skill / content-graph)
```
---
## Roadmap
- [x] Engine adapter spec — one interface, N backends
- [x] Template metadata format — license-first, agent-readable
- [x] Multi-frame storyboard workflow (content-graph)
- [x] Studio: live template gallery, agent switcher, per-frame text editing
- [x] Source material: article / GitHub-repo → video
- [x] AI soundtrack (MiniMax music + narration), mixed at export
- [x] Real MP4 render — Hyperframes engine via headless Chromium + ffmpeg
- [x] Agent model selection — Open Design (Vela) backend, live model catalog
- [ ] Adapters for Remotion / Motion Canvas / Revideo
- [ ] Agent skill packages + a template marketplace
---
## References & lineage
| Project | Role here |
|---|---|
| [Open Design](https://github.com/nexu-io/open-design) | Sister project — the design-agent meta-layer; same team, shared philosophy |
| [HTML Anything](https://github.com/nexu-io/html-anything) | Sister project — HTML for *static* deliverables; html-video is the *motion* side |
| [Hyperframes](https://github.com/heygen-com/hyperframes) | The shipped engine adapter; the HTML+CSS+GSAP rendering paradigm and the source of several Apache-2.0 templates |
## License
[Apache-2.0](LICENSE)
## Built by
[nexu-io](https://github.com/nexu-io) — the team behind [Open Design](https://github.com/nexu-io/open-design). Join the [Discord](https://github.com/nexu-io/open-design#community) · follow [@nexudotio](https://x.com/nexudotio).
README.wehub.md
+7
View File
@@ -0,0 +1,7 @@
# WeHub 来源说明
- 原始项目：`nexu-io/html-video`
- 原始仓库：https://github.com/nexu-io/html-video
- 导入方式：上游默认分支的最新快照
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
- 本文件仅用于记录来源，不代表 WeHub 是原项目作者
README.zh-CN.md
+288
View File
@@ -0,0 +1,288 @@
# html-video
<p align="center">
<img src="docs/assets/hero.png" alt="html-video — 在你的电脑上，把 HTML 变成视频" width="100%" />
</p>
> **在你的电脑上，把 HTML 变成视频。** 接上你本地的 coding agent（Open Design · Windsurf CLI · Trae CLI · Claude Code · Cursor · Codex · Gemini · Grok · Qwen · OpenCode · Copilot · Aider · Hermes · 或 Anthropic API），描述一个视频，或者**直接粘一个文章链接 / GitHub 仓库**，agent 就把它变成一支多帧、带动画的视频 —— 然后就在你这台机器上渲染成真实 MP4。一个 agent 循环、可插拔渲染引擎、精选模板库、可选 AI 配乐。Apache-2.0，无单次渲染费用，不绑定厂商。
<p align="center">
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-Apache%202.0-blue.svg?style=flat-square" /></a>
<a href="#支持的-agent"><img alt="Agents" src="https://img.shields.io/badge/agents-14%20backends-111?style=flat-square" /></a>
<a href="#作品展示"><img alt="Templates" src="https://img.shields.io/badge/templates-21-3ce6ac?style=flat-square" /></a>
<a href="#把链接变成视频"><img alt="Sources" src="https://img.shields.io/badge/from-article%20%C2%B7%20repo%20%C2%B7%20prompt-9b59b6?style=flat-square" /></a>
<a href="#配乐"><img alt="Soundtrack" src="https://img.shields.io/badge/soundtrack-AI%20music%20%2B%20narration-e67e22?style=flat-square" /></a>
<a href="#快速开始"><img alt="Quickstart" src="https://img.shields.io/badge/quickstart-3%20commands-22a34a?style=flat-square" /></a>
</p>
<p align="center">
<a href="https://github.com/nexu-io/open-design#community"><img alt="Discord" src="https://img.shields.io/badge/discord-join-5865f2?style=flat-square&logo=discord&logoColor=white" /></a>
<a href="https://x.com/nexudotio"><img alt="Follow @nexudotio on X" src="https://img.shields.io/badge/follow-%40nexudotio-000000?style=flat-square&logo=x&logoColor=white" /></a>
<a href="https://github.com/nexu-io/open-design"><img alt="By the Open Design team" src="https://img.shields.io/badge/by-nexu--io%2Fopen--design-ff7043?style=flat-square&logo=github&logoColor=white" /></a>
</p>
<p align="center">
<b>Open Design 团队官方出品</b> · <a href="https://open-design.ai">open-design.ai</a>
</p>
<p align="center"><a href="README.md">English</a> · <b>简体中文</b></p>
---
## 作品展示
下面每个模板都是一支真实、带动画的单文件 HTML 视频 —— 这些是实时渲染截图，不是效果图。挑一个，让 agent 填进你的内容，导出 MP4。
<table>
<tr>
<td width="50%"><img src="docs/assets/templates/frame-data-chart-nyt.png" alt="NYT 风格数据图表" /></td>
<td width="50%"><img src="docs/assets/templates/frame-glitch-title.png" alt="故障标题" /></td>
</tr>
<tr>
<td><b>frame-data-chart-nyt</b> · 数据可视化<br/>纽约时报风格的动态折线图 —— 大标题、标注数据点、来源行。适合「数字涨上去了」类叙事。</td>
<td><b>frame-glitch-title</b> · 标题卡<br/>带色彩偏移与扫描线的故障标题。适合开场、爆点、「系统上线」式的能量感。</td>
</tr>
<tr>
<td><img src="docs/assets/templates/frame-liquid-bg-hero.png" alt="液态背景主视觉" /></td>
<td><img src="docs/assets/templates/frame-light-leak-cinema.png" alt="漏光电影感" /></td>
</tr>
<tr>
<td><b>frame-liquid-bg-hero</b> · 主视觉<br/>极光液态渐变背景 + 居中大标题。适合产品发布与有力的口号。</td>
<td><b>frame-light-leak-cinema</b> · 电影感<br/>暖色胶片颗粒 + 漏光的电影感画面。适合氛围片、品牌片、叙事短片。</td>
</tr>
<tr>
<td><img src="docs/assets/templates/vfx-text-cursor.png" alt="打字机光标特效" /></td>
<td><img src="docs/assets/templates/frame-logo-outro.png" alt="Logo 片尾" /></td>
</tr>
<tr>
<td><b>vfx-text-cursor</b> · 特效<br/>打字机文字 + 闪烁的终端光标。适合代码风揭示、CLI 演示。</td>
<td><b>frame-logo-outro</b> · 片尾<br/>干净的 Logo 动画结束卡。适合任何视频结尾的署名与品牌落版。</td>
</tr>
</table>
……还有 15 个，包括多场景产品宣传、动感排版、瑞士网格与 Vignelli 数据卡、决策树解说、Takram 有机动效、暖色颗粒杂志风。全部 21 个都可在 studio 模板库里实时浏览。
---
## 为什么做这个
HTML→Video 是个真实存在的品类 —— 但每个引擎都各有主张，且都要你学*它自己*的创作模型：
| 引擎 | 范式 | 取舍 | 在 html-video 中 |
|---|---|---|---|
| [Hyperframes](https://github.com/heygen-com/hyperframes) | HTML + CSS + GSAP，agent skill 驱动 | 单一渲染范式 | ✅ **已发布** —— 默认引擎；经无头 Chromium + ffmpeg 渲染出真实 MP4 |
| [Remotion](https://www.remotion.dev/) | React 组件 | source-available，4 人以上收费 | 🗺️ 计划中 |
| [Motion Canvas](https://github.com/motion-canvas/motion-canvas) · [Revideo](https://github.com/redotvideo/revideo) | canvas 上的 TypeScript 生成器 | 最适合解说类、代码优先 | 🗺️ 计划中 |
| [Manim](https://github.com/3b1b/manim) 等 | 数学 / 3D 优先 | 小众 | 🗺️ 调研中 |
按场景挑引擎、学每一种创作模型、再把它们拼成一条工作流，都要耗真实的工程时间。多数团队挑一个、然后忍受它的局限。
**html-video 是凌驾于它们之上的 meta-layer。** 你跟 agent 对话，它挑引擎、挑模板、填进你的内容、渲染视频。引擎只是一个适配器接口背后的实现细节 —— 一份 `render(input, ctx)` 契约，任何后端满足它即可接入。加一个新引擎，所有模板、所有 agent、整条 studio 工作流就都自动用上了。不用学新的 DSL，换引擎也不用重写。
同一套思路也驱动着 [Open Design](https://github.com/nexu-io/open-design) 在*设计*领域的产品 —— 凌驾于众多工具之上的 agent meta-layer。html-video 是同一团队在*动态视频*这一面的对应物。
> **状态：** 可插拔引擎架构已就位，**Hyperframes 引擎已完整接通、能渲染出真实 MP4** —— 无头 Chromium 逐帧录制带动画的 HTML，再用 ffmpeg 编码（libx264）。Remotion、Motion Canvas / Revideo、Manim 在路线图上：适配器接口已为它们设计好，但适配器本身还没写。上表「在 html-video 中」这一列，是当下真正可运行内容的唯一权威来源。
---
## 速览
| | |
|---|---|
| **Coding agent（14 个）** | Open Design (Vela) · Windsurf CLI · Trae CLI · Claude Code · Cursor Agent · Codex CLI · Gemini CLI · Grok Build · Qwen Code · OpenCode · GitHub Copilot CLI · Aider · Hermes · Anthropic Messages API —— 在 `PATH` 上自动探测，顶栏一键切换。 |
| **真实 MP4 渲染** | 无头 Chromium 录制带动画的 HTML，ffmpeg 编码（libx264）—— 全在本地，无云端渲染，无单片费用。 |
| **文章 / 仓库 → 视频** | 粘一个 URL 或 GitHub 仓库；studio 在服务端抓取（支持微信公众号文章），用真实内容生成视频。 |
| **21 个模板** | 精选、许可清晰的样式：数据可视化、产品宣传、社媒短片、解说、动感排版、转场 —— 在模板库里实时预览。 |
| **多帧故事板** | content-graph 驱动多场景视频；逐帧改文案、重排、重渲染。 |
| **AI 配乐** | 可选背景音乐 + 旁白（MiniMax），导出时混进 MP4。 |
| **Studio + CLI** | 一个本地浏览器 studio，外加一个可脚本化的 `html-video` CLI。 |
| **许可** | Apache-2.0 —— 无单次渲染费、无席位上限、无贡献者协议。 |
---
## 它如何工作
一句话（或一个链接）进去，一支真实 MP4 出来。不论你从一句 prompt、一篇文章还是一个仓库开始，管线都是同一条：
```
prompt / 链接 / 仓库
│
▼
① 来源抓取 studio 在服务端拉取 URL 或仓库，扁平成 Markdown
│
▼
② agent 循环 agent 读素材 + 所选模板的风格，产出一份
│ content-graph（故事板）+ 每帧一个 HTML 块
▼
③ content-graph 多帧中间表示 —— 节点（实体 / 数据 / 文本）+ 边（顺序 /
│ 依赖 / 对比）；拓扑排序成帧序与时长
▼
④ 逐帧 HTML 每个节点变成一个自包含、带动画的 HTML 帧，落到磁盘
│
▼
⑤ Hyperframes 渲染 无头 Chromium 加载每一帧并录制（会自动延长时长
│ 以覆盖该帧自身的动画）→ 每帧一个 webm
▼
⑥ ffmpeg 每个 webm → mp4（libx264），再 concat 成一支视频；
│ 可选混入 MiniMax 的音乐 + 旁白
▼
你的.mp4
```
第 ②–④ 步正是「meta-layer」所在：agent 决定故事板，引擎决定怎么画，两者互不渗透。第 ⑤ 步是引擎相关的 —— 以后换成 Remotion 或 Motion Canvas，只替换这一个环节，故事板和 agent 循环原封不动。全程在你本机运行；唯二的网络调用是可选的来源抓取和可选的配乐。
单帧视频走一条快速路径，跳过 content-graph —— 一个模板、一个 HTML，直接渲染。
---
## 把链接变成视频
这是大多数人最想要的用法：丢一个链接给 agent，拿回一支视频。agent 作为本地 CLI 运行、自身没有联网能力，所以 studio 在**服务端**抓取来源、把真实内容喂进生成 prompt —— 不用手动复制正文，像微信公众号这种无需登录的服务端渲染页面也直接能用。
```
你： 做一个解读视频 https://mp.weixin.qq.com/s/…
Agent：好，我读完了《用嘴剪视频的时代来了？…》这篇文章 — 这就基于它生成。下一步选风格。
→ 一支多帧解说视频，基于文章的真实要点
```
- **网页文章** → 抓取并扁平成 Markdown。像**微信公众号**这种服务端渲染的页面开箱即用。
- **GitHub 仓库** → 通过公开 API 拉取简介、顶层结构、README —— 很适合做「解读某开源项目」的视频。
- **只给一句话** → 描述主题，agent 从零写内容。
无论哪种来源，它都会成为视频真正依据的素材 —— 不是套模板时的摆设。agent 读取抓来的内容，自己决定需要几个场景，写出一份 **content-graph 故事板**：要点变成帧，要点之间的关系（这个承接那个、这个对比那个）变成边，再把所选模板的视觉风格逐帧应用上去。于是一篇 1500 字的文章变成一支有节奏的多场景解说，每一句都能追溯回原文的某处；一个仓库变成一段「这个项目到底是什么」的结构化讲解。
---
## 快速开始
### 前置依赖
| 依赖 | 最低版本 | 检查方式 |
|---|---|---|
| **Node.js** | 20+ | `node --version` |
| **pnpm** | 9+ | `pnpm --version` |
| **ffmpeg** | 任意较新版本 | `ffmpeg -version` |
| **Chromium**（或 Playwright 浏览器）| — | `npx playwright install chromium` |
默认渲染引擎用无头 Chromium 录制带动画的 HTML，再用 ffmpeg（libx264）编码为 MP4。如果没有系统安装的 Chromium，装 Playwright 内置的：
```bash
npx playwright install chromium
```
### 安装 & 运行
```bash
pnpm install
pnpm -r build
node packages/cli/dist/bin.js studio # 在 http://127.0.0.1:3071 打开 studio
```
在 studio 里：挑一个模板（或直接描述视频 / 粘链接），跟 agent 对话，逐帧改文案，加配乐，导出 MP4。
CLI 工具：
```bash
node packages/cli/dist/bin.js doctor # 探测已安装的 agent + 引擎
node packages/cli/dist/bin.js search-templates --intent "github stars race" --top 3
```
---
## 支持的 Agent
在 `PATH` 上自动探测；在 studio 顶栏切换当前 agent。studio 默认把 **Open Design (Vela)** 排在最前 —— 一次登录、多种模型、成本更低 —— 然后回落到第一个*可用*的 agent，保证新项目永远有一个能用的后端。
| Agent | 探测 | 调用 |
|---|---|---|
| **Open Design (Vela)** | `vela` / Open Design 应用内置 | ACP over stdio —— 在 Open Design 里登录一次，任选模型 |
| **Windsurf CLI** | `windsurf` | `windsurf --yolo`，ACP over stdio |
| **Trae CLI** | `traecli` | `traecli acp serve --yolo`，ACP over stdio |
| **Claude Code** | `claude` | `claude --print`，prompt 走 stdin |
| **Cursor Agent** | `cursor-agent` | `cursor-agent --print` |
| **Codex CLI** | `codex` | `codex exec`，prompt 走 stdin |
| **Hermes** | `hermes` | Hermes ACP CLI |
| **Gemini CLI** | `gemini` | prompt 走 stdin |
| **Grok Build** | `grok` | `grok -p <prompt>` |
| **Qwen Code** | `qwen` | prompt 走 stdin |
| **OpenCode** | `opencode` | `opencode run`，prompt 走 stdin |
| **GitHub Copilot CLI** | `copilot` | `copilot --allow-all-tools`，prompt 走 stdin |
| **Aider** | `aider` | `aider --message <prompt>` |
| **Anthropic API** | BYOK | 直连 Messages API —— 不装任何 CLI 也能用 |
什么都没装？配一个 Anthropic key，studio 直接走 Messages API。
---
## 配乐
给成片加上声音。在 **Settings → Audio** 填入 MiniMax API key，然后在每个项目的 **Soundtrack** 面板：
- **背景音乐** —— 描述一种情绪（`舒缓的电影感氛围，缓慢推进`），MiniMax 生成一段器乐。
- **旁白** —— 输入文案，MiniMax 朗读（TTS）。
两者都会通过 ffmpeg 混进导出的 MP4（音乐压低到人声之下，可选淡入淡出）。没配 key？studio 其余部分照常工作。
---
## 模板库
这 21 个模板不是随手凑的一堆 —— 每一个都是自包含、agent 可读的单元，由一份 `template.html-video.yaml` 清单描述，studio 启动时扫描加载。一份清单带齐了 agent 挑选和驱动这个模板所需的一切，根本不用打开 HTML：
- **它用来做什么** —— `category`、`tags`，外加一个 `best_for` 清单（如*「企业幻灯片」*、*「极简报告卡」*），`search-templates` 拿你的意图去匹配它。
- **它输出什么** —— 支持的分辨率、画幅比、fps、时长上下限、是否有 alpha 通道或音频。
- **要喂什么进去** —— 一份 `inputs` JSON schema，让 agent 精确知道要填哪些文本 / 数据槽位。
- **许可来源** —— 一个 SPDX 标识，外加明确的 `attribution_required` / `redistribution_allowed` / `commercial_use` 标志，以及一个指向上游来源 URL 的 `assets_attribution` 块。
最后这一点是刻意为之。每个模板都**从构造上就许可清晰**：fork 来的保留其原始许可，仓库根的 [`NOTICE.md`](templates/NOTICE.md) 记下每个来源与 SPDX，没有清晰宽松许可的一律不收。所以你可以把其中任何一个放进商业作品里，无需再做审查。模板覆盖数据可视化（NYT 风格图表、Swiss / Vignelli 网格）、标题与特效（故障、动感排版、打字机光标）、主视觉与电影感（液态渐变、漏光、暖色颗粒）、产品宣传（15 秒 / 30 秒多场景）、解说骨架（决策树）—— 而且格式是开放的，社区模板用同样的方式接入。
---
## 架构
```
packages/
├── core/ Project / Asset / ContentGraph 类型、registry、orchestrator、
│ MiniMax provider + ffmpeg 音轨混合
├── content-graph/ 多帧故事板中间表示（节点 + 边，拓扑排序）
│ runtime/ Agent 运行时 —— 探测 / spawn / 流式
│ （Open Design/Vela · Windsurf CLI · Trae CLI · Claude · Cursor · Codex · Gemini · Grok · Qwen · OpenCode · Copilot · Aider · Hermes · Anthropic API）
├── adapter-hyperframes/ Hyperframes 引擎适配器 —— 经 Chromium + ffmpeg 真实渲染
├── cli/ `html-video` 命令 + studio HTTP server + 来源抓取
└── project-studio/ 浏览器 studio UI（对话、模板库、帧、配乐、导出）
templates/ 21 个精选、许可清晰的视频模板
research/ RFC（引擎适配器 / 模板元数据 / agent skill / content-graph）
```
---
## 路线图
- [x] 引擎适配器规范 —— 一个接口，N 个后端
- [x] 模板元数据格式 —— 许可优先、agent 可读
- [x] 多帧故事板工作流（content-graph）
- [x] Studio：实时模板库、agent 切换器、逐帧改文案
- [x] 来源素材：文章 / GitHub 仓库 → 视频
- [x] AI 配乐（MiniMax 音乐 + 旁白），导出时混合
- [x] 真实 MP4 渲染 —— Hyperframes 引擎经无头 Chromium + ffmpeg
- [x] Agent 模型选择 —— Open Design (Vela) 后端，实时模型目录
- [ ] Remotion / Motion Canvas / Revideo 适配器
- [ ] Agent skill 包 + 模板市场
---
## 参考与渊源
| 项目 | 在这里的角色 |
|---|---|
| [Open Design](https://github.com/nexu-io/open-design) | 姊妹项目 —— 设计 agent meta-layer；同一团队、同一理念 |
| [HTML Anything](https://github.com/nexu-io/html-anything) | 姊妹项目 —— 面向*静态*交付物的 HTML；html-video 是*动态*那一面 |
| [Hyperframes](https://github.com/heygen-com/hyperframes) | 已发布的引擎适配器；HTML+CSS+GSAP 渲染范式，也是若干 Apache-2.0 模板的来源 |
## 许可
[Apache-2.0](LICENSE)
## 出品
[nexu-io](https://github.com/nexu-io) —— [Open Design](https://github.com/nexu-io/open-design) 背后的团队。加入 [Discord](https://github.com/nexu-io/open-design#community) · 关注 [@nexudotio](https://x.com/nexudotio)。
assets/social-preview.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 943 KiB

biome.json
+32
View File
@@ -0,0 +1,32 @@
{
"$schema": "https://biomejs.dev/schemas/1.9.4/schema.json",
"files": {
"ignore": ["**/dist/**", "**/node_modules/**", "**/.html-video/**", "**/templates/*/source/**"]
},
"formatter": {
"indentStyle": "space",
"indentWidth": 2,
"lineWidth": 100,
"lineEnding": "lf"
},
"linter": {
"enabled": true,
"rules": {
"recommended": true,
"style": {
"noNonNullAssertion": "warn"
},
"suspicious": {
"noExplicitAny": "warn"
}
}
},
"javascript": {
"formatter": {
"quoteStyle": "single",
"trailingCommas": "all",
"semicolons": "always",
"arrowParentheses": "always"
}
}
}
docs/assets/hero.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.3 MiB

docs/assets/templates/frame-data-chart-nyt.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 101 KiB

docs/assets/templates/frame-glitch-title.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 267 KiB

docs/assets/templates/frame-light-leak-cinema.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 828 KiB

docs/assets/templates/frame-liquid-bg-hero.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 396 KiB

docs/assets/templates/frame-logo-outro.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 245 KiB

docs/assets/templates/vfx-text-cursor.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 322 KiB

notes/2026-05-26-foundation-decisions.md
+183
View File
@@ -0,0 +1,183 @@
# 2026-05-26 · 项目奠基决策日志
> 第一次会话产出的关键决策。串起 RFC-01/02/03，记录"为什么这么设计而不是别的"，留 open question 给后续会话。
---
## 三份 RFC 的关系
```
RFC-01: Engine Adapter 接口 ← 技术抽象层（最底）
↓ 定义 EngineAdapter / Capabilities / RenderInput / Output
RFC-02: Template Metadata 格式 ← 内容描述层
↓ 描述每个 template，含 engine 归属 + 跨引擎检索元数据
RFC-03: Agent Skill 设计 ← 用户交互层（最顶）
↓ CLI Contract + Claude Code SKILL.md + transpile 到其他 agent
```
三者必须**自下而上**实现：先有 adapter 抽象（RFC-01），才能写 template metadata 校验它（RFC-02），才能让 agent CLI 决策（RFC-03）。
---
## ⭐ 产品形态修订（mid-session 关键澄清）
Joey 在前 3 份 RFC 写完后澄清核心工作流："产品内支持上传图文资产 → 整合 → 生成 HTML 分镜 → 用户审 → 导出 MP4"。这是**两段式 storyboard-first** 工作流，跟前 3 份 RFC 假设的 "agent fill vars → 一步 render MP4" 显著不同。
→ 新增 [RFC-04: Storyboard-first Workflow](../research/2026-05-26-spec-04-storyboard-workflow.md)，作为前 3 份的修订指引。
**新核心命题**：
- 用户不是"工程师 with template id 和 vars"，是"创作者 with 一堆图文 + 一句话意图"
- HTML Storyboard 是关键中间产物——人可读、可调、可批准
- 只有 Storyboard 通过用户验收才 render MP4，避免渲染浪费 + 让用户保留创作主导权
**新护城河**：资产理解 + storyboard 编排 + 跨引擎渲染三层叠加，HF / Remotion 都没做前两层（他们假设用户已经知道想做什么）。
## 六个关键设计抉择（写下来不让后续会话动摇）
### 1. **不发明新 DSL，做 thin meta-layer**
拒绝："让用户写 html-video.json，我们编译到 4 个 backend"。
理由：永远落后任意 backend 的新 feature；4 个 transpile 维护地狱；用户写一段 React 想用 Remotion 优势但被 DSL 阉割 → 反向。
做法：用户写的是 backend-native 代码，html-video 只在外面包一层。
### 2. **每个 template = 一个 engine 归属**
拒绝："同一个 idea 出多 engine 实现的同一个 template id"。
理由：维护多份会漂移；用户视角下"同一模板"两版做出不同效果是 bug 不是 feature。
做法：要多 engine → 发布多个 template id（例 `bar-chart-race-mc` / `bar-chart-race-hf`），各自独立 metadata。
### 3. **CLI 是 agent 唯一接口**
拒绝："agent 直接 import @html-video/core"。
理由：进程隔离 + 跨 agent 复用（Claude Code / Cursor / Codex 都靠 shell 调），一个 contract 走天下。
做法：所有功能从 CLI 走，stdout 严格 JSON，stderr 做 progress / debug。
### 4. **不发明 cross-engine fallback**
拒绝："HF 渲染挂自动用 Revideo 重试"。
理由：用户选 engine 通常有理由（license、风格、团队偏好）；偷偷换会让结果不一致。
做法：失败 = 失败，agent 跟用户说，让 agent / 用户重新 search-templates。
### 5. **License 是一等公民**
拒绝："license 字段先留空之后再说"。
理由：商用 / 二改 / attribution 是开源生态的真实问题，跑起来再补成本翻倍；OD 已经因为 plugins 的 license 漂移踩过坑。
做法：metadata 强制要 SPDX id；CLI search 默认带 license-allow filter；CC-BY-SA / GPL 强 copyleft 给 warning。
### 6. **Storyboard 是用户审批 gate**（mid-session 新增）
拒绝："agent 一气呵成 render 出 MP4 就完了"。
理由：渲染慢且贵；用户对 HTML 改的成本远低于对 MP4 改；分镜阶段批改是创作产业的标准范式（电影、广告、动画都先 storyboard）。
做法：所有用户主流程必经 `storyboard preview` 阶段；`html-video render --template ...` 只留给开发者直跑（template 测试 / CI 用）；agent 不允许跳过 storyboard 直接 render MP4。
---
## v0.1 MVP 范围（按 RFC-04 修订后）
接下来写代码的范围：
- ✅ `@html-video/core`：Asset / AssetBundle / Storyboard / Scene 数据结构 + sqlite 持久化 + content-addressed asset store + Template registry + Engine selector
- ✅ `@html-video/adapter-hyperframes`：第一个 reference adapter，实现 `validate / render / renderToHtml / preview`
- ✅ `@html-video/cli`：`doctor / list-engines / assets upload / storyboard generate / storyboard edit / storyboard preview / storyboard render`（用户主流程）+ `render` / `inspect-template`（开发者 dev-mode）
- ✅ `@html-video/storyboard-ui`：浏览器 storyboard preview 页面（timeline + scene grid + inline edit + 全屏播放 mock）
- ✅ `@html-video/agent-skill-claude-code`：storyboard-first SKILL.md + 3-5 个 dogfood use case
- ✅ 5 个 reference templates（覆盖 intro / data-chart / image-pan / text-card / outro 五种 scene role）
- ✅ Monorepo（pnpm workspace）+ 基础 CI（type-check / lint / unit test）
**不**进 MVP（推到 v0.2+）：
- ❌ adapter-remotion / motion-canvas / revideo（先把 HF 路径 + storyboard 闭环跑通）
- ❌ AI 智能编排（v0.1 是 LLM 即兴选 template，v0.2 加规则 + ML）
- ❌ AI 视觉理解资产（v0.1 用 mime + 文件名启发）
- ❌ 协作 / 多人审稿
- ❌ Cloud / Lambda 部署（先本地 chromium）
- ❌ Telemetry（默认 off，alpha 不发）
- ❌ Template marketplace 网站（先 GitHub repo + npm 发布）
---
## 命名约定（避免后续 bikeshed）
| 类型 | 命名 | 例 |
|---|---|---|
| Repo | `nexu-io/html-video` | — |
| Monorepo workspace | `pnpm` | — |
| Core 包 | `@html-video/core` | — |
| Adapter 包 | `@html-video/adapter-<engine-id>` | `@html-video/adapter-hyperframes` |
| CLI 包 | `@html-video/cli` 或 binary 名 `html-video` | — |
| Agent skill 包 | `@html-video/agent-skill-<agent>` | `@html-video/agent-skill-claude-code` |
| Template 包 | `@html-video/template-<kebab-id>` | `@html-video/template-data-bar-chart-race` |
| Template metadata 文件 | `template.html-video.yaml` | — |
| Config 文件 | `html-video.config.json` | — |
---
## 跟姊妹项目的接口（提前想好）
| 项目 | 接口点 | 状态 |
|---|---|---|
| `growth-dashboard` (T7) | 数据源（SQLite）→ html-video template vars | 待 v0.2，写一个 `html-video-data` adapter |
| `open-design`（T5）/ OD 客户端 | OD 客户端 plugins-home 是否要列 html-video templates？ | 待 v0.2，等 html-video alpha 发布 |
| `od-pitch` (T6) | launch deck / 杂志风介绍页 | 等用户决定何时 launch |
| `od-landing` (T8) | nexu.io 主域 落地页给 html-video 加入口？ | 待 v1 |
---
## 风险 / 不确定性 register
| 风险 | 严重度 | 缓解 |
|---|---|---|
| HF 上游 breaking change | 高 | adapter peerDep 锁 semver 范围；CI 跑 HF latest 兼容性 |
| Remotion license 变更 | 中 | 不依赖 Remotion 收入模型；adapter 只是壳 |
| Motion Canvas 无 server render → adapter 难做 | 中 | v0.2 才接，先用 Revideo 替代 explainer 路径 |
| 用户嫌 4 个 backend 太重 | 中 | adapter lazy install，doctor 自动 hint，按需 pull |
| HF 自己迭代到 multi-backend → 跟我们重叠 | 高 | 早点 ship + 强调 "neutral aggregator" 定位（HF 是参与者不是中立方） |
| AI generative video（Sora / Veo / Runway）取代 HTML 路线 | 中 | v0.3+ 加 generative-video adapter（"生成 → HTML 模板再加工"两段式） |
---
## 下次会话该做什么（顺序明确）
按重要性 / 解锁度排：
1. **写 `@html-video/core` 骨架**（pnpm workspace + TS 配置 + EngineAdapter type 落地）
2. **写 `@html-video/adapter-hyperframes` 骨架**（先实现 capabilities + validate，render 跑通再说）
3. **挑 1 个 HF native template 当 reference**（直接拷一个 HF 仓库的 example template，套上 RFC-02 metadata 验证 schema 合理性）
4. **写 CLI `html-video doctor` + `list-engines`**（最简单的两个命令，跑通端到端最小回路）
5. **手写一次 SKILL.md，让 Claude Code 这边的本人会话试用**（dogfood，发现 prompt-level 问题）
每一步独立 commit，PR 评审走 nexu-io org（等 GitHub repo 推上去后开始）。
---
## Open Questions 集合（汇总三份 RFC 的）
### 来自 RFC-01
1. template variables 的 schema 校验放 core 还是 adapter？
2. 音频混音 core helper vs adapter 各自处理？
3. 字幕 burn-in / sidecar 谁烧？
4. 资产 resolver（图片字体）放哪？
5. 跨 engine retry —— v0.1 已决定不做
### 来自 RFC-02
1. i18n 多语言 metadata
2. 复杂模板的 input grouping
3. Template 之间的依赖（transition + main 组合）
4. 外部数据源（CSV / Sheets URL）支持
5. A/B 变体共享同源代码
### 来自 RFC-03
1. Skill 之间协作（html-video → growth-dashboard 直接读 SQLite vs MCP）
2. Multi-shot timeline composer skill
3. Content-hash cache
4. OpenCode 集成路径
这些不是 v0.1 阻塞，但写代码时遇到要回查这一节避免重新发明。
---
## 给 Joey 的复盘 sentence
> 一次会话沉淀了 4 份 RFC（adapter / template / agent-skill / storyboard）+ 1 份决策日志，定下：**meta-layer / 不发明 DSL / template engine-locked / CLI 唯一 agent 接口 / license 一等公民 / Storyboard-first 用户审批 gate**。下次会话可直接进 `@html-video/core` 代码骨架（按 RFC-04 修订后的 v0.1 范围），不再 bikeshed 抽象。
notes/2026-06-04-provenance-audit.md
+66
View File
@@ -0,0 +1,66 @@
# Provenance audit — 模板来源核查报告
> 2026-06-04。起因:发现今天新增的 3 个模板(已回退)与仓库已有模板视觉撞车,
> 进而对整个模板库的"原创性声明 + 署名"做了一次真实上游比对。
## TL;DR
仓库里 6 个模板声明了 provenance,来自 2 个 MIT 上游。核查后发现 **3 个严肃问题**:
1. **署名链条缺了一层**:`Pentagram` / `Build` / `Takram` 不是上游(huashu-design)原创,
而是 huashu 在**致敬真实世界的知名设计公司/设计师**。我们直接拿这些公司名当了模板
id 和显示名,而 huashu 自己的文档明确告诫"不要直接写 in the style of Pentagram"。
2. **"original / no source copied" 声明未经核实**:这句话是模板生成时自我宣称的,从未
真去上游 diff。实测复刻程度逐个不同(Pentagram 红+白全同;Build 共享 4 个暖灰色)。
3. **连示例数据都是照搬的**:`95.7 / 73.8 / 87.4 / AIME / SWE-bench` 直接来自上游
ppt 源页,不是我们自编的占位数据。
## 上游核实
| 上游 repo | 真实版权人 | License | 用于我们的模板 |
|---|---|---|---|
| [alchaincyf/huashu-design](https://github.com/alchaincyf/huashu-design) | **alchaincyf(花叔 · 花生)** © 2026 | MIT ✅ 已核实 | build-minimal, pentagram-stat, takram-organic |
| [zarazhangrui/frontend-slides](https://github.com/zarazhangrui/frontend-slides) | **Zara Zhang** © 2025 | MIT ✅ 已核实 | bold-signal, creative-voltage, electric-studio |
License 真的是 MIT,允许商用 + 再分发,**所以这不是"能不能用"的问题**——MIT 完全允许。
问题在**署名是否写清楚、风格名是否该直接挪用**。
## 关键发现:Pentagram/Build/Takram 是三层署名,不是两层
huashu-design 的 `references/design-styles.md` 明确写道:
- **01 Pentagram** = "Michael Bierut 风格 / Pentagram"(搜索词:pentagram hillary logo system)
—— 真实的全球顶级设计公司 Pentagram,设计师 Michael Bierut
- **11 Build** = "Build studio luxury minimalism"(搜索词:build studio london branding)
—— 伦敦设计工作室 Build
- **17 Takram** = "Takram Japanese speculative design"(搜索词:takram nhk data visualization)
—— 日本设计公司 Takram
→ 真实署名链是:**Pentagram / Build / Takram(真实工作室)→ huashu-design 提炼为"风格 +
HTML 示例"(MIT,花叔)→ html-video 转成视频模板**。我们当前 provenance 只写到 huashu
这一层,且把工作室名当成了模板名直接挂出。
huashu 自己的规矩(design-styles.md):
> ❌ 不要直接写 "in the style of Pentagram" → ✅ 用具体设计特征描述
我们的模板 id 恰恰是 `frame-pentagram-stat` / `frame-takram-radar` / `frame-build-minimal`。
## 实质比对(上游 ppt 源页 vs 我们的模板)
| 风格 | 共享配色 | 字体 | 复刻程度 |
|---|---|---|---|
| Pentagram | 2/2(#E63946 红 + #FFFFFF 全同) | 我们换成 Archivo | 高 — 配色照搬 |
| Build | 4/7(#A8A4A0 #B0ACA4 #D4A574 暖灰全同) | 同 Inter | 高 — 配色+字体同源 |
| Takram | 0/7(配色我们重新调过) | 我们换成 Manrope | 中 — 重新着色 |
示例数据 `95.7/73.8/87.4 + AIME/SWE-bench`:**三个上游 ppt 页里都有,我们照搬。**
## 结论
- **不是侵权**(MIT 授权,允许商用+演绎),但**署名不充分 + 风格命名不当**。
- 对一个要公开发布(Apache-2.0)、给 nexu.io / OD 生态导流的项目,这是要修的诚信/署名问题。
## 整改方案 + 标准转换规范
规范已定:见 [RFC-07 PPT Skill → Hyperframes 模板 转换规范](../research/2026-06-04-spec-07-ppt-to-template.md)。
下一步用 RFC-07 回头整改本报告列出的 6 个模板(三层署名 + 重命名 + 自有示例)。
package.json
+41
View File
@@ -0,0 +1,41 @@
{
"name": "html-video-monorepo",
"version": "0.0.0",
"private": true,
"description": "HTML→Video meta-layer for coding agents",
"license": "Apache-2.0",
"homepage": "https://github.com/nexu-io/html-video",
"repository": {
"type": "git",
"url": "git+https://github.com/nexu-io/html-video.git"
},
"engines": {
"node": ">=20",
"pnpm": ">=9"
},
"packageManager": "pnpm@9.15.0",
"workspaces": [
"packages/*",
"templates/*"
],
"scripts": {
"build": "pnpm -r build",
"typecheck": "pnpm -r typecheck",
"test": "pnpm -r test",
"smoke": "pnpm --filter @html-video/cli smoke",
"format": "biome format --write .",
"lint": "biome lint ."
},
"devDependencies": {
"@biomejs/biome": "^1.9.4",
"@types/node": "^22.10.0",
"typescript": "^5.7.2"
},
"dependencies": {
"@remotion/bundler": "^4",
"@remotion/renderer": "^4",
"react": "^18",
"react-dom": "^18",
"remotion": "^4"
}
}
packages/adapter-hyperframes/package.json
+36
View File
@@ -0,0 +1,36 @@
{
"name": "@html-video/adapter-hyperframes",
"version": "0.1.0",
"description": "Hyperframes adapter for html-video meta-layer",
"license": "Apache-2.0",
"type": "module",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
".": {
"import": "./dist/index.js",
"types": "./dist/index.d.ts"
}
},
"files": ["dist", "README.md"],
"scripts": {
"build": "tsc -p tsconfig.json",
"typecheck": "tsc -p tsconfig.json --noEmit",
"test": "node --test test/"
},
"dependencies": {
"@html-video/core": "workspace:*",
"playwright": "^1.49.0"
},
"peerDependencies": {
"hyperframes": "^0.4.0"
},
"peerDependenciesMeta": {
"hyperframes": {
"optional": true
}
},
"devDependencies": {
"typescript": "^5.7.2"
}
}
packages/adapter-hyperframes/src/capabilities.ts
+31
View File
@@ -0,0 +1,31 @@
import type { EngineCapabilities } from '@html-video/core';
/**
* Static capability declaration for Hyperframes.
* v0.1: hard-coded; v0.2 may detect upstream version and adjust.
*/
export const capabilities: EngineCapabilities = {
paradigms: ['html-css-gsap'],
outputFormats: ['mp4', 'webm', 'webm-alpha', 'png-sequence'],
maxResolution: { width: 3840, height: 2160 },
alpha: true,
audio: 'multi',
subtitles: ['burn-in', 'sidecar'],
renderTarget: ['local-chromium', 'lambda'],
licensing: 'free-osi',
renderSpeedHint: {
resolution: '1080p',
durationSec: 10,
fps: 60,
estimatedRenderSec: 18,
},
bestFor: [
'social-shorts',
'product-marketing',
'logo-reveal',
'gsap-animations',
'ken-burns',
'text-card',
],
weaknesses: ['no-react-ecosystem', 'limited-3d-without-three.js'],
};
packages/adapter-hyperframes/src/index.ts
+17
View File
@@ -0,0 +1,17 @@
import type { EngineAdapter } from '@html-video/core';
import { capabilities } from './capabilities.js';
import { validate } from './validate.js';
import { render, renderToHtml } from './render.js';
const adapter: EngineAdapter = {
id: 'hyperframes',
name: 'Hyperframes',
upstreamVersion: '0.4.x',
capabilities,
validate,
render,
renderToHtml,
};
export default adapter;
export { adapter };
packages/adapter-hyperframes/src/render.ts
+617
View File
@@ -0,0 +1,617 @@
/**
* Hyperframes render() — real recording via Playwright + ffmpeg.
*
* Per-frame strategy (orchestrator already loops per node and concats):
* 1. Launch chromium headless at the configured resolution
* 2. recordVideo into a tmp dir
* 3. file:// load the frame HTML
* 4. wait `durationSec` so any opening animation completes + plays
* 5. close → playwright dumps a webm
* 6. ffmpeg transmux/encode the webm to mp4 at `outputPath`
*
* Upstream Hyperframes was never required at runtime for this adapter —
* our generated HTML is plain inline-CSS+JS, chromium runs it as-is.
*/
import { copyFile, mkdir, mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises';
import { existsSync, readdirSync } from 'node:fs';
import { spawn } from 'node:child_process';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { pathToFileURL } from 'node:url';
import type {
HtmlSceneOutput,
RenderContext,
RenderInput,
RenderOutput,
} from '@html-video/core';
import { HtmlVideoError } from '@html-video/core';
const ADAPTER_VERSION = '0.2.0-playwright';
/** Real render: chromium records the page, ffmpeg transcodes to MP4. */
export async function render(input: RenderInput, ctx: RenderContext): Promise<RenderOutput> {
const t0 = Date.now();
ctx.onProgress?.(5, 'preparing');
const outDir = dirname(input.config.outputPath);
await mkdir(outDir, { recursive: true });
if (ctx.signal?.aborted) throw new HtmlVideoError('cancelled', 'Aborted');
// Resolve the source HTML path. Templates pass an absolute path already;
// multi-frame `core` calls pass the per-frame HTML path the same way.
if (!existsSync(input.template.sourcePath)) {
throw new HtmlVideoError(
'template-invalid',
`Source HTML not found: ${input.template.sourcePath}`,
);
}
let totalDuration =
input.config.duration === 'auto' ? 5 : Math.max(0.5, Number(input.config.duration));
const { width, height } = input.config.resolution;
const fps = input.config.fps || 30;
// Lazy-load playwright so the import cost only hits actual exports.
ctx.onProgress?.(15, 'launching browser');
const playwright = await import('playwright').catch((err) => {
throw new HtmlVideoError(
'render-failed',
`playwright not installed (run \`pnpm install\` from the monorepo root). ${err instanceof Error ? err.message : err}`,
);
});
const recordDir = await mkdtemp(join(tmpdir(), 'hv-render-'));
let browser: import('playwright').Browser | undefined;
let webmPath: string | undefined;
let cleanupSrc: (() => Promise<void>) | undefined;
// Wall-clock offset (ms) from when the webm starts recording to when we
// actually start the animation, so ffmpeg can trim the dead opening lead-in.
let leadInMs = 0;
try {
browser = await playwright.chromium.launch({
headless: true,
args: ['--no-sandbox', '--disable-blink-features=AutomationControlled'],
});
// recordVideo starts capturing the moment the context exists, so this is
// the webm's t=0 reference.
const tWebmStart = Date.now();
const context = await browser.newContext({
viewport: { width, height },
deviceScaleFactor: 1,
recordVideo: { dir: recordDir, size: { width, height } },
});
const page = await context.newPage();
// Freeze all CSS/SMIL animations the instant the document starts parsing,
// BEFORE any @keyframes can begin counting down. Single-file templates are
// pure CSS `animation: … forwards` timelines with no JS trigger — they
// start running on the wall clock the moment the element is styled, i.e.
// right after goto(). Meanwhile we then spend ~2–3s waiting for the Google
// Fonts faces (Shrikhand et al.) to download. Without this freeze the whole
// opening (text fading in while the real face is still downloading, then
// the swap) plays out during that font wait and gets recorded. Pausing all
// animations up front lets us hold the timeline at frame 0 until fonts are
// ready, then release it so capture and motion start together — the same
// shape as the multi-composition paused→drive path below.
await page.addInitScript(() => {
const style = document.createElement('style');
style.id = '__hv_freeze';
style.textContent =
'*, *::before, *::after { animation-play-state: paused !important;' +
' -webkit-animation-play-state: paused !important; }';
const attach = () => (document.head || document.documentElement).appendChild(style);
if (document.head || document.documentElement) attach();
else document.addEventListener('DOMContentLoaded', attach, { once: true });
(window as unknown as { __hvUnfreeze?: () => void }).__hvUnfreeze = () => {
document.getElementById('__hv_freeze')?.remove();
};
});
ctx.onProgress?.(30, 'loading frame');
// Multi-composition templates ship an entry index.html that only stitches
// sub-scenes via `data-composition-src="compositions/x.html"`; loaded raw
// over file:// the scenes never appear (chromium blocks file:// fetch, so
// the studio's client-side fetch player can't run here). Inline the
// composition files into the HTML up front so chromium records real motion
// instead of an empty shell. Single-file templates pass through untouched.
const prepared = await prepareSourceHtml(input.template.sourcePath);
cleanupSrc = prepared.cleanup;
const fileUrl = pathToFileURL(prepared.loadPath).href;
// Wait only for the DOM + same-document scripts (GSAP, the inline player),
// NOT `load` — `load` blocks on every external asset, and some templates
// reference a cross-origin A-Roll video (e.g. an S3 mp4 with no CORS
// header) that chromium retries for ~4s before giving up. Under `load`
// those ~4s get recorded into the webm as a frozen first scene before the
// timeline ever plays, so the clip opens on several dead seconds. Fonts are
// awaited separately below (document.fonts.ready); GSAP is a synchronous
// <head> script so it's ready at DOMContentLoaded.
await page.goto(fileUrl, { waitUntil: 'domcontentloaded' });
// Wait for all web fonts to finish loading BEFORE recording. Templates
// pull display faces (Shrikhand, Libre Baskerville, Archivo Black, …) from
// Google Fonts with `font-display: swap`, which paints text in a fallback
// system font immediately and swaps in the real face once it downloads.
// If we start recording before the swap, the video shows a visible flash:
// the text renders in the fallback for the first frames, then the glyphs,
// widths and weights snap to the intended font mid-clip.
//
// `document.fonts.ready` alone is NOT enough here, and this was the bug in
// the first cut of this fix. We load the page with `domcontentloaded` (so a
// CORS-blocked A-Roll video can't freeze the opening — see above), which
// means at this point the Google Fonts <link> stylesheet has usually not
// come back yet. Until that CSS arrives, its @font-face rules are not in
// `document.fonts` at all, so `fonts.ready` sees an empty set and resolves
// INSTANTLY — recording starts, then the CSS lands, the faces download, and
// the swap happens mid-clip anyway. So we must, in order:
// 1. wait for every stylesheet <link> to load (or error) — this is what
// actually registers the @font-face rules into document.fonts;
// 2. explicitly fonts.load() each registered face — `display: swap` does
// NOT auto-download a face until something paints with it, and our
// off-screen/pre-animation text may not have triggered that yet;
// 3. then await fonts.ready, plus one rAF, so layout settles on the real
// glyph metrics before frame 0.
// Everything is capped so a slow/blocked font CDN can't stall forever —
// worst case we fall back to the previous behavior for that one frame.
ctx.onProgress?.(32, 'loading fonts');
await page
.evaluate(
() =>
new Promise<void>((resolve) => {
const doc = document as Document & { fonts?: FontFaceSet };
const fonts = doc.fonts;
if (!fonts || typeof fonts.ready?.then !== 'function') {
resolve();
return;
}
let settled = false;
const finish = () => {
if (settled) return;
settled = true;
// One more frame so the relayout on the real face is painted.
requestAnimationFrame(() => requestAnimationFrame(() => resolve()));
};
// Hard cap: a blocked CDN must never stall the render.
const cap = setTimeout(finish, 8000);
// 1. Wait for stylesheet <link>s to load (registers @font-face).
const links = Array.from(
document.querySelectorAll<HTMLLinkElement>('link[rel="stylesheet"]'),
);
const linkDone = links.map((link) => {
// An already-loaded sheet exposes cssRules without throwing.
try {
if (link.sheet && link.sheet.cssRules) return Promise.resolve();
} catch {
/* not ready yet — fall through to event wait */
}
return new Promise<void>((r) => {
const done = () => r();
link.addEventListener('load', done, { once: true });
link.addEventListener('error', done, { once: true });
// Per-link safety so one wedged link can't hold the batch.
setTimeout(done, 6000);
});
});
Promise.all(linkDone)
.then(() => {
// 2. Force every registered face to actually download. Under
// `display: swap` the browser otherwise defers the fetch.
const loads: Promise<unknown>[] = [];
fonts.forEach((face) => {
try {
loads.push(face.load().catch(() => undefined));
} catch {
/* some faces reject load() pre-paint — ignore */
}
});
return Promise.all(loads);
})
// 3. Now ready() reflects the real face set.
.then(() => fonts.ready)
.then(() => {
clearTimeout(cap);
finish();
})
.catch(() => {
clearTimeout(cap);
finish();
});
}),
)
.catch(() => {});
// Pages sometimes set up animations on the load tick — give a frame
// for animations to actually start before we count the duration.
await page.waitForTimeout(100);
// Probe the frame's own animation length so we never cut it off. A short
// per-frame duration set by the user could be < the frame's opening
// animation, truncating it mid-play. Take the longer of the two: the frame
// gets at least as long as its non-looping CSS animations / GSAP timeline.
try {
const animMs = await page.evaluate(() => {
let maxMs = 0;
Array.from(document.querySelectorAll('*')).forEach((el) => {
const s = getComputedStyle(el);
const durs = (s.animationDuration || '').split(',');
const dels = (s.animationDelay || '').split(',');
const iters = (s.animationIterationCount || '').split(',');
durs.forEach((d, i) => {
if ((iters[i] || '').trim() === 'infinite') return; // ignore looping bg anims
maxMs = Math.max(maxMs, ((parseFloat(d) || 0) + (parseFloat(dels[i] || '0') || 0)) * 1000);
});
});
// GSAP: do NOT use globalTimeline.totalDuration() — an infinitely
// repeating tween (repeat:-1, e.g. a blinking cursor) makes it ~1e10s.
// Walk the children and take the longest FINITE (non-repeat:-1) tween.
const g = (window as unknown as {
gsap?: { globalTimeline?: { getChildren?: (b?: boolean, t?: boolean, tl?: boolean) => Array<{ totalDuration?: () => number; repeat?: () => number; vars?: { repeat?: number } }> } };
}).gsap;
let gsapMs = 0;
const children = g?.globalTimeline?.getChildren?.(true, true, true) ?? [];
for (const c of children) {
const repeat = typeof c.repeat === 'function' ? c.repeat() : (c.vars?.repeat ?? 0);
if (repeat === -1) continue; // infinite loop — ignore
const td = typeof c.totalDuration === 'function' ? c.totalDuration() : 0;
if (Number.isFinite(td)) gsapMs = Math.max(gsapMs, td * 1000);
}
return Math.max(maxMs, gsapMs);
});
// +0.4s settle so the final animation frame is actually captured; cap at
// 30s so a stray huge value can't make a frame run away.
const needed = Math.min(30, (animMs + 400) / 1000);
// Only extend when the duration is a soft 'auto' fallback. When the user
// set an explicit per-frame length (multi-frame export), it's a hard cap —
// honoring it keeps "每帧 4s" at 4s instead of letting one long animation
// stretch the frame toward the 30s ceiling.
if (input.config.durationMode !== 'explicit' && needed > totalDuration) {
ctx.onProgress?.(38, `extending to ${needed.toFixed(1)}s for animation`);
totalDuration = needed;
}
} catch { /* probe failed — fall back to the requested duration */ }
// Multi-composition templates register their master timeline paused so the
// probe above can read its real (finite) duration. Now that the recording
// window is fixed, drive playback from frame zero so capture and animation
// start together — otherwise the auto-play fallback would have already run
// part of the timeline before we begin recording.
const drove = await page
.evaluate(() => {
const w = window as unknown as { __hvPlayAll?: () => void; __hvPlayed?: boolean };
if (typeof w.__hvPlayAll === 'function') {
w.__hvPlayed = true;
w.__hvPlayAll();
return true;
}
return false;
})
.catch(() => false);
// Release the animation freeze now that fonts are ready. Every template —
// single-file CSS keyframes and multi-composition GSAP alike — has been
// held at frame 0 since before goto(), so the entire recorded lead-in
// (page load + cold font fetch, ~2–4s) is a still hold of the first frame
// with no motion. Unfreezing here is the true t=0 of the animation, so the
// lead-in is always dead and always safe to trim. (Previously only
// multi-composition templates were parked, so single-file ones recorded
// their opening during the font wait and showed the fallback-font flash.)
await page
.evaluate(() => {
(window as unknown as { __hvUnfreeze?: () => void }).__hvUnfreeze?.();
})
.catch(() => {});
leadInMs = Date.now() - tWebmStart;
void drove; // playback already driven above for multi-composition timelines
ctx.onProgress?.(40, `recording ${totalDuration}s`);
// Stream a single coarse progress tick per second so the user sees
// "recording 1/5s …" type signal in the studio progress bar.
const totalMs = Math.round(totalDuration * 1000);
const tick = 250;
const start = Date.now();
while (Date.now() - start < totalMs) {
if (ctx.signal?.aborted) throw new HtmlVideoError('cancelled', 'Aborted');
await page.waitForTimeout(Math.min(tick, totalMs - (Date.now() - start)));
const pct = 40 + Math.floor(((Date.now() - start) / totalMs) * 45);
ctx.onProgress?.(pct, 'recording');
}
ctx.onProgress?.(85, 'finalising recording');
await context.close();
// playwright drops the webm into recordDir; pick the freshest .webm
const candidates = readdirSync(recordDir).filter((f) => f.endsWith('.webm'));
if (candidates.length === 0) {
throw new HtmlVideoError('render-failed', `Playwright produced no webm in ${recordDir}`);
}
candidates.sort();
webmPath = join(recordDir, candidates[candidates.length - 1]!);
} finally {
if (browser) await browser.close().catch(() => {});
if (cleanupSrc) await cleanupSrc().catch(() => {});
}
// ---- ffmpeg: webm → mp4 ----
ctx.onProgress?.(90, 'encoding mp4');
// Trim the dead lead-in (page load + font fetch before the timeline played)
// off the front of multi-composition webms. Back off 120ms so rounding /
// recorder start jitter can't clip the first real animation frame — a couple
// of still frames at the head are harmless, a missing opening beat is not.
const seekSec = leadInMs > 200 ? Math.max(0, (leadInMs - 120) / 1000) : 0;
// When the user set an explicit per-frame length, the output must be EXACTLY
// that long. The recorded webm can come up a little short (recorder start
// jitter, the lead-in trim, a sub-duration animation that finished early), so
// pad the tail by holding the last frame (`tpad stop_mode=clone`) up to the
// target. -t then trims to the precise length. For 'auto' we keep the old
// behavior (just -t, no padding) — there the duration is a soft fallback.
const explicit = input.config.durationMode === 'explicit';
await runFfmpeg([
'-y',
// -ss before -i = fast input seek, drops the frozen lead-in entirely.
...(seekSec > 0 ? ['-ss', seekSec.toFixed(3)] : []),
'-i', webmPath!,
// Pad-then-trim so an explicit per-frame length lands exactly (e.g. user
// asked 4s, animation ran 2.8s → hold the final frame to fill 4s).
...(explicit ? ['-vf', `tpad=stop_mode=clone:stop_duration=${totalDuration}`] : []),
// Force exact duration: playwright's recordVideo sometimes overshoots
// by the time it takes to close the context. -t trims to the requested
// length (seconds, accepts fractions).
'-t', String(totalDuration),
'-r', String(fps),
'-c:v', 'libx264',
'-pix_fmt', 'yuv420p',
'-preset', 'medium',
'-crf', '20',
'-movflags', '+faststart',
input.config.outputPath,
]);
// Clean tmp dir
await rm(recordDir, { recursive: true, force: true }).catch(() => {});
const st = await stat(input.config.outputPath);
ctx.onProgress?.(100, 'done');
return {
outputPath: input.config.outputPath,
meta: {
durationSec: totalDuration,
fileSizeBytes: st.size,
actualResolution: input.config.resolution,
fps,
renderedFrames: Math.round(totalDuration * fps),
renderWallClockSec: (Date.now() - t0) / 1000,
engineVersion: `hyperframes-playwright@${ADAPTER_VERSION}`,
},
diagnostics: [`recorded via playwright/chromium then encoded with ffmpeg (libx264 crf20)`],
};
}
function runFfmpeg(args: string[]): Promise<void> {
return new Promise((resolve, reject) => {
const proc = spawn('ffmpeg', args, { stdio: ['ignore', 'pipe', 'pipe'] });
let stderr = '';
proc.stderr.on('data', (chunk: Buffer) => {
stderr += chunk.toString('utf8');
});
proc.on('error', (err: NodeJS.ErrnoException) => {
if (err.code === 'ENOENT') {
reject(new HtmlVideoError('render-failed',
'ffmpeg not found on PATH. Install with `brew install ffmpeg` (macOS).'));
} else reject(err);
});
proc.on('exit', (code) => {
if (code === 0) resolve();
else reject(new HtmlVideoError(
'render-failed',
`ffmpeg exited ${code}: ${stderr.slice(-2000)}`,
));
});
});
}
/**
* Resolve the HTML to actually load into chromium.
*
* Single-file templates load as-is. Multi-composition templates declare their
* scenes as `<div data-composition-src="compositions/x.html">` placeholders;
* each composition file is a `<template>` wrapping markup + <style> + a <script>
* that registers a paused GSAP timeline on `window.__timelines[name]`. The
* studio preview assembles these client-side via fetch — but chromium blocks
* file:// fetch, so over file:// the scenes would never appear.
*
* This reads each composition file on the Node side, inlines them into a
* `window.__COMPOSITIONS__` map, and injects a player that grafts each
* `<template>.content` into its placeholder and re-executes the composition
* scripts (cloned <script> nodes never run on their own). The result is a
* self-contained HTML written next to the source (so sibling relative assets
* still resolve) and loaded over file://. Returns a cleanup() to remove it.
*/
async function prepareSourceHtml(
sourcePath: string,
): Promise<{ loadPath: string; cleanup?: () => Promise<void> }> {
const raw = await readFile(sourcePath, 'utf8');
const srcMatches = Array.from(raw.matchAll(/data-composition-src=["']([^"']+)["']/g));
if (srcMatches.length === 0) return { loadPath: sourcePath };
const srcDir = dirname(sourcePath);
const compMap: Record<string, string> = {};
for (const m of srcMatches) {
const rel = m[1]!;
if (compMap[rel] !== undefined) continue;
const compPath = join(srcDir, rel);
if (!existsSync(compPath)) continue;
compMap[rel] = await readFile(compPath, 'utf8');
}
if (Object.keys(compMap).length === 0) return { loadPath: sourcePath };
// Escape `</` (and the comment opener) so the JSON survives the inline
// <script> context — composition files contain their own </script> tags.
const safeJson = JSON.stringify(compMap).replace(/<\//g, '<\\/').replace(/<!--/g, '<\\!--');
let out = raw
.replace(/__VIDEO_DURATION__/g, '15')
.replace(/__VIDEO_SRC__/g, 'data:video/mp4;base64,');
// Seed the timeline registry in <head> so the entry's own early
// `window.__timelines["x"] = …` assignments don't throw on undefined.
const head = `<script>window.__timelines=window.__timelines||{};window.__COMPOSITIONS__=${safeJson};</script>`;
out = /<head[^>]*>/i.test(out)
? out.replace(/<head[^>]*>/i, (mm) => `${mm}\n${head}`)
: `${head}\n${out}`;
const player = `
<script>
(function () {
function reexec(root) {
root.querySelectorAll('script').forEach(function (old) {
if (old.src) { old.parentNode.removeChild(old); return; }
var s = document.createElement('script');
// Wrap each composition's inline script in a block so top-level
// \`const tl = …\` locals don't collide across scenes; the
// window.__timelines assignments still escape the block.
s.textContent = '{\\n' + old.textContent + '\\n}';
old.parentNode.replaceChild(s, old);
});
}
function mountOne(host) {
var src = host.getAttribute('data-composition-src');
var text = (window.__COMPOSITIONS__ || {})[src];
if (!text) return;
var holder = document.createElement('div');
holder.innerHTML = text;
var tpl = holder.querySelector('template');
host.appendChild(tpl ? tpl.content.cloneNode(true) : holder);
reexec(host);
}
// Play every registered timeline once from the start. Do NOT force
// repeat(-1): these composition timelines are finite, scene-by-scene
// narratives (e.g. kinetic-type is a 14.7s master timeline that wipes
// through 6 scenes). Looping them broke two things — it replayed the intro
// over the outro, and the renderer's duration probe SKIPS repeat:-1 tweens
// as "infinite background anim", so a looped master timeline read as 0s and
// the clip got truncated to the default 5s. Leaving them finite lets the
// probe see the real 14.7s and record the whole story.
window.__hvPlayAll = function () {
var tls = window.__timelines || {};
Object.keys(tls).forEach(function (k) {
var tl = tls[k];
if (tl && typeof tl.play === 'function') tl.play(0);
});
};
function boot() {
window.__timelines = window.__timelines || {};
Array.prototype.slice
.call(document.querySelectorAll('[data-composition-src]'))
.forEach(mountOne);
// The composition <script>s register their (paused) timelines synchronously
// as they're injected, so they're on window.__timelines now. Leave them
// paused here — the renderer probes their duration first, then calls
// window.__hvPlayAll() at the exact moment recording starts so playback and
// capture are aligned. If no driver calls it (e.g. opened standalone), fall
// back to auto-playing shortly after load.
setTimeout(function () { if (!window.__hvPlayed) window.__hvPlayAll(); }, 250);
}
if (document.readyState === 'loading') {
document.addEventListener('DOMContentLoaded', boot);
} else { boot(); }
})();
</script>`;
out = out.includes('</body>') ? out.replace('</body>', `${player}\n</body>`) : out + player;
const loadPath = join(srcDir, `.hv-render-${Date.now()}.html`);
await writeFile(loadPath, out, 'utf8');
return {
loadPath,
cleanup: async () => {
await rm(loadPath, { force: true }).catch(() => {});
},
};
}
/**
* Render template to a single HTML preview.
*
* v0.1: read the source HTML file (a Hyperframes template is HTML+CSS+JS),
* inject a banner showing the variables, copy referenced assets, write to ctx.workDir.
* Real upstream Hyperframes integration will replace the inject + add a frame-bound clock.
*/
export async function renderToHtml(
input: RenderInput,
ctx: RenderContext,
): Promise<HtmlSceneOutput> {
if (!existsSync(input.template.sourcePath)) {
throw new HtmlVideoError(
'template-invalid',
`Source not found: ${input.template.sourcePath}`,
);
}
await mkdir(ctx.workDir, { recursive: true });
const htmlPath = join(ctx.workDir, 'preview.html');
const posterPath = join(ctx.workDir, 'poster.svg');
const sourceHtml = await readFile(input.template.sourcePath, 'utf8');
const augmented = sourceHtml.replace(
'</body>',
`<script>
window.__HV_VARS__ = ${JSON.stringify(input.variables)};
window.__HV_DURATION__ = ${typeof input.config.duration === 'number' ? input.config.duration : 5};
console.log('html-video preview vars', window.__HV_VARS__);
</script></body>`,
);
await writeFile(htmlPath, augmented, 'utf8');
// Cheap poster: an SVG placeholder we draw ourselves (no headless chromium yet).
const { width, height } = input.config.resolution;
const title = String(input.variables.title ?? input.template.id);
const poster = `<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 ${width} ${height}" width="${width}" height="${height}">
<rect width="100%" height="100%" fill="#1a1a1a"/>
<text x="50%" y="50%" fill="#eee" font-family="Inter, system-ui, sans-serif"
font-size="72" text-anchor="middle" dominant-baseline="middle">${escapeXml(title)}</text>
<text x="50%" y="${height - 80}" fill="#888" font-family="monospace" font-size="32"
text-anchor="middle">hyperframes · ${input.template.id}</text>
</svg>`;
await writeFile(posterPath, poster, 'utf8');
// Copy any referenced asset files mentioned in variables (best-effort)
const referencedAssets: { assetId: string; usagePath: string }[] = [];
for (const v of Object.values(input.variables)) {
if (typeof v !== 'string') continue;
if (!v.includes('/.html-video/bundles/')) continue;
if (!existsSync(v)) continue;
const dest = join(ctx.workDir, 'assets', v.split('/').pop() ?? 'asset');
await mkdir(dirname(dest), { recursive: true });
if (!existsSync(dest)) await copyFile(v, dest);
const m = /assets\/([0-9a-f]{40})\./.exec(v);
if (m && m[1]) {
referencedAssets.push({ assetId: m[1], usagePath: dest });
}
}
const totalDuration =
input.config.duration === 'auto' ? 5 : input.config.duration;
return {
htmlPath,
referencedAssets,
posterPath,
durationSec: totalDuration,
};
}
function escapeXml(s: string): string {
return s.replace(/[&<>"']/g, (c) => {
const map: Record<string, string> = {
'&': '&amp;',
'<': '&lt;',
'>': '&gt;',
'"': '&quot;',
"'": '&apos;',
};
return map[c] ?? c;
});
}
// silence unused imports warning until real impl uses them
void stat;
packages/adapter-hyperframes/src/validate.ts
+49
View File
@@ -0,0 +1,49 @@
import { existsSync } from 'node:fs';
import type { TemplateRef, ValidationError, ValidationResult } from '@html-video/core';
import { capabilities } from './capabilities.js';
/**
* Validate a template against the Hyperframes adapter capabilities.
* Cheap & read-only per RFC-01.
*/
export function validate(template: TemplateRef): ValidationResult {
const errors: ValidationError[] = [];
const warnings: ValidationError[] = [];
if (template.engine !== 'hyperframes') {
errors.push({
code: 'engine-mismatch',
message: `Template engine "${template.engine}" is not "hyperframes"`,
fix: `Use the @html-video/adapter-${template.engine} adapter instead`,
});
return { ok: false, errors, warnings };
}
if (!template.sourcePath) {
errors.push({
code: 'missing-source',
message: 'Template has no sourcePath',
});
} else if (!existsSync(template.sourcePath)) {
errors.push({
code: 'source-not-found',
message: `Template source not found: ${template.sourcePath}`,
fix: 'Check that the template directory contains the file declared in source_entry',
});
}
// Sanity: max resolution from caps
const cap = capabilities;
if (cap.maxResolution.width < 1920) {
warnings.push({
code: 'low-max-resolution',
message: `Adapter max resolution is ${cap.maxResolution.width}x${cap.maxResolution.height}`,
});
}
return {
ok: errors.length === 0,
errors,
warnings,
};
}
packages/adapter-hyperframes/tsconfig.json
+9
View File
@@ -0,0 +1,9 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "dist",
"noEmit": false
},
"include": ["src/**/*"]
}
packages/adapter-remotion/package.json
+41
View File
@@ -0,0 +1,41 @@
{
"name": "@html-video/adapter-remotion",
"version": "0.1.0",
"description": "Remotion adapter for html-video meta-layer — deterministic frame-by-frame rendering, the second real engine. Phase 1 bridges existing HTML/CSS/GSAP frames onto Remotion's timeline; Phase 2 (separate RFC) adds native React-tsx templates.",
"license": "Apache-2.0",
"type": "module",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
".": {
"import": "./dist/index.js",
"types": "./dist/index.d.ts"
}
},
"files": ["dist", "bridge", "README.md"],
"scripts": {
"build": "tsc -p tsconfig.json",
"typecheck": "tsc -p tsconfig.json --noEmit",
"test": "npm run build && node --test \"test/**/*.test.mjs\""
},
"dependencies": {
"@html-video/core": "workspace:*"
},
"peerDependencies": {
"remotion": "^4.0.0",
"@remotion/bundler": "^4.0.0",
"@remotion/renderer": "^4.0.0",
"react": "^18 || ^19",
"react-dom": "^18 || ^19"
},
"peerDependenciesMeta": {
"remotion": { "optional": true },
"@remotion/bundler": { "optional": true },
"@remotion/renderer": { "optional": true },
"react": { "optional": true },
"react-dom": { "optional": true }
},
"devDependencies": {
"typescript": "^5.7.2"
}
}
packages/adapter-remotion/src/bridge/HtmlFrameDriver.tsx
+154
View File
@@ -0,0 +1,154 @@
// HtmlFrameDriver — RFC-08 §3.2 "time-driver injection".
//
// Renders an existing html-video HTML frame (passed inline as `html`) inside a
// Remotion composition via iframe `srcdoc`, and keeps its CSS / GSAP animation
// in sync with Remotion's deterministic frame clock. Without this, Remotion
// freezes its clock at frame N, screenshots, jumps to N+1 — but the iframe's CSS
// keyframes / GSAP run on the browser's own wall-clock, so each screenshot
// catches the animation at a random real-time point → flicker. We pause the
// iframe's clock and seek every animation to Remotion's current time per frame.
//
// Not compiled by the adapter's tsc; it's a static asset handed to Remotion's
// bundle() (webpack understands the JSX).
import React, { useCallback, useEffect, useRef, useState } from 'react';
import { useCurrentFrame, useVideoConfig, delayRender, continueRender } from 'remotion';
export type HtmlFrameDriverProps = {
/** The HTML frame's full source, inlined and rendered via iframe srcdoc. */
html: string;
width: number;
height: number;
};
type GsapTimeline = { pause: () => void; time: (s?: number) => unknown };
type GsapWindow = Window & { gsap?: { globalTimeline?: GsapTimeline } };
/** Read the iframe's content document, or undefined if not reachable yet. */
function frameDoc(iframe: HTMLIFrameElement | null): Document | undefined {
try {
return iframe?.contentWindow?.document ?? undefined;
} catch {
return undefined; // cross-origin (shouldn't happen for srcdoc)
}
}
/**
* The srcdoc document is "ready to seek + screenshot" only when BOTH hold:
*
* 1. Content is parsed — `readyState === 'complete'` is NOT enough (an empty
* doc reports complete too, and seeking it silently no-ops). We require a
* populated body or ≥1 registered animation.
*
* 2. Web fonts / external stylesheets have settled — `doc.fonts.status ===
* 'loaded'`. This is the one that actually caused the all-black render: a
* template with `<link href="fonts.googleapis.com/...">` keeps the iframe's
* render tree from painting until that external CSS resolves, so Remotion
* screenshots a fully black frame even though the DOM (opacity, etc.) is
* already correct. Waiting on fonts also kills FOUT (same idea as the
* hyperframes-side font-freeze fix). `fonts.status` starts 'loading' while
* any face/stylesheet is pending and flips to 'loaded' when all settle; a
* doc with no web fonts reports 'loaded' immediately.
*/
function docReady(doc: Document | undefined): boolean {
if (!doc || !doc.body) return false;
const hasAnims = (doc.getAnimations?.()?.length ?? 0) > 0;
const hasBody = doc.body.innerHTML.trim().length > 0;
if (!hasAnims && !hasBody) return false;
// doc.fonts may be undefined in exotic engines — treat absence as ready.
const fontsLoaded = (doc as Document & { fonts?: FontFaceSet }).fonts?.status !== 'loading';
return fontsLoaded;
}
export const HtmlFrameDriver: React.FC<HtmlFrameDriverProps> = ({ html, width, height }) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const iframeRef = useRef<HTMLIFrameElement | null>(null);
const tMs = (frame / fps) * 1000;
const seek = useCallback((timeMs: number) => {
const win = iframeRef.current?.contentWindow as GsapWindow | null | undefined;
const doc = frameDoc(iframeRef.current);
if (!win || !doc) return;
// (A) CSS Animations / Web Animations API — pure CSS @keyframes.
try {
const anims = (doc.getAnimations?.() ?? []) as Animation[];
for (const a of anims) {
try {
a.pause();
a.currentTime = timeMs;
} catch {
/* idle/finished animation rejects currentTime — ignore */
}
}
} catch {
/* getAnimations unsupported — fall through */
}
// (B) GSAP global timeline.
try {
const tl = win.gsap?.globalTimeline;
if (tl) {
tl.pause();
tl.time(timeMs / 1000);
}
} catch {
/* no gsap — fine */
}
}, []);
// Per-frame delayRender: Remotion captures each frame in its own pass, so the
// bridge must hold *each* frame open until (1) the srcdoc document has truly
// loaded its animations and (2) we've seeked them to this frame's time. The
// earlier single-handle-on-mount version only synced the first frame; every
// later frame screenshotted before the async re-seek landed → black output.
//
// We create a fresh handle for the current `tMs`, poll until the doc is ready,
// seek, then continueRender. A short rAF settle lets the seeked styles paint
// before Remotion screenshots. The handle is keyed to tMs so a new frame can't
// clear a stale one.
const [handle] = useState(() => delayRender(`HTML frame seek @0ms`));
const firstClearedRef = useRef(false);
useEffect(() => {
let cancelled = false;
// For the very first frame we reuse the mount-time handle; subsequent frames
// open their own so Remotion waits for each re-seek.
const h = firstClearedRef.current ? delayRender(`HTML frame seek @${Math.round(tMs)}ms`) : handle;
let tries = 0;
const finish = () => {
if (cancelled) return;
seek(tMs);
// One rAF so the seeked computed styles are committed before screenshot.
requestAnimationFrame(() => {
if (cancelled) return;
firstClearedRef.current = true;
continueRender(h);
});
};
const tick = () => {
if (cancelled) return;
if (docReady(frameDoc(iframeRef.current)) || tries++ >= 200) {
finish();
} else {
setTimeout(tick, 25);
}
};
tick();
return () => {
cancelled = true;
};
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [tMs, seek]);
return (
<iframe
ref={iframeRef}
srcDoc={html}
width={width}
height={height}
style={{ width, height, border: 'none', display: 'block' }}
sandbox="allow-same-origin allow-scripts"
/>
);
};
packages/adapter-remotion/src/bridge/Root.tsx
+29
View File
@@ -0,0 +1,29 @@
// Root — registers the single generic bridge composition. render.ts overrides
// width/height/fps/durationInFrames per call via selectComposition(), and passes
// the HTML frame path + size through inputProps. (RFC-08 §5)
import React from 'react';
import { Composition } from 'remotion';
import { HtmlFrameDriver, type HtmlFrameDriverProps } from './HtmlFrameDriver';
const DEFAULTS: HtmlFrameDriverProps = { html: '<!doctype html><html><body></body></html>', width: 1920, height: 1080 };
export const RemotionRoot: React.FC = () => {
return (
<Composition
id="HtmlFrame"
component={HtmlFrameDriver}
// Placeholder metadata; render.ts overrides all of these at render time.
durationInFrames={150}
fps={30}
width={1920}
height={1080}
defaultProps={DEFAULTS}
// Let inputProps.width/height drive the canvas so a 9:16 / 1:1 frame isn't
// squashed into 16:9. duration/fps come from selectComposition overrides.
calculateMetadata={({ props }) => ({
width: props.width ?? 1920,
height: props.height ?? 1080,
})}
/>
);
};
packages/adapter-remotion/src/bridge/entry.ts
+6
View File
@@ -0,0 +1,6 @@
// Bundle entry point — passed to @remotion/bundler bundle(). registerRoot wires
// the bridge composition into Remotion. (RFC-08 §5)
import { registerRoot } from 'remotion';
import { RemotionRoot } from './Root';
registerRoot(RemotionRoot);
packages/adapter-remotion/src/bridge/public/frame-index.html
+99
View File
@@ -0,0 +1,99 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>Glitch Title · SIGNAL_LOST</title>
<script src="https://cdn.tailwindcss.com"></script>
<link href="https://fonts.googleapis.com/css2?family=Space+Grotesk:wght@500;700;800&family=JetBrains+Mono:wght@400;700&display=swap" rel="stylesheet" />
<style>
body { font-family:'Space Grotesk',sans-serif; background:#0d0e10; color:#f5f5f7; margin:0; min-height:100vh; overflow:hidden; position:relative; }
.mono { font-family:'JetBrains Mono',monospace; }
/* scanlines */
body::before {
content:''; position:absolute; inset:0; pointer-events:none; z-index:3;
background-image: repeating-linear-gradient(0deg, rgba(0,0,0,0.18) 0px, rgba(0,0,0,0.18) 1px, transparent 1px, transparent 3px);
mix-blend-mode: multiply; opacity:0.6;
}
/* grid */
body::after {
content:''; position:absolute; inset:0; pointer-events:none; z-index:2;
background-image: linear-gradient(rgba(0,255,180,0.04) 1px, transparent 1px), linear-gradient(90deg,rgba(0,255,180,0.04) 1px, transparent 1px);
background-size: 56px 56px;
}
/* grain */
.grain { position:absolute; inset:0; opacity:0.1; mix-blend-mode:overlay; z-index:4; pointer-events:none;
background-image: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='200' height='200'%3E%3Cfilter id='n'%3E%3CfeTurbulence baseFrequency='0.9' numOctaves='2'/%3E%3C/filter%3E%3Crect width='200' height='200' filter='url(%23n)'/%3E%3C/svg%3E"); }
@keyframes glitch {
0%, 92%, 100% { transform: translate(0,0); filter: none }
93% { transform: translate(-6px, 1px); filter: url(#rgb) }
94% { transform: translate(8px, -2px); filter: url(#rgb) }
95% { transform: translate(-4px, 2px); }
96% { transform: translate(4px, -1px); filter: url(#rgb) }
97% { transform: translate(0,0) }
}
.glitch-host { position:relative; animation: glitch 4s infinite; }
.layer { position:absolute; inset:0; mix-blend-mode:screen; }
.layer-c { color:#00f0ff; transform: translate(-3px, 1px); }
.layer-m { color:#ff2bd6; transform: translate(3px, -1px); }
.vignette { position:absolute; inset:0; background: radial-gradient(circle at center, transparent 50%, rgba(0,0,0,0.7) 100%); z-index:5; pointer-events:none; }
.ascii { font-family:'JetBrains Mono',monospace; letter-spacing:2px; line-height:1; }
@media (prefers-reduced-motion: reduce) { .glitch-host { animation: none } }
</style>
</head>
<body class="flex flex-col items-center justify-center relative">
<svg width="0" height="0" style="position:absolute">
<defs>
<filter id="rgb">
<feColorMatrix type="matrix" values="1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0"/>
<feOffset in="SourceGraphic" dx="3" dy="0" result="r"/>
<feOffset in="SourceGraphic" dx="-3" dy="0" result="b"/>
<feMerge><feMergeNode in="r"/><feMergeNode in="b"/><feMergeNode in="SourceGraphic"/></feMerge>
</filter>
</defs>
</svg>
<header class="absolute top-10 left-12 right-12 flex items-baseline justify-between mono text-[11px] opacity-70 z-10 tracking-[0.16em]">
<span>&gt;&gt; SIGNAL_LOST · CH-04 · 14:32:08</span>
<span>FRAME · GLITCH-TITLE</span>
<span>REC ●</span>
</header>
<!-- ASCII chunks corners -->
<pre class="ascii absolute top-24 left-12 text-[12px] opacity-30 z-10">█▓▒░ █▓▒░
▒▓█▓ ░▒▓
░▒▓█ ▓▒░░
▓▒░░ ▒▓█▓</pre>
<pre class="ascii absolute bottom-24 right-12 text-[12px] opacity-30 z-10 text-right">█▓▒░ ▓▒░█
▒░░▓ ░▒▓█
▓▒░░ █▓▒░
░▒▓█ ▒░░▓</pre>
<main class="relative z-10 text-center px-12">
<div class="mono text-[14px] opacity-60 tracking-[0.22em] mb-6 uppercase">— Transmission Halted —</div>
<div class="glitch-host relative inline-block">
<h1 class="font-black tracking-[-0.02em] leading-[0.9]" style="font-size:clamp(80px,12vw,200px)">
SIGNAL<br/>_LOST
</h1>
<h1 class="layer layer-c font-black tracking-[-0.02em] leading-[0.9]" style="font-size:clamp(80px,12vw,200px)">
SIGNAL<br/>_LOST
</h1>
<h1 class="layer layer-m font-black tracking-[-0.02em] leading-[0.9]" style="font-size:clamp(80px,12vw,200px)">
SIGNAL<br/>_LOST
</h1>
</div>
<p class="mt-10 mono text-[16px] opacity-70 tracking-[0.16em]">
04 · DATA_CORRUPTION · <span style="color:#ffb547">w̶̶a̷i̶t̶i̶n̶g̶ ̷f̷o̶r̶ ̶r̷e̶c̶o̶v̷e̶r̶y̶</span>
</p>
</main>
<div class="grain"></div>
<div class="vignette"></div>
<footer class="absolute bottom-10 left-12 right-12 flex items-baseline justify-between mono text-[11px] opacity-50 z-10 tracking-[0.16em]">
<span>HTML-ANYTHING / VFX</span>
<span style="color:#ff2bd6">CHROMATIC · CYAN × MAGENTA</span>
<span class="mono">⏵ 00:14:32:08</span>
</footer>
</body>
</html>
packages/adapter-remotion/src/capabilities.ts
+48
View File
@@ -0,0 +1,48 @@
import type { EngineCapabilities } from '@html-video/core';
/**
* Static capability declaration for Remotion (RFC-08 §1, cross-checked
* 2026-06-06 against remotion@4.0.x).
*
* `licensing: 'commercial-restricted'` is a FIRST-CLASS field here, not a
* footnote: Remotion is free for individuals / companies ≤3 people / non-profit,
* but 4+ person teams pay. The agent reads this to steer "free / cheap" jobs
* toward the free-osi engines (hyperframes / revideo) and only suggest Remotion
* when the team already uses React or needs Lambda scale. That honesty is the
* meta-layer's advantage over a Remotion-only tool — surface it, don't hide it.
*/
export const capabilities: EngineCapabilities = {
// Phase 1 bridges html-css-gsap frames; Phase 2 (RFC-09) adds react-tsx natives.
paradigms: ['react-tsx', 'html-css-gsap'],
outputFormats: ['mp4', 'webm', 'gif', 'png-sequence'],
// Remotion is bounded by Chrome's 2^29-px single-image cap; 8K frames are fine.
maxResolution: { width: 7680, height: 4320 },
alpha: true,
audio: 'multi',
subtitles: ['burn-in', 'sidecar'],
// Deterministic local chromium, and one-line scale-out to Lambda / Cloud Run.
renderTarget: ['local-chromium', 'lambda', 'cloud-run'],
licensing: 'commercial-restricted',
renderSpeedHint: {
resolution: '1080p',
durationSec: 10,
fps: 30,
// Deterministic frame-by-frame is slower than realtime screen capture but
// never drops frames; rough order-of-magnitude only.
estimatedRenderSec: 40,
},
bestFor: [
'data-driven',
'long-form-narration',
'spring-physics',
'parametric-batch',
'lambda-scale',
'react-ecosystem',
],
weaknesses: [
'commercial-license-for-teams',
'slower-than-realtime-capture',
'bundle-overhead',
'css-gsap-needs-time-driver-bridge',
],
};
packages/adapter-remotion/src/index.ts
+16
View File
@@ -0,0 +1,16 @@
import type { EngineAdapter } from '@html-video/core';
import { capabilities } from './capabilities.js';
import { validate, remotionInstalled } from './validate.js';
import { render } from './render.js';
const adapter: EngineAdapter = {
id: 'remotion',
name: 'Remotion',
upstreamVersion: '4.x',
capabilities,
validate,
render,
};
export default adapter;
export { adapter, remotionInstalled };
packages/adapter-remotion/src/render.ts
+242
View File
@@ -0,0 +1,242 @@
// Remotion adapter render() — RFC-08 §5.
//
// bundle() once (cached) → selectComposition() with per-call metadata overrides
// → renderMedia() into a tmp file → rename to the final path. The HTML frame is
// copied into the bundle's public/ dir and handed to the bridge composition as
// inputProps.htmlSrc; the bridge keeps its CSS/GSAP animation in sync with
// Remotion's frame clock (HtmlFrameDriver.tsx).
import { createRequire } from 'node:module';
import { fileURLToPath } from 'node:url';
import { dirname, join, resolve } from 'node:path';
import { mkdtemp, mkdir, copyFile, readFile, rename, rm, stat } from 'node:fs/promises';
import { existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import type { RenderInput, RenderContext, RenderOutput } from '@html-video/core';
const require = createRequire(import.meta.url);
const here = dirname(fileURLToPath(import.meta.url));
// Minimal structural types for the Remotion APIs we touch (peer deps, may be absent).
type Bundler = (opts: { entryPoint: string; publicDir?: string }) => Promise<string>;
type Renderer = {
selectComposition: (opts: {
serveUrl: string;
id: string;
inputProps?: Record<string, unknown>;
}) => Promise<{ id: string; width: number; height: number; fps: number; durationInFrames: number }>;
renderMedia: (opts: Record<string, unknown>) => Promise<unknown>;
};
class EngineError extends Error {
constructor(public code: string, message: string) {
super(message);
this.name = 'EngineError';
}
}
function loadRemotion(): { bundle: Bundler; renderer: Renderer } {
try {
const { bundle } = require('@remotion/bundler') as { bundle: Bundler };
const renderer = require('@remotion/renderer') as Renderer;
return { bundle, renderer };
} catch (err) {
throw new EngineError(
'engine-not-installed',
`Remotion is not installed. Run \`pnpm add remotion @remotion/bundler @remotion/renderer react react-dom\` in the workspace root. (${err instanceof Error ? err.message : err})`,
);
}
}
// bundle() is an expensive webpack build — cache it across multi-frame renders.
// Keyed by entry path so the bridge entry AND each native template entry each
// bundle once and are reused across frames in one process. A video that enhances
// three data frames with the same native template bundles it once, not thrice.
// (RFC-08 §5 "bundle 复用" — generalized for Phase 2 native templates.)
const bundleCache = new Map<string, string>(); // entryPath -> serveUrl
async function bundleOnce(bundle: Bundler, entry: string, ctx: RenderContext): Promise<string> {
const cached = bundleCache.get(entry);
if (cached) return cached;
ctx.onProgress?.(15, 'bundling');
const serveUrl = await bundle({ entryPoint: entry });
bundleCache.set(entry, serveUrl);
return serveUrl;
}
/** Locate bridge/entry — present in src during dev (ts-node/tsx) and copied to
* the package root `bridge/` when published. We ship the .tsx; Remotion bundles it. */
function bridgeEntry(): string {
const candidates = [
join(here, 'bridge', 'entry.ts'), // dist/bridge if ever emitted
join(here, '..', 'src', 'bridge', 'entry.ts'), // dist/ -> src/bridge (dev)
join(here, '..', 'bridge', 'entry.ts'), // published package `bridge/`
];
for (const c of candidates) if (existsSync(c)) return c;
throw new EngineError('render-failed', `bridge entry not found (looked in: ${candidates.join(', ')})`);
}
/**
* Make an HTML frame safe to render inside Remotion's headless chromium iframe.
*
* A render-blocking external `<link rel="stylesheet">` (e.g. Google Fonts) keeps
* the iframe's render tree from painting until the stylesheet resolves. In the
* headless render environment that request is slow or unreachable, so it never
* resolves and Remotion screenshots a fully black frame — even though the DOM is
* correct. (Same family as the file:// fetch issue tracked in #16/#18.)
*
* Video rendering must be deterministic and offline-safe, so we do NOT gamble on
* a network font load: we convert blocking external stylesheet links into
* non-blocking async loads (media="print" + onload swap). The font applies if it
* arrives in time; if not, the CSS `font-family` fallback (templates declare one,
* e.g. `'Archivo Black', sans-serif`) renders instead. Either way paint is never
* blocked. Inline <style> and same-document CSS are untouched.
*/
export function neutralizeBlockingResources(html: string): string {
return html.replace(
/<link\b[^>]*\brel=(["']?)stylesheet\1[^>]*>/gi,
(tag) => {
// Only neutralize links to an external origin (http/https/protocol-relative).
if (!/\bhref=(["'])(?:https?:)?\/\//i.test(tag)) return tag;
if (/\bmedia=/i.test(tag)) return tag; // already has media handling — leave it
// Load async (media="print" doesn't block paint), then swap to "all" onload.
return tag.replace(
/\s*\/?>$/,
` media="print" onload="this.media='all'">`,
);
},
);
}
/**
* Shared select→render→atomic-rename, used by both the bridge and native paths.
* Selects `compositionId` on the bundle, overrides its metadata per call, renders
* to a tmp file, then renames into place (RFC-01 immutability).
*/
async function renderComposition(args: {
renderer: Renderer;
serveUrl: string;
compositionId: string;
inputProps: Record<string, unknown>;
width: number;
height: number;
fps: number;
durationInFrames: number;
config: RenderInput['config'];
ctx: RenderContext;
}): Promise<void> {
const { renderer, serveUrl, compositionId, inputProps, width, height, fps, durationInFrames, config, ctx } = args;
if (ctx.signal?.aborted) throw new EngineError('cancelled', 'Aborted');
// --- select composition (override metadata per call) ---
ctx.onProgress?.(30, 'selecting composition');
const composition = await renderer.selectComposition({ serveUrl, id: compositionId, inputProps });
const comp = { ...composition, width, height, fps, durationInFrames };
// --- render to tmp, then atomic rename (RFC-01 immutability) ---
const tmpDir = await mkdtemp(join(tmpdir(), 'hv-remotion-'));
const tmpOut = join(tmpDir, `out.${config.format === 'webm' ? 'webm' : 'mp4'}`);
const codec = config.format === 'webm' ? 'vp8' : config.format === 'gif' ? 'gif' : 'h264';
try {
await renderer.renderMedia({
composition: comp,
serveUrl,
codec,
outputLocation: tmpOut,
inputProps,
onProgress: ({ progress }: { progress: number }) => {
// renderMedia 0..1 → our 30..90 band.
ctx.onProgress?.(30 + Math.round(progress * 60), 'rendering');
},
});
ctx.onProgress?.(92, 'finalising');
await rename(tmpOut, config.outputPath).catch(async () => {
// cross-device rename fails → copy fallback
await copyFile(tmpOut, config.outputPath);
});
} catch (err) {
if (ctx.signal?.aborted) throw new EngineError('cancelled', 'Aborted');
throw new EngineError('render-failed', `Remotion renderMedia failed: ${err instanceof Error ? err.message : err}`);
} finally {
await rm(tmpDir, { recursive: true, force: true }).catch(() => {});
}
}
export async function render(input: RenderInput, ctx: RenderContext): Promise<RenderOutput> {
const t0 = Date.now();
const { bundle, renderer } = loadRemotion();
ctx.onProgress?.(5, 'preparing');
const { sourcePath } = input.template;
const isNative = input.template.mode === 'native';
if (!sourcePath || !existsSync(sourcePath)) {
throw new EngineError(
'template-invalid',
`${isNative ? 'Native template entry' : 'Source HTML'} not found: ${sourcePath}`,
);
}
const { width, height } = input.config.resolution;
const fps = input.config.fps || 30;
const requested = input.config.duration === 'auto' ? 5 : Math.max(0.5, Number(input.config.duration));
const durationInFrames = Math.max(1, Math.round(requested * fps));
const outDir = dirname(input.config.outputPath);
await mkdir(outDir, { recursive: true });
if (isNative) {
// --- Phase 2 native path: bundle the template's OWN entry and render its
// composition directly. The real data rides in as inputProps (the component
// animates props.data via interpolate/spring); no HTML, no neutralize. ---
const compositionId = input.template.nativeCompositionId;
if (!compositionId) {
throw new EngineError('template-invalid', 'Native template missing nativeCompositionId');
}
const serveUrl = await bundleOnce(bundle, sourcePath, ctx);
// The native component reads `data` (+ optional accent/background/foreground);
// width/height let calculateMetadata size the canvas for non-16:9 frames.
const { data, ...restVars } = input.variables ?? {};
const inputProps: Record<string, unknown> = { data, width, height, ...restVars };
await renderComposition({
renderer, serveUrl, compositionId, inputProps,
width, height, fps, durationInFrames, config: input.config, ctx,
});
} else {
// --- Phase 1 bridge path: render the given HTML on the bridge composition. ---
const serveUrl = await bundleOnce(bundle, bridgeEntry(), ctx);
// Read the HTML frame and pass its full source as inputProps. The bridge
// renders it via iframe srcdoc — no staticFile / publicDir / serve-path
// resolution, and srcdoc is same-origin so the per-frame animation seek works.
const rawHtml = await readFile(sourcePath, 'utf8');
const html = neutralizeBlockingResources(rawHtml);
await renderComposition({
renderer, serveUrl, compositionId: 'HtmlFrame', inputProps: { html, width, height },
width, height, fps, durationInFrames, config: input.config, ctx,
});
}
const st = await stat(input.config.outputPath);
ctx.onProgress?.(100, 'done');
return {
outputPath: resolve(input.config.outputPath),
meta: {
durationSec: durationInFrames / fps,
fileSizeBytes: st.size,
actualResolution: { width, height },
fps,
renderedFrames: durationInFrames,
renderWallClockSec: (Date.now() - t0) / 1000,
engineVersion: remotionVersion(),
},
diagnostics: [],
};
}
function remotionVersion(): string {
try {
return (require('remotion/package.json') as { version: string }).version;
} catch {
return '4.x';
}
}
packages/adapter-remotion/src/validate.ts
+61
View File
@@ -0,0 +1,61 @@
import { existsSync } from 'node:fs';
import { createRequire } from 'node:module';
import type { TemplateRef, ValidationError, ValidationResult } from '@html-video/core';
/**
* Validate a template against the Remotion adapter. Cheap & read-only (RFC-01):
* no bundling, no model calls — just engine match + source existence + a soft
* note when the optional Remotion peer deps aren't installed yet.
*
* Phase 1: a template's `sourcePath` is an HTML frame the bridge renders. The
* `engine` field may be 'remotion' (native, Phase 2) or 'hyperframes' (an HTML
* frame the user explicitly chose to render through Remotion) — accept both so
* the bridge can take any HTML frame.
*/
export function validate(template: TemplateRef): ValidationResult {
const errors: ValidationError[] = [];
const warnings: ValidationError[] = [];
if (template.engine !== 'remotion' && template.engine !== 'hyperframes') {
errors.push({
code: 'engine-mismatch',
message: `Template engine "${template.engine}" cannot be rendered by the Remotion adapter`,
fix: `Use the @html-video/adapter-${template.engine} adapter instead`,
});
return { ok: false, errors, warnings };
}
if (!template.sourcePath) {
errors.push({ code: 'missing-source', message: 'Template has no sourcePath' });
} else if (!existsSync(template.sourcePath)) {
errors.push({
code: 'source-not-found',
message: `Template source not found: ${template.sourcePath}`,
fix: 'Check that the template directory contains the file declared in source_entry',
});
}
// Soft signal — render() will fail clearly if the peer deps are truly missing,
// but flag it early so `doctor` / the agent can prompt an install.
if (!remotionInstalled()) {
warnings.push({
code: 'engine-not-installed',
message: 'Remotion peer dependencies are not installed',
fix: 'Run `pnpm add remotion @remotion/bundler @remotion/renderer react react-dom` in the workspace root',
});
}
return { ok: errors.length === 0, errors, warnings };
}
/** Best-effort, synchronous check that the Remotion renderer can be resolved. */
export function remotionInstalled(): boolean {
try {
const req = createRequire(import.meta.url);
req.resolve('@remotion/renderer');
req.resolve('@remotion/bundler');
return true;
} catch {
return false;
}
}
packages/adapter-remotion/test/neutralize-resources.test.mjs
+57
View File
@@ -0,0 +1,57 @@
// Unit tests for neutralizeBlockingResources — the render-blocking-stylesheet
// guard that fixed the all-black Remotion render (external Google Fonts <link>
// kept the srcdoc iframe from painting in headless chromium). See
// notes/2026-06-06-remotion-adapter-verify.md.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { neutralizeBlockingResources } from '../dist/render.js';
test('async-loads an external Google Fonts stylesheet link', () => {
const html = `<head><link href="https://fonts.googleapis.com/css2?family=Archivo+Black&display=swap" rel="stylesheet" /></head>`;
const out = neutralizeBlockingResources(html);
assert.match(out, /media="print"/);
assert.match(out, /onload="this\.media='all'"/);
// still references the font so it applies if it loads in time
assert.match(out, /fonts\.googleapis\.com/);
});
test('handles rel before href, single quotes, no self-closing slash', () => {
const html = `<link rel='stylesheet' href='https://cdn.example.com/x.css'>`;
const out = neutralizeBlockingResources(html);
assert.match(out, /media="print"/);
});
test('handles protocol-relative external links', () => {
const html = `<link rel="stylesheet" href="//cdn.example.com/a.css">`;
const out = neutralizeBlockingResources(html);
assert.match(out, /media="print"/);
});
test('leaves relative / same-document stylesheet links untouched', () => {
const html = `<link rel="stylesheet" href="./local.css">`;
assert.equal(neutralizeBlockingResources(html), html);
});
test('leaves inline <style> untouched', () => {
const html = `<style>@import url(https://fonts.googleapis.com/x);body{color:red}</style>`;
assert.equal(neutralizeBlockingResources(html), html);
});
test('does not double-process a link that already declares media', () => {
const html = `<link rel="stylesheet" href="https://fonts.googleapis.com/x" media="screen">`;
assert.equal(neutralizeBlockingResources(html), html);
});
test('neutralizes every external stylesheet in a multi-link head', () => {
const html = `<link rel="stylesheet" href="https://fonts.googleapis.com/a">
<link rel="stylesheet" href="https://use.typekit.net/b.css">
<link rel="stylesheet" href="/local.css">`;
const out = neutralizeBlockingResources(html);
assert.equal((out.match(/media="print"/g) ?? []).length, 2);
assert.match(out, /href="\/local\.css"/); // local one unchanged
});
test('is a no-op for HTML with no stylesheet links', () => {
const html = `<!doctype html><html><body><h1>hi</h1></body></html>`;
assert.equal(neutralizeBlockingResources(html), html);
});
packages/adapter-remotion/tsconfig.json
+11
View File
@@ -0,0 +1,11 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "dist",
"noEmit": false
},
"comment": "Only the adapter's own .ts is compiled. The bridge/*.tsx are NOT compiled here — they are static assets handed to Remotion's bundle() (webpack+esbuild), which understands JSX. They ship via package.json `files`.",
"include": ["src/**/*.ts"],
"exclude": ["src/bridge/**/*", "dist", "node_modules"]
}
packages/cli/package.json
+31
View File
@@ -0,0 +1,31 @@
{
"name": "@html-video/cli",
"version": "0.1.0",
"description": "html-video command-line interface",
"license": "Apache-2.0",
"type": "module",
"bin": {
"html-video": "./dist/bin.js"
},
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
"files": ["dist", "README.md"],
"scripts": {
"build": "tsc -p tsconfig.json && chmod +x dist/bin.js",
"typecheck": "tsc -p tsconfig.json --noEmit",
"smoke": "node dist/smoke.js",
"test": "npm run build && node --test --experimental-strip-types \"test/**/*.test.ts\""
},
"dependencies": {
"@html-video/content-graph": "workspace:*",
"@html-video/core": "workspace:*",
"@html-video/adapter-hyperframes": "workspace:*",
"@html-video/adapter-remotion": "workspace:*",
"@html-video/project-studio": "workspace:*",
"@html-video/runtime": "workspace:*",
"cac": "^6.7.14"
},
"devDependencies": {
"typescript": "^5.7.2"
}
}
packages/cli/src/bin.ts
+215
View File
@@ -0,0 +1,215 @@
#!/usr/bin/env node
import cacModule from 'cac';
import { bootstrap } from './context.js';
import { setJsonMode, ok, fail } from './output.js';
import { runDoctor } from './commands/doctor.js';
import { listEngines } from './commands/list-engines.js';
import { searchTemplates, inspectTemplate } from './commands/templates.js';
import {
projectCreate,
projectList,
projectShow,
projectDelete,
projectAddAsset,
projectRemoveAsset,
projectSetTemplate,
projectSetVar,
projectSetVars,
projectPreview,
projectRender,
} from './commands/project.js';
import { startStudioServer } from './studio-server.js';
// cac is a CJS default export; ESM interop sometimes wraps it in `.default`
// biome-ignore lint/suspicious/noExplicitAny: cac's types don't expose this shape
const cacFn: (name: string) => any =
typeof cacModule === 'function' ? (cacModule as any) : (cacModule as any).default;
const cli = cacFn('html-video');
cli.option('--json', 'JSON output (default: on)', { default: true });
cli.option('--no-color', 'disable ANSI colors');
cli.option('--cwd <path>', 'project root');
cli.command('doctor', 'Diagnose environment').action(async (opts: any) => {
setJsonMode(!!opts.json);
const ctx = await bootstrap({ cwd: opts.cwd });
await runDoctor(ctx);
});
cli.command('list-engines', 'List installed engine adapters').action(async (opts: any) => {
setJsonMode(!!opts.json);
const ctx = await bootstrap({ cwd: opts.cwd });
await listEngines(ctx);
});
cli
.command('search-templates', 'Search templates by intent')
.option('--intent <text>', 'Free-text user intent')
.option('--aspect <ratio>', '16:9 / 9:16 / 1:1')
.option('--license-allow <list>', 'Comma-separated SPDX ids')
.option('--top <n>', 'Top N matches', { default: 5 })
.action(async (opts: any) => {
setJsonMode(!!opts.json);
const ctx = await bootstrap({ cwd: opts.cwd });
await searchTemplates(ctx, {
intent: opts.intent,
aspect: opts.aspect,
licenseAllow: opts.licenseAllow,
top: Number(opts.top),
});
});
cli
.command('inspect-template <id>', 'Show full metadata for a template')
.action(async (id: string, opts: any) => {
setJsonMode(!!opts.json);
const ctx = await bootstrap({ cwd: opts.cwd });
await inspectTemplate(ctx, id);
});
// ====== project-* commands (RFC-05) ======
cli
.command('project-create', 'Create a new project')
.option('--name <text>', 'Project name (required)')
.option('--intent <text>', 'One-sentence description')
.option('--aspect <ratio>', '16:9 / 9:16 / 1:1')
.option('--commercial', 'Mark as commercial-use')
.action(async (opts: any) => {
setJsonMode(!!opts.json);
const ctx = await bootstrap({ cwd: opts.cwd });
await projectCreate(ctx, {
name: opts.name,
intent: opts.intent,
aspect: opts.aspect,
commercial: !!opts.commercial,
});
});
cli.command('project-list', 'List all projects').action(async (opts: any) => {
setJsonMode(!!opts.json);
const ctx = await bootstrap({ cwd: opts.cwd });
await projectList(ctx);
});
cli
.command('project-show <id>', 'Show project details')
.action(async (id: string, opts: any) => {
setJsonMode(!!opts.json);
const ctx = await bootstrap({ cwd: opts.cwd });
await projectShow(ctx, id);
});
cli
.command('project-delete <id>', 'Delete a project')
.action(async (id: string, opts: any) => {
setJsonMode(!!opts.json);
const ctx = await bootstrap({ cwd: opts.cwd });
await projectDelete(ctx, id);
});
cli
.command('project-add-asset <id>', 'Add an asset to a project')
.option('--file <path>', 'Path to a file (image / video / audio / data)')
.option('--inline-text <text>', 'Inline text content')
.option('--inline-data-file <path>', 'Path to JSON / CSV file to embed as data asset')
.option('--caption <text>', 'Optional caption')
.action(async (id: string, opts: any) => {
setJsonMode(!!opts.json);
const ctx = await bootstrap({ cwd: opts.cwd });
await projectAddAsset(ctx, id, {
file: opts.file,
inlineText: opts.inlineText,
inlineDataFile: opts.inlineDataFile,
caption: opts.caption,
});
});
cli
.command('project-remove-asset <id>', 'Remove an asset from a project')
.option('--asset <assetId>', 'Asset id to remove (required)')
.action(async (id: string, opts: any) => {
setJsonMode(!!opts.json);
if (!opts.asset) fail('invalid-input', '--asset required');
const ctx = await bootstrap({ cwd: opts.cwd });
await projectRemoveAsset(ctx, id, opts.asset);
});
cli
.command('project-set-template <id>', 'Pick a template for the project')
.option('--template <templateId>', 'Template id (required)')
.action(async (id: string, opts: any) => {
setJsonMode(!!opts.json);
if (!opts.template) fail('invalid-input', '--template required');
const ctx = await bootstrap({ cwd: opts.cwd });
await projectSetTemplate(ctx, id, opts.template);
});
cli
.command('project-set-var <id>', 'Set a single variable')
.option('--key <name>', 'Variable name')
.option('--value <json>', 'Value (parsed as JSON if possible, else string)')
.action(async (id: string, opts: any) => {
setJsonMode(!!opts.json);
if (!opts.key) fail('invalid-input', '--key required');
const ctx = await bootstrap({ cwd: opts.cwd });
await projectSetVar(ctx, id, opts.key, opts.value);
});
cli
.command('project-set-vars <id>', 'Replace all variables from a JSON file')
.option('--vars-file <path>', 'Path to JSON file (required)')
.action(async (id: string, opts: any) => {
setJsonMode(!!opts.json);
if (!opts.varsFile) fail('invalid-input', '--vars-file required');
const ctx = await bootstrap({ cwd: opts.cwd });
await projectSetVars(ctx, id, opts.varsFile);
});
cli
.command('project-preview <id>', 'Render an HTML preview of the project')
.action(async (id: string, opts: any) => {
setJsonMode(!!opts.json);
const ctx = await bootstrap({ cwd: opts.cwd });
await projectPreview(ctx, id);
});
cli
.command('project-render <id>', 'Export the project to MP4')
.option('--output <path>', 'Output MP4 path')
.option('--stream-progress', 'Emit progress as NDJSON')
.action(async (id: string, opts: any) => {
setJsonMode(!!opts.json);
const ctx = await bootstrap({ cwd: opts.cwd });
await projectRender(ctx, id, {
output: opts.output,
streamProgress: !!opts.streamProgress,
});
});
// ====== Studio (HTML Anything-style three-pane UI) ======
cli
.command('studio', 'Launch the project studio in the browser')
.option('--port <n>', 'Port (default 3071)', { default: 3071 })
.action(async (opts: any) => {
setJsonMode(!!opts.json);
const ctx = await bootstrap({ cwd: opts.cwd });
const handle = await startStudioServer(ctx, Number(opts.port));
ok({
url: handle.url,
port: handle.port,
pid: process.pid,
project_count: (await ctx.orchestrator.list()).length,
template_count: ctx.templates.list().length,
note: 'Studio running. Press Ctrl+C to stop.',
});
process.on('SIGINT', () => {
handle.close();
process.exit(0);
});
});
cli.help();
cli.version('0.1.0');
cli.parse();
packages/cli/src/commands/doctor.ts
+108
View File
@@ -0,0 +1,108 @@
import { execSync } from 'node:child_process';
import type { CliContext } from '../context.js';
import { ok } from '../output.js';
interface Check {
name: string;
status: 'ok' | 'warning' | 'missing' | 'error';
value?: string;
install_hint?: string;
detail?: string;
}
function which(cmd: string): string | null {
try {
return execSync(`which ${cmd}`, { stdio: ['ignore', 'pipe', 'ignore'] })
.toString()
.trim() || null;
} catch {
return null;
}
}
function version(cmd: string, args = '--version'): string | null {
try {
return execSync(`${cmd} ${args}`, { stdio: ['ignore', 'pipe', 'ignore'] })
.toString()
.trim()
.split('\n')[0]
?? null;
} catch {
return null;
}
}
export async function runDoctor(ctx: CliContext): Promise<void> {
const checks: Check[] = [];
// Node
const nodeV = process.version;
checks.push({
name: 'node-version',
status: parseInt(nodeV.slice(1)) >= 20 ? 'ok' : 'warning',
value: nodeV,
detail: 'html-video targets Node 20+',
});
// ffmpeg
if (which('ffmpeg')) {
checks.push({ name: 'ffmpeg', status: 'ok', value: version('ffmpeg', '-version')?.split(' ')[2] ?? '?' });
} else {
checks.push({
name: 'ffmpeg',
status: 'missing',
install_hint: 'brew install ffmpeg (macOS) / apt install ffmpeg (Linux)',
});
}
// chromium / chrome (for HF puppeteer)
const chromePaths = [
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
'/usr/bin/chromium',
'/usr/bin/google-chrome',
];
const chromiumOk = chromePaths.some((p) => {
try {
execSync(`test -x "${p}"`);
return true;
} catch {
return false;
}
});
checks.push({
name: 'chromium',
status: chromiumOk ? 'ok' : 'warning',
detail: chromiumOk ? 'Chrome found in standard location' : 'Chrome/Chromium not detected; HF render will need a browser',
});
// Engines
for (const engine of ctx.engines.list()) {
checks.push({
name: `adapter-${engine.id}`,
status: 'ok',
value: engine.upstreamVersion,
detail: `${engine.name} adapter loaded`,
});
}
// Templates
const tcount = ctx.templates.list().length;
checks.push({
name: 'templates',
status: tcount >= 1 ? 'ok' : 'warning',
value: `${tcount} discovered`,
detail: tcount === 0 ? 'No templates found in templates/ — install or scaffold some' : undefined,
});
const overall: 'ok' | 'warning' | 'error' = checks.some((c) => c.status === 'error')
? 'error'
: checks.some((c) => c.status === 'missing' || c.status === 'warning')
? 'warning'
: 'ok';
ok({
overall,
project_root: ctx.projectRoot,
checks,
});
}
packages/cli/src/commands/list-engines.ts
+12
View File
@@ -0,0 +1,12 @@
import type { CliContext } from '../context.js';
import { ok } from '../output.js';
export async function listEngines(ctx: CliContext): Promise<void> {
const engines = ctx.engines.list().map((a) => ({
id: a.id,
name: a.name,
upstream_version: a.upstreamVersion,
capabilities: a.capabilities,
}));
ok({ engines });
}
packages/cli/src/commands/project.ts
+155
View File
@@ -0,0 +1,155 @@
/**
* Project-centric CLI commands per RFC-05.
*/
import { readFile } from 'node:fs/promises';
import { existsSync } from 'node:fs';
import { resolve } from 'node:path';
import type { CliContext } from '../context.js';
import { fail, ok, progress } from '../output.js';
export async function projectCreate(
ctx: CliContext,
opts: { name: string; intent?: string; aspect?: string; commercial?: boolean },
): Promise<void> {
if (!opts.name) fail('invalid-input', '--name required');
const project = await ctx.orchestrator.create({
name: opts.name,
...(opts.intent !== undefined && { intent: opts.intent }),
preferences: {
...(opts.aspect !== undefined && { aspect: opts.aspect }),
...(opts.commercial !== undefined && { commercial: opts.commercial }),
},
});
ok({ project_id: project.id, name: project.name, status: project.status });
}
export async function projectList(ctx: CliContext): Promise<void> {
const projects = await ctx.orchestrator.list();
ok({
projects: projects.map((p) => ({
id: p.id,
name: p.name,
template_id: p.templateId,
asset_count: p.assets.length,
status: p.status,
updated_at: p.updatedAt,
})),
});
}
export async function projectShow(ctx: CliContext, id: string): Promise<void> {
const project = await ctx.orchestrator.load(id);
ok({ project });
}
export async function projectDelete(ctx: CliContext, id: string): Promise<void> {
await ctx.orchestrator.remove(id);
ok({ project_id: id, deleted: true });
}
export async function projectAddAsset(
ctx: CliContext,
id: string,
opts: {
file?: string;
inlineText?: string;
inlineDataFile?: string;
caption?: string;
},
): Promise<void> {
let project;
if (opts.file) {
const f = resolve(opts.file);
if (!existsSync(f)) fail('asset-not-found', `File not found: ${f}`);
project = await ctx.orchestrator.addFileAsset(id, f, opts.caption);
} else if (opts.inlineText) {
project = await ctx.orchestrator.addInlineAsset(id, opts.inlineText, 'text', opts.caption);
} else if (opts.inlineDataFile) {
const f = resolve(opts.inlineDataFile);
if (!existsSync(f)) fail('asset-not-found', `Data file not found: ${f}`);
const content = await readFile(f, 'utf8');
project = await ctx.orchestrator.addInlineAsset(id, content, 'data', opts.caption);
} else {
fail('invalid-input', 'Provide one of --file, --inline-text, --inline-data-file');
}
ok({
project_id: project!.id,
asset_count: project!.assets.length,
last_added: project!.assets[project!.assets.length - 1],
});
}
export async function projectRemoveAsset(
ctx: CliContext,
id: string,
assetId: string,
): Promise<void> {
const project = await ctx.orchestrator.removeAsset(id, assetId);
ok({ project_id: project.id, asset_count: project.assets.length });
}
export async function projectSetTemplate(
ctx: CliContext,
id: string,
templateId: string,
): Promise<void> {
const project = await ctx.orchestrator.setTemplate(id, templateId);
ok({
project_id: project.id,
template_id: project.templateId,
variables: project.variables,
});
}
export async function projectSetVar(
ctx: CliContext,
id: string,
key: string,
valueJson: string,
): Promise<void> {
let value: unknown;
try {
value = JSON.parse(valueJson);
} catch {
// not JSON → keep raw string
value = valueJson;
}
const project = await ctx.orchestrator.setVariable(id, key, value);
ok({ project_id: project.id, variables: project.variables });
}
export async function projectSetVars(
ctx: CliContext,
id: string,
varsFile: string,
): Promise<void> {
const f = resolve(varsFile);
if (!existsSync(f)) fail('invalid-input', `vars file not found: ${f}`);
const vars = JSON.parse(await readFile(f, 'utf8')) as Record<string, unknown>;
const project = await ctx.orchestrator.setVariables(id, vars);
ok({ project_id: project.id, variables: project.variables });
}
export async function projectPreview(ctx: CliContext, id: string): Promise<void> {
const { project, htmlPath } = await ctx.orchestrator.renderPreviewHtml(id);
ok({
project_id: project.id,
html_path: htmlPath,
poster_path: project.lastPreviewPosterPath,
note: 'Open html_path in a browser to preview, or use `html-video studio` for full UI.',
});
}
export async function projectRender(
ctx: CliContext,
id: string,
opts: { output?: string; streamProgress?: boolean },
): Promise<void> {
const { project, outputPath } = await ctx.orchestrator.exportMp4({
projectId: id,
...(opts.output !== undefined && { outputPath: resolve(opts.output) }),
onProgress: opts.streamProgress ? (pct, stage) => progress(stage, pct) : undefined,
});
ok({ project_id: project.id, output_path: outputPath, status: project.status });
}
packages/cli/src/commands/templates.ts
+47
View File
@@ -0,0 +1,47 @@
import type { CliContext } from '../context.js';
import { fail, ok } from '../output.js';
interface SearchOpts {
intent?: string;
aspect?: string;
licenseAllow?: string;
top?: number;
}
export async function searchTemplates(ctx: CliContext, opts: SearchOpts): Promise<void> {
const allowed = opts.licenseAllow?.split(',').map((s) => s.trim()).filter(Boolean);
const enginesAvailable = ctx.engines.list().map((e) => e.id);
const matches = ctx.templates.search({
...(opts.intent !== undefined && { intent: opts.intent }),
...(opts.aspect !== undefined && { aspect: opts.aspect }),
...(allowed && { licenseAllow: allowed }),
enginesAvailable,
top: opts.top ?? 5,
});
ok({
matches: matches.map((m) => ({
id: m.template.id,
name: m.template.name,
engine: m.template.engine,
engine_installed: ctx.engines.has(m.template.engine),
score: m.score,
score_reason: m.reason,
preview_poster: m.template.preview.poster,
best_for: m.template.best_for,
category: m.template.category,
license: m.template.license.spdx,
duration_min_sec: m.template.output.duration.min_sec,
duration_max_sec: m.template.output.duration.max_sec,
})),
});
}
export async function inspectTemplate(ctx: CliContext, id: string): Promise<void> {
if (!ctx.templates.has(id)) {
fail('template-not-found', `Template "${id}" not found`, {
available: ctx.templates.list().map((t) => t.id),
});
}
const t = ctx.templates.get(id);
ok({ template: t });
}
packages/cli/src/context.ts
+84
View File
@@ -0,0 +1,84 @@
/**
* Bootstrap shared CLI context: project root, registries, stores, orchestrator.
*/
import { existsSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import {
AssetStore,
EngineRegistry,
ProjectOrchestrator,
ProjectStore,
TemplateRegistry,
} from '@html-video/core';
import hfAdapter from '@html-video/adapter-hyperframes';
import remotionAdapter, { remotionInstalled } from '@html-video/adapter-remotion';
import { MediaConfigStore } from './media-config.js';
export interface CliContext {
projectRoot: string;
engines: EngineRegistry;
templates: TemplateRegistry;
projects: ProjectStore;
assets: AssetStore;
orchestrator: ProjectOrchestrator;
templatesDir: string;
mediaConfig: MediaConfigStore;
}
export function findProjectRoot(start: string = process.cwd()): string {
let dir = start;
for (let i = 0; i < 8; i++) {
if (existsSync(join(dir, '.html-video'))) return dir;
if (existsSync(join(dir, 'pnpm-workspace.yaml'))) return dir;
if (existsSync(join(dir, 'package.json')) && existsSync(join(dir, 'templates'))) return dir;
const parent = dirname(dir);
if (parent === dir) break;
dir = parent;
}
return start;
}
function findTemplatesDir(projectRoot: string): string {
const candidates = [
join(projectRoot, 'templates'),
// packages/cli/dist/context.js → up 3 levels → monorepo root → templates/
join(dirname(fileURLToPath(import.meta.url)), '..', '..', '..', 'templates'),
];
for (const c of candidates) {
if (existsSync(c)) return c;
}
return candidates[0]!;
}
export async function bootstrap(opts: { cwd?: string } = {}): Promise<CliContext> {
const projectRoot = findProjectRoot(opts.cwd);
const engines = new EngineRegistry();
engines.register(hfAdapter);
// Register Remotion only when its peer deps are actually installed, so the
// registry never lists an engine that would throw on render. (RFC-08)
if (remotionInstalled()) {
engines.register(remotionAdapter);
}
const templates = new TemplateRegistry();
const templatesDir = findTemplatesDir(projectRoot);
await templates.scan(templatesDir);
const projects = new ProjectStore(projectRoot);
const assets = new AssetStore({ projectRoot });
const orchestrator = new ProjectOrchestrator({
projectRoot,
engines,
templates,
projects,
assets,
});
const mediaConfig = new MediaConfigStore(projectRoot);
return { projectRoot, engines, templates, projects, assets, orchestrator, templatesDir, mediaConfig };
}
packages/cli/src/fetch-source.ts
+306
View File
@@ -0,0 +1,306 @@
/**
* Fetch external content sources (web articles + GitHub repos) server-side and
* turn them into Markdown the agent can read.
*
* Why server-side: the studio's agents (claude --print / cursor-agent / codex /
* the Messages API) have no network access and only consume a plain-text
* prompt. So when a user pastes a link, the server fetches + flattens it here,
* stores it as a text asset, and lets the existing attachment→prompt pipeline
* feed it to the agent.
*
* Zero runtime deps (matches the CLI package's minimalism): native fetch +
* a lean regex HTML→Markdown pass, GitHub's public REST API for repos.
*/
const ARTICLE_MAX = 8_000; // chars of markdown kept from an article
const README_MAX = 10_000; // chars of README kept from a repo
const FETCH_TIMEOUT_MS = 12_000;
const UA =
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36';
export interface FetchedSource {
url: string;
title: string;
markdown: string;
kind: 'article' | 'repo';
truncated: boolean;
}
/** Extract up to `max` distinct http(s) URLs from free text (in order). */
export function extractUrls(text: string, max = 3): string[] {
if (!text) return [];
const re = /https?:\/\/[^\s<>"'`)\]}]+/gi;
const seen = new Set<string>();
const out: string[] = [];
for (const m of text.matchAll(re)) {
// Trim common trailing punctuation that isn't part of the URL.
const u = m[0].replace(/[.,;:!?]+$/, '');
if (!seen.has(u)) {
seen.add(u);
out.push(u);
if (out.length >= max) break;
}
}
return out;
}
/**
* Reject URLs that point at localhost / link-local / private network ranges
* (SSRF guard). Only plain http(s) public hosts are allowed.
*/
export function assertPublicHttpUrl(raw: string): URL {
let u: URL;
try {
u = new URL(raw);
} catch {
throw new Error(`invalid URL: ${raw}`);
}
if (u.protocol !== 'http:' && u.protocol !== 'https:') {
throw new Error(`only http(s) URLs are allowed (got ${u.protocol})`);
}
const host = u.hostname.toLowerCase();
if (
host === 'localhost' ||
host === '0.0.0.0' ||
host === '::1' ||
host.endsWith('.localhost') ||
host.endsWith('.internal') ||
host.endsWith('.local')
) {
throw new Error(`refusing to fetch local host: ${host}`);
}
// IPv4 private / loopback / link-local ranges.
const m = host.match(/^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/);
if (m) {
const [a, b] = [Number(m[1]), Number(m[2])];
if (
a === 127 || // loopback
a === 10 || // private
(a === 172 && b >= 16 && b <= 31) || // private
(a === 192 && b === 168) || // private
(a === 169 && b === 254) || // link-local (cloud metadata)
a === 0
) {
throw new Error(`refusing to fetch private IP: ${host}`);
}
}
return u;
}
/** Dispatch: GitHub repo URL → repo summary, anything else → article. */
export async function fetchSource(rawUrl: string, signal?: AbortSignal): Promise<FetchedSource> {
const u = assertPublicHttpUrl(rawUrl);
const repo = parseGithubRepo(u);
if (repo) return fetchRepo(repo.owner, repo.repo, rawUrl, signal);
return fetchArticle(rawUrl, signal);
}
// --------------------------------------------------------------------------
// Article
// --------------------------------------------------------------------------
async function fetchArticle(url: string, signal?: AbortSignal): Promise<FetchedSource> {
const html = await fetchText(url, { accept: 'text/html,application/xhtml+xml' }, signal);
const title = extractTitle(html);
let body = htmlToMarkdown(extractMainHtml(html));
const truncated = body.length > ARTICLE_MAX;
if (truncated) body = body.slice(0, ARTICLE_MAX);
const markdown = `# ${title || url}\n\nSource: ${url}\n\n${body}`.trim();
return { url, title, markdown, kind: 'article', truncated };
}
/** Extract the inner HTML of the first element matching `openTagRe`, scanning
* forward and balancing nested `<tag>`/`</tag>` so we capture the WHOLE
* container — not just up to the first inner close tag. A naive
* `(.*?)</tag>` regex collapses on deeply-nested markup (e.g. WeChat's
* #js_content wraps hundreds of nested <div>/<section>), which is why the
* old single-regex approach returned an almost-empty body. */
function extractBalanced(html: string, tag: string, openTagRe: RegExp): string | null {
const m = openTagRe.exec(html);
if (!m) return null;
const start = m.index + m[0].length;
const tagRe = new RegExp(`<(/)?${tag}\\b[^>]*>`, 'gi');
tagRe.lastIndex = start;
let depth = 1;
let t: RegExpExecArray | null;
while ((t = tagRe.exec(html))) {
if (t[1]) {
depth--;
if (depth === 0) return html.slice(start, t.index);
} else if (!/\/>$/.test(t[0])) {
depth++;
}
}
return html.slice(start); // unbalanced — take the rest
}
/** Prefer the article's main content container when we can spot one
* (WeChat's #js_content, <article>, <main>), else the whole document. */
function extractMainHtml(html: string): string {
// WeChat official-account articles are server-rendered into #js_content
// (class attribute precedes id, and the open tag spans newlines).
const wx = extractBalanced(html, 'div', /<div[^>]*\bid=["']js_content["'][^>]*>/i);
if (wx && wx.length > 200) return wx;
const article = extractBalanced(html, 'article', /<article[^>]*>/i);
if (article && article.length > 200) return article;
const main = extractBalanced(html, 'main', /<main[^>]*>/i);
if (main && main.length > 200) return main;
// Fall back to <body> so we don't carry <head> noise.
const bodyM = html.match(/<body[^>]*>([\s\S]*?)<\/body>/i);
return bodyM && bodyM[1] ? bodyM[1] : html;
}
function extractTitle(html: string): string {
const og = html.match(/<meta[^>]+property=["']og:title["'][^>]*content=["']([^"']+)["']/i);
if (og && og[1]) return decodeEntities(og[1]).trim();
const t = html.match(/<title[^>]*>([\s\S]*?)<\/title>/i);
return t && t[1] ? decodeEntities(t[1]).replace(/\s+/g, ' ').trim() : '';
}
/** Lean, dependency-free HTML→Markdown. Not a full converter — just enough to
* give the agent readable prose with headings, lists, links kept. */
export function htmlToMarkdown(html: string): string {
let s = html;
// Drop non-content elements entirely.
s = s.replace(/<(script|style|noscript|svg|head|nav|footer|form|iframe)[^>]*>[\s\S]*?<\/\1>/gi, '');
s = s.replace(/<!--[\s\S]*?-->/g, '');
// Block-ish → newlines / markers.
s = s.replace(/<h([1-6])[^>]*>([\s\S]*?)<\/h\1>/gi, (_m, lvl, inner) => `\n\n${'#'.repeat(Number(lvl))} ${strip(inner)}\n`);
s = s.replace(/<li[^>]*>([\s\S]*?)<\/li>/gi, (_m, inner) => `\n- ${strip(inner)}`);
s = s.replace(/<(p|div|section|article|tr|h[1-6]|ul|ol|blockquote)[^>]*>/gi, '\n');
s = s.replace(/<\/(p|div|section|article|tr|li|ul|ol|blockquote)>/gi, '\n');
s = s.replace(/<br\s*\/?>/gi, '\n');
// Inline: links + images keep their target.
s = s.replace(/<a[^>]*href=["']([^"']+)["'][^>]*>([\s\S]*?)<\/a>/gi, (_m, href, inner) => {
const text = strip(inner);
return text ? `[${text}](${href})` : '';
});
s = s.replace(/<img[^>]*alt=["']([^"']*)["'][^>]*src=["']([^"']+)["'][^>]*>/gi, (_m, alt, src) => `![${alt}](${src})`);
s = s.replace(/<img[^>]*src=["']([^"']+)["'][^>]*>/gi, (_m, src) => `![](${src})`);
// Strip every remaining tag.
s = s.replace(/<[^>]+>/g, '');
s = decodeEntities(s);
// Collapse whitespace: trim each line, drop 3+ blank lines.
s = s
.split('\n')
.map((l) => l.replace(/[ \t ]+/g, ' ').trimEnd())
.join('\n')
.replace(/\n{3,}/g, '\n\n')
.trim();
return s;
}
/** Strip tags + decode entities + collapse spaces (for inline fragments). */
function strip(html: string): string {
return decodeEntities(html.replace(/<[^>]+>/g, '')).replace(/\s+/g, ' ').trim();
}
function decodeEntities(s: string): string {
return s
.replace(/&nbsp;/g, ' ')
.replace(/&amp;/g, '&')
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&quot;/g, '"')
.replace(/&#0?39;/g, "'")
.replace(/&apos;/g, "'")
.replace(/&#x([0-9a-f]+);/gi, (_m, h) => safeCodePoint(parseInt(h, 16)))
.replace(/&#(\d+);/g, (_m, d) => safeCodePoint(parseInt(d, 10)));
}
function safeCodePoint(n: number): string {
try {
return Number.isFinite(n) && n > 0 && n <= 0x10ffff ? String.fromCodePoint(n) : '';
} catch {
return '';
}
}
// --------------------------------------------------------------------------
// GitHub repo
// --------------------------------------------------------------------------
function parseGithubRepo(u: URL): { owner: string; repo: string } | null {
if (u.hostname.toLowerCase() !== 'github.com') return null;
const parts = u.pathname.split('/').filter(Boolean);
if (parts.length < 2) return null;
// Skip non-repo paths (search, marketplace, etc. have reserved first segments,
// but a 2-segment owner/repo is the common case; reject known non-repo roots).
const reserved = new Set(['search', 'marketplace', 'topics', 'collections', 'sponsors', 'about', 'features']);
if (reserved.has(parts[0]!.toLowerCase())) return null;
return { owner: parts[0]!, repo: parts[1]!.replace(/\.git$/, '') };
}
async function fetchRepo(owner: string, repo: string, url: string, signal?: AbortSignal): Promise<FetchedSource> {
const api = `https://api.github.com/repos/${owner}/${repo}`;
const ghHeaders = { accept: 'application/vnd.github+json', 'x-github-api-version': '2022-11-28' };
// Repo metadata (required — fails loudly if repo is private/missing).
const metaRaw = await fetchText(api, ghHeaders, signal);
const meta = JSON.parse(metaRaw) as {
full_name?: string;
description?: string;
language?: string;
stargazers_count?: number;
topics?: string[];
license?: { spdx_id?: string };
homepage?: string;
};
// README (raw) + top-level tree — best-effort, don't fail the whole thing.
const readme = await fetchText(`${api}/readme`, { ...ghHeaders, accept: 'application/vnd.github.raw' }, signal).catch(
() => '',
);
const tree = await fetchTopLevelTree(api, ghHeaders, signal).catch(() => [] as string[]);
const title = meta.full_name || `${owner}/${repo}`;
const lines: string[] = [`# ${title}`, '', `Source: ${url}`, ''];
if (meta.description) lines.push(`> ${meta.description}`, '');
const facts: string[] = [];
if (meta.language) facts.push(`Language: ${meta.language}`);
if (typeof meta.stargazers_count === 'number') facts.push(`Stars: ${meta.stargazers_count.toLocaleString('en-US')}`);
if (meta.license?.spdx_id && meta.license.spdx_id !== 'NOASSERTION') facts.push(`License: ${meta.license.spdx_id}`);
if (meta.homepage) facts.push(`Homepage: ${meta.homepage}`);
if (meta.topics?.length) facts.push(`Topics: ${meta.topics.join(', ')}`);
if (facts.length) lines.push(...facts.map((f) => `- ${f}`), '');
if (tree.length) {
lines.push('## Top-level structure', '', ...tree.map((t) => `- ${t}`), '');
}
let readmeMd = readme.trim();
const truncated = readmeMd.length > README_MAX;
if (truncated) readmeMd = readmeMd.slice(0, README_MAX);
if (readmeMd) lines.push('## README', '', readmeMd);
return { url, title, markdown: lines.join('\n').trim(), kind: 'repo', truncated };
}
async function fetchTopLevelTree(
api: string,
headers: Record<string, string>,
signal?: AbortSignal,
): Promise<string[]> {
const raw = await fetchText(`${api}/contents`, headers, signal);
const items = JSON.parse(raw) as { name?: string; type?: string }[];
return items
.filter((i) => i.name)
.slice(0, 40)
.map((i) => (i.type === 'dir' ? `${i.name}/` : i.name!));
}
// --------------------------------------------------------------------------
// shared fetch
// --------------------------------------------------------------------------
async function fetchText(url: string, extraHeaders: Record<string, string>, signal?: AbortSignal): Promise<string> {
assertPublicHttpUrl(url);
const res = await fetch(url, {
headers: { 'user-agent': UA, ...extraHeaders },
redirect: 'follow',
signal: signal ?? AbortSignal.timeout(FETCH_TIMEOUT_MS),
});
if (!res.ok) {
throw new Error(`fetch ${url} → HTTP ${res.status}`);
}
return res.text();
}
packages/cli/src/index.ts
+4
View File
@@ -0,0 +1,4 @@
// Programmatic API surface
export { bootstrap, findProjectRoot } from './context.js';
export type { CliContext } from './context.js';
export { startStudioServer } from './studio-server.js';
packages/cli/src/media-config.ts
+94
View File
@@ -0,0 +1,94 @@
/**
* Studio media-provider config — persists API credentials entered through the
* Settings UI to `.html-video/media-config.json` under the project root, so
* users don't have to set environment variables by hand.
*
* Credential precedence when resolving (config file wins over env, since the
* GUI is the explicit user choice):
* media-config.json → OD_MINIMAX_API_KEY / MINIMAX_API_KEY env
*
* Mirrors open-design's `.od/media-config.json` shape loosely; we only need
* MiniMax here. The file holds the raw key, so it lives in the gitignored
* `.html-video/` runtime dir, never the repo.
*/
import { existsSync, readFileSync, writeFileSync, mkdirSync } from 'node:fs';
import { join } from 'node:path';
import { resolveMinimaxCredentials, type MinimaxCredentials } from '@html-video/core';
interface MediaConfig {
minimax?: { apiKey?: string; baseUrl?: string };
}
export class MediaConfigStore {
private readonly path: string;
private readonly dir: string;
constructor(projectRoot: string) {
this.dir = join(projectRoot, '.html-video');
this.path = join(this.dir, 'media-config.json');
}
private read(): MediaConfig {
if (!existsSync(this.path)) return {};
try {
return JSON.parse(readFileSync(this.path, 'utf8')) as MediaConfig;
} catch {
return {};
}
}
private write(cfg: MediaConfig): void {
if (!existsSync(this.dir)) mkdirSync(this.dir, { recursive: true });
writeFileSync(this.path, JSON.stringify(cfg, null, 2), { mode: 0o600 });
}
/** What the Settings UI shows: whether a key is set + masked key + base URL.
* Never returns the raw key. Reports the source (config file vs env). */
getMinimaxStatus(): { configured: boolean; source: 'config' | 'env' | 'none'; maskedKey: string; baseUrl: string } {
const cfg = this.read().minimax;
if (cfg?.apiKey) {
return { configured: true, source: 'config', maskedKey: mask(cfg.apiKey), baseUrl: cfg.baseUrl ?? '' };
}
const env = resolveMinimaxCredentials();
if (env) {
return { configured: true, source: 'env', maskedKey: mask(env.apiKey), baseUrl: env.baseUrl };
}
return { configured: false, source: 'none', maskedKey: '', baseUrl: '' };
}
/** Persist a key (and optional base URL) entered in the UI. */
setMinimax(apiKey: string, baseUrl?: string): void {
const cfg = this.read();
cfg.minimax = { apiKey: apiKey.trim() };
const b = (baseUrl ?? '').trim();
if (b) cfg.minimax.baseUrl = b;
this.write(cfg);
}
/** Forget the stored MiniMax key (env fallback, if any, still applies). */
clearMinimax(): void {
const cfg = this.read();
delete cfg.minimax;
this.write(cfg);
}
/** Resolve usable credentials: config file first, then env. null if neither. */
resolveMinimax(): MinimaxCredentials | null {
const cfg = this.read().minimax;
if (cfg?.apiKey) {
// Default to the international endpoint when none is set. The old
// api.minimaxi.chat host is RETIRED server-side (issue #4); MiniMax now
// splits into api.minimax.io (international) and api.minimaxi.com (China),
// and keys are region-bound — so the Settings UI asks the user to pick.
const baseUrl = (cfg.baseUrl || '').trim().replace(/\/$/, '') || 'https://api.minimax.io/v1';
return { apiKey: cfg.apiKey, baseUrl };
}
return resolveMinimaxCredentials();
}
}
function mask(key: string): string {
if (key.length <= 8) return '••••';
return `${key.slice(0, 4)}••••${key.slice(-4)}`;
}
packages/cli/src/output.ts
+44
View File
@@ -0,0 +1,44 @@
/**
* JSON-first output helpers per RFC-03.
* `--json` (default for agent) emits NDJSON-friendly single-object lines.
* Non-JSON mode uses simple readable text.
*/
let JSON_MODE = true;
export function setJsonMode(on: boolean) {
JSON_MODE = on;
}
export function ok(payload: unknown): void {
if (JSON_MODE) {
process.stdout.write(`${JSON.stringify({ status: 'ok', ...(payload as object) })}\n`);
} else {
process.stdout.write(`${pretty(payload)}\n`);
}
}
export function fail(code: string, message: string, ctx: Record<string, unknown> = {}): never {
if (JSON_MODE) {
process.stdout.write(`${JSON.stringify({ status: 'error', code, message, ...ctx })}\n`);
} else {
process.stderr.write(`✘ ${code}: ${message}\n`);
}
process.exit(1);
}
export function progress(stage: string, pct: number, extra: Record<string, unknown> = {}): void {
if (JSON_MODE) {
process.stdout.write(
`${JSON.stringify({ type: 'progress', stage, pct, ...extra })}\n`,
);
} else {
process.stdout.write(` ${stage}: ${pct}%\n`);
}
}
function pretty(p: unknown): string {
if (p == null) return '(empty)';
if (typeof p === 'string') return p;
return JSON.stringify(p, null, 2);
}
packages/cli/src/smoke.ts
+173
View File
@@ -0,0 +1,173 @@
/**
* End-to-end smoke test for project-centric workflow (RFC-05).
* Asserts: bootstrap → create project → add assets → set template → preview → render
*/
import { mkdtemp, mkdir, writeFile, readFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { existsSync } from 'node:fs';
import { bootstrap } from './context.js';
const log = (msg: string) => process.stdout.write(`▸ ${msg}\n`);
const ok = (msg: string) => process.stdout.write(` ✓ ${msg}\n`);
async function main() {
const projectRoot = await mkdtemp(join(tmpdir(), 'html-video-smoke-'));
await mkdir(join(projectRoot, '.html-video'), { recursive: true });
log(`workdir: ${projectRoot}`);
const monorepoRoot = resolve(__dirname_polyfill(), '..', '..', '..');
const fakeLogoPath = join(projectRoot, 'fake-logo.png');
const PNG_1x1 = Buffer.from(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR4nGNgYAAAAAMAASsJTYQAAAAASUVORK5CYII=',
'base64',
);
await writeFile(fakeLogoPath, PNG_1x1);
log('bootstrap context');
const ctx = await bootstrap({ cwd: projectRoot });
if (ctx.templates.list().length === 0) {
await ctx.templates.scan(join(monorepoRoot, 'templates'));
}
ok(`engines: ${ctx.engines.list().map((e) => e.id).join(', ')}`);
ok(`templates: ${ctx.templates.list().map((t) => t.id).join(', ')}`);
// 1. Create a project
log('project create');
const project1 = await ctx.orchestrator.create({
name: 'OD Plugin Library Demo',
intent: 'Show OD plugin library distribution',
preferences: { aspect: '16:9', commercial: true },
});
ok(`project ${project1.id} status=${project1.status}`);
// 2. Add assets
log('add image asset');
let p = await ctx.orchestrator.addFileAsset(project1.id, fakeLogoPath, 'OD logo');
ok(`assets=${p.assets.length}`);
log('add inline text asset');
p = await ctx.orchestrator.addInlineAsset(project1.id, 'Design that evolves itself', 'text');
ok(`assets=${p.assets.length}`);
log('add inline data asset');
const chartData = JSON.stringify([
{ label: 'Templates', value: 231, color: '#ffb84d' },
{ label: 'Skills', value: 15, color: '#9b87f5' },
{ label: 'Systems', value: 150, color: '#6dd99c' },
{ label: 'Craft', value: 11, color: '#ff8a4d' },
]);
p = await ctx.orchestrator.addInlineAsset(project1.id, chartData, 'data');
ok(`assets=${p.assets.length}`);
// 3. Pick a template
log('set template = frame-data-chart-nyt');
p = await ctx.orchestrator.setTemplate(project1.id, 'frame-data-chart-nyt');
ok(`templateId=${p.templateId} variables(after-defaults)=${JSON.stringify(p.variables).slice(0, 80)}…`);
// 4. Set variables (use the chart data we just added)
log('set variables');
p = await ctx.orchestrator.setVariables(project1.id, {
title: 'OD Plugin Library Distribution',
subtitle: '2026-05-27',
data: JSON.parse(chartData),
value_format: 'number',
duration_sec: 8,
});
ok('variables saved');
// 5. Render preview HTML
log('render preview html');
const { project: previewedProj, htmlPath } = await ctx.orchestrator.renderPreviewHtml(project1.id);
if (!existsSync(htmlPath)) throw new Error('Preview HTML missing: ' + htmlPath);
const content = await readFile(htmlPath, 'utf8');
if (!content.includes('<html')) throw new Error('Preview HTML malformed');
ok(`status=${previewedProj.status} html=${htmlPath}`);
// 6. Switch template to test variable preservation
log('switch template to frame-glitch-title');
p = await ctx.orchestrator.setTemplate(project1.id, 'frame-glitch-title');
ok(`now templateId=${p.templateId} kept-vars=${JSON.stringify(p.variables)}`);
// 7. Switch back + render again
log('switch back to frame-data-chart-nyt');
p = await ctx.orchestrator.setTemplate(project1.id, 'frame-data-chart-nyt');
p = await ctx.orchestrator.setVariables(project1.id, {
title: 'OD Plugin Library Distribution',
data: JSON.parse(chartData),
duration_sec: 8,
});
// 8. Export MP4 (stub)
log('export MP4 (stub)');
const { project: rendered, outputPath } = await ctx.orchestrator.exportMp4({
projectId: project1.id,
onProgress: (pct, stage) => {
if (pct === 0 || pct === 100 || pct % 25 === 0) ok(`render ${stage} ${pct}%`);
},
});
if (!existsSync(outputPath)) throw new Error('MP4 output missing');
ok(`status=${rendered.status} mp4=${outputPath}`);
// 9. v0.8: ContentGraph + multi-frame self-test
log('v0.8 multi-frame: write content-graph + 3 frames');
const project2 = await ctx.orchestrator.create({
name: 'Multi-frame explainer demo',
intent: 'Test content-graph + frames pipeline',
preferences: {},
});
const graph = {
schemaVersion: 1 as const,
intent: 'explainer' as const,
synopsis: 'Smoke-test explainer with three frames',
nodes: [
{ id: 'intro', kind: 'text' as const, text: 'Hello world', durationSec: 2 },
{ id: 'middle', kind: 'data' as const, data: { v: 42 }, durationSec: 4 },
{ id: 'outro', kind: 'entity' as const, props: { logo: 'OD' }, durationSec: 3 },
],
edges: [
{ from: 'intro', to: 'middle', kind: 'sequence' as const },
{ from: 'middle', to: 'outro', kind: 'dependency' as const },
],
};
await ctx.orchestrator.writeContentGraph(project2.id, graph);
ok('graph persisted + validated');
for (const node of graph.nodes) {
const html = `<!doctype html><html><head><title>${node.id}</title></head><body data-hv-text="${node.id}">${node.id}</body></html>`;
const { frame } = await ctx.orchestrator.writeFrameHtml(project2.id, node.id, html);
ok(`frame written: ${frame.graphNodeId} order=${frame.order} dur=${frame.durationSec}s path=${frame.htmlPath.split('/').slice(-3).join('/')}`);
}
const finalProject = await ctx.orchestrator.load(project2.id);
if (!finalProject.frames || finalProject.frames.length !== 3) {
throw new Error(`expected 3 frames, got ${finalProject.frames?.length}`);
}
// Order should be: intro (no deps), middle (after intro by sequence + before outro), outro (depends on middle)
const order = finalProject.frames.map((f) => f.graphNodeId).join(',');
if (order !== 'intro,middle,outro') {
throw new Error(`unexpected play order: ${order}`);
}
ok(`play order: ${order}`);
// 10. Verify project list works
log('list projects');
const all = await ctx.orchestrator.list();
ok(`${all.length} project(s) in store`);
process.stdout.write('\n✅ smoke test passed\n');
}
function __dirname_polyfill(): string {
const url = import.meta.url;
const path = url.replace(/^file:\/\//, '');
return path.replace(/\/[^/]*$/, '');
}
main().catch((err) => {
process.stderr.write(`\n❌ smoke test failed: ${err.message ?? err}\n`);
if (err.stack) process.stderr.write(err.stack + '\n');
process.exit(1);
});
packages/cli/src/studio-server.ts
+3472
View File
File diff suppressed because it is too large Load Diff
packages/cli/src/task-registry.ts
+148
View File
@@ -0,0 +1,148 @@
/**
* In-process registry of long-running generation tasks (video frames, audio).
*
* The point: a generation must NOT die when the browser navigates away or the
* SSE connection drops. Previously each generate endpoint streamed straight to
* the request's `res`; closing it (switching project / refresh) killed the run.
*
* Now a task runs detached from any request. It accumulates its events, so a
* client can (re)subscribe at any time and get a full replay + live tail. The
* runner's promise drives the work; subscribers come and go freely.
*
* In-memory only — tasks live for the studio process lifetime. A server restart
* loses in-flight tasks (acceptable: the studio is a local single-process tool).
*/
export type TaskKind = 'message' | 'audio';
export type TaskStatus = 'running' | 'done' | 'failed';
export interface TaskEvent {
/** Monotonic per-task sequence, so a reconnecting client can skip what it saw. */
seq: number;
data: unknown;
}
interface Task {
id: string;
projectId: string;
kind: TaskKind;
status: TaskStatus;
events: TaskEvent[];
subscribers: Set<(e: TaskEvent) => void>;
error?: string;
startedAt: number;
endedAt?: number;
}
/** Emit handle handed to a task runner — it just calls emit(data). */
export interface TaskEmitter {
taskId: string;
emit: (data: unknown) => void;
}
export class TaskRegistry {
private tasks = new Map<string, Task>();
private seq = 0;
private idCounter = 0;
/** Tasks completed > this long ago are pruned on the next create(). */
private static readonly TTL_MS = 10 * 60_000;
/**
* Start a detached task. `runner` receives an emitter; whatever it emits is
* fanned out to current subscribers AND retained for replay. The runner's
* resolved value is ignored (state lives in emitted events + the project);
* a thrown error marks the task failed and is emitted as a final event.
*/
create(
projectId: string,
kind: TaskKind,
runner: (emitter: TaskEmitter) => Promise<void>,
): string {
this.prune();
const id = `task_${Date.now().toString(36)}_${(this.idCounter++).toString(36)}`;
const task: Task = {
id,
projectId,
kind,
status: 'running',
events: [],
subscribers: new Set(),
startedAt: Date.now(),
};
this.tasks.set(id, task);
const emit = (data: unknown) => {
const ev: TaskEvent = { seq: ++this.seq, data };
task.events.push(ev);
for (const fn of task.subscribers) {
try { fn(ev); } catch { /* a dead subscriber shouldn't break the task */ }
}
};
// Run detached. Never rejects out of here.
void runner({ taskId: id, emit })
.then(() => {
task.status = 'done';
task.endedAt = Date.now();
emit({ type: 'task_done' });
})
.catch((err: unknown) => {
task.status = 'failed';
task.error = err instanceof Error ? err.message : String(err);
task.endedAt = Date.now();
emit({ type: 'task_failed', message: task.error });
});
return id;
}
/** The newest still-running (or just-finished) task for a project, if any. */
activeTaskFor(projectId: string): { id: string; kind: TaskKind; status: TaskStatus } | null {
let newest: Task | null = null;
for (const t of this.tasks.values()) {
if (t.projectId !== projectId) continue;
if (!newest || t.startedAt > newest.startedAt) newest = t;
}
if (!newest) return null;
return { id: newest.id, kind: newest.kind, status: newest.status };
}
/**
* Subscribe to a task: immediately replays events after `sinceSeq`, then calls
* `onEvent` for each new one. Returns an unsubscribe fn, plus whether the task
* is already finished (so the caller can close the stream). `null` = no such task.
*/
subscribe(
taskId: string,
sinceSeq: number,
onEvent: (e: TaskEvent) => void,
): { unsubscribe: () => void; finished: boolean } | null {
const task = this.tasks.get(taskId);
if (!task) return null;
for (const ev of task.events) {
if (ev.seq > sinceSeq) onEvent(ev);
}
if (task.status !== 'running') {
return { unsubscribe: () => {}, finished: true };
}
task.subscribers.add(onEvent);
return {
unsubscribe: () => { task.subscribers.delete(onEvent); },
finished: false,
};
}
get(taskId: string): Task | undefined {
return this.tasks.get(taskId);
}
private prune(): void {
const now = Date.now();
for (const [id, t] of this.tasks) {
if (t.status !== 'running' && t.endedAt && now - t.endedAt > TaskRegistry.TTL_MS) {
this.tasks.delete(id);
}
}
}
}
packages/cli/test/decode-upload-filename.test.ts
+42
View File
@@ -0,0 +1,42 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { decodeUploadFilename } from '../dist/studio-server.js';
// Regression coverage for issue #9 (CJK attachment filename mojibake).
// The multipart body is read as a latin1 string, so a UTF-8 filename in the
// plain `filename="..."` form arrives with each byte as a latin1 char; we
// round-trip latin1->utf8 to restore it.
// Simulate what the latin1-read header carries for a UTF-8 filename: take the
// real name, encode to UTF-8 bytes, reinterpret those bytes as latin1 chars.
function asLatin1(realUtf8Name: string): string {
return Buffer.from(realUtf8Name, 'utf8').toString('latin1');
}
test('plain filename: CJK name restored from latin1 bytes', () => {
const real = '扑克流程简写语言小白教程.md';
assert.equal(decodeUploadFilename(undefined, asLatin1(real)), real);
});
test('plain filename: mixed CJK + digits + ext', () => {
const real = '局部截取_20260604_223241.png';
assert.equal(decodeUploadFilename(undefined, asLatin1(real)), real);
});
test('plain ASCII filename is unchanged (round-trip is a no-op)', () => {
assert.equal(decodeUploadFilename(undefined, 'report.csv'), 'report.csv');
});
test('RFC 5987 filename* wins and is percent-decoded', () => {
const star = "UTF-8''" + encodeURIComponent('中文文件.md');
assert.equal(decodeUploadFilename(star, 'fallback.md'), '中文文件.md');
});
test('RFC 5987 with language tag prefix', () => {
const star = "UTF-8'zh-CN'" + encodeURIComponent('数据.json');
assert.equal(decodeUploadFilename(star, undefined), '数据.json');
});
test('neither form present yields a safe default', () => {
assert.equal(decodeUploadFilename(undefined, undefined), 'upload');
});
packages/cli/test/parse-format-reply.test.ts
+54
View File
@@ -0,0 +1,54 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { parseFormatReply } from '../dist/studio-server.js';
// Regression coverage for issue #2: a typed free-text format answer (when the
// model asked in prose instead of rendering the hv-form card) must parse into
// the same {aspect, duration, frame_count} shape a card submit produces, so the
// flow advances to confirm instead of re-asking.
test('the exact issue #2 reply: "16:9 横屏 / 5s / 10"', () => {
const r = parseFormatReply('16:9 横屏 / 5s / 10');
assert.equal(r?.aspect, '16:9 横屏');
assert.equal(r?.duration, '5');
assert.equal(r?.frame_count, '10');
});
test('chinese commas + 秒/帧 units: "9:16 竖屏，3秒，6帧"', () => {
const r = parseFormatReply('9:16 竖屏，3秒，6帧');
assert.equal(r?.aspect, '9:16 手机竖屏');
assert.equal(r?.duration, '3');
assert.equal(r?.frame_count, '6');
});
test('keyword-only aspect, no ratio: "方形 5s"', () => {
const r = parseFormatReply('方形 5s');
assert.equal(r?.aspect, '1:1 方形');
assert.equal(r?.duration, '5');
});
test('xiaohongshu keyword maps to 4:5', () => {
assert.equal(parseFormatReply('小红书 10秒 8帧')?.aspect, '4:5 小红书');
});
test('partial: just an aspect still counts', () => {
assert.deepEqual(parseFormatReply('横屏'), { aspect: '16:9 横屏' });
});
test('does not treat duration "s" as a frame count', () => {
const r = parseFormatReply('16:9 / 5s');
assert.equal(r?.duration, '5');
assert.equal(r?.frame_count, undefined);
});
test('long prose is content, not a format answer', () => {
assert.equal(
parseFormatReply('我想做一个介绍我们公司新产品发布的视频，重点讲三个核心卖点和定价'),
undefined,
);
});
test('unrelated short text yields no signal', () => {
assert.equal(parseFormatReply('你好'), undefined);
assert.equal(parseFormatReply('继续'), undefined);
});
packages/cli/tsconfig.json
+9
View File
@@ -0,0 +1,9 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "dist",
"noEmit": false
},
"include": ["src/**/*"]
}
packages/content-graph/README.md
+33
View File
@@ -0,0 +1,33 @@
# @html-video/content-graph
Structured intermediate representation produced by the agent's first round and consumed by the second round to render HTML frame sequences.
See `research/2026-05-28-understand-anything-takeaways.md` (#1 content-graph, #4 graph-then-sort) for the design rationale, and the upcoming RFC-06 draft for the full spec.
## Schema
```ts
interface ContentGraph {
schemaVersion: 1;
intent: 'single-frame' | 'explainer' | 'data-viz' | 'promo' | 'comparison' | 'other';
synopsis?: string;
nodes: Array<EntityNode | DataNode | TextNode>;
edges: Array<{ from: string; to: string; kind: 'sequence' | 'contrast' | 'dependency'; reason?: string }>;
}
```
- **dependency edges** are hard constraints (topo sort).
- **sequence edges** are soft preferences (tie-break order between independent nodes).
- **contrast edges** carry semantics for the frame composer (does not affect order).
## API
```ts
import { validate, topoSort, totalDurationSec } from '@html-video/content-graph';
const result = validate(graph);
if (!result.ok) throw new Error(result.errors[0].message);
const playOrder = topoSort(graph); // string[] of node ids
const totalSec = totalDurationSec(graph);
```
packages/content-graph/package.json
+23
View File
@@ -0,0 +1,23 @@
{
"name": "@html-video/content-graph",
"version": "0.1.0",
"description": "Content graph: structured intermediate representation between user intent and rendered HTML frames (RFC-06)",
"license": "Apache-2.0",
"type": "module",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
".": {
"import": "./dist/index.js",
"types": "./dist/index.d.ts"
}
},
"files": ["dist", "README.md"],
"scripts": {
"build": "tsc -p tsconfig.json",
"typecheck": "tsc -p tsconfig.json --noEmit"
},
"devDependencies": {
"typescript": "^5.7.2"
}
}
packages/content-graph/src/index.ts
+351
View File
@@ -0,0 +1,351 @@
/**
* @html-video/content-graph — RFC-06.
*
* Structured intermediate representation produced by the agent's first round,
* consumed by the second round to render HTML frame sequences.
*
* See research/2026-05-28-understand-anything-takeaways.md (#1 content-graph,
* #4 graph-then-sort) for the design rationale.
*/
export type NodeKind = 'entity' | 'data' | 'text';
export interface BaseNode {
/** Stable id; agent picks readable strings like "intro_logo", "stat_users". */
id: string;
kind: NodeKind;
/**
* Short label for UI (graph view). Optional — falls back to id.
*/
label?: string;
/**
* Optional intent hint for the frame composer:
* "intro" / "data-bar" / "image-pan" / "quote" / "outro" / "list" / ...
* Free-form; the frame-composer agent maps it to template choice.
*/
frameIntent?: string;
/**
* Suggested duration in seconds for this frame. Defaults to 3s if absent.
*/
durationSec?: number;
}
export interface EntityNode extends BaseNode {
kind: 'entity';
/**
* Free-form props for branding entities (logo path, brand color, etc).
* The frame-composer reads these to seed the HTML.
*/
props: Record<string, unknown>;
}
export interface DataNode extends BaseNode {
kind: 'data';
/**
* Concrete data points to visualise: numbers, percentages, time series.
* Schema is permissive — any JSON the composer can render.
*/
data: unknown;
}
export interface TextNode extends BaseNode {
kind: 'text';
/**
* Headline / quote / caption / paragraph copy.
*/
text: string;
}
export type Node = EntityNode | DataNode | TextNode;
export type EdgeKind = 'sequence' | 'contrast' | 'dependency';
export interface Edge {
/** Source node id */
from: string;
/** Target node id */
to: string;
kind: EdgeKind;
/**
* Optional human-readable reason ("contrasts before/after", "depends on
* concept introduced in B"). Helps the frame-composer pick layout cues.
*/
reason?: string;
}
export interface ContentGraph {
/** Schema version. v1 = this RFC-06 draft. */
schemaVersion: 1;
/**
* High-level intent classification. Steers the frame-composer:
* - "single-frame": short brand/title card; collapse to one frame.
* - "explainer": teach a concept; honour dependency edges.
* - "data-viz": walk through numbers; sequence edges drive order.
* - "promo": pacy social-cut style.
* - "comparison": before/after; contrast edges drive layout.
*/
intent:
| 'single-frame'
| 'explainer'
| 'data-viz'
| 'promo'
| 'comparison'
| 'other';
/**
* One-line synopsis the agent writes for itself / the user. Shown in
* studio's graph view as the "what is this video about?" header.
*/
synopsis?: string;
nodes: Node[];
edges: Edge[];
}
// ---------------------------------------------------------------------------
// Validation
// ---------------------------------------------------------------------------
export interface GraphValidationError {
code:
| 'duplicate-node-id'
| 'edge-from-unknown-node'
| 'edge-to-unknown-node'
| 'self-edge'
| 'cycle'
| 'empty-graph'
| 'invalid-kind';
message: string;
/** Offending node or edge for UI highlighting. */
ref?: string;
}
export interface GraphValidationResult {
ok: boolean;
errors: GraphValidationError[];
warnings: GraphValidationError[];
}
/**
* Validate a ContentGraph. Stops at the first cycle (reports it) but collects
* all other errors so the agent gets one round-trip of feedback.
*/
export function validate(graph: ContentGraph): GraphValidationResult {
const errors: GraphValidationError[] = [];
const warnings: GraphValidationError[] = [];
if (!graph.nodes || graph.nodes.length === 0) {
errors.push({ code: 'empty-graph', message: 'Graph has no nodes' });
return { ok: false, errors, warnings };
}
const ids = new Set<string>();
for (const n of graph.nodes) {
if (ids.has(n.id)) {
errors.push({
code: 'duplicate-node-id',
message: `Duplicate node id "${n.id}"`,
ref: n.id,
});
}
ids.add(n.id);
const kind = (n as { kind: string }).kind;
if (kind !== 'entity' && kind !== 'data' && kind !== 'text') {
errors.push({
code: 'invalid-kind',
message: `Node "${(n as { id: string }).id}" has unknown kind "${kind}"`,
ref: (n as { id: string }).id,
});
}
}
for (const e of graph.edges) {
if (e.from === e.to) {
errors.push({
code: 'self-edge',
message: `Edge ${e.from} → ${e.to} is a self-edge`,
ref: `${e.from}->${e.to}`,
});
}
if (!ids.has(e.from)) {
errors.push({
code: 'edge-from-unknown-node',
message: `Edge from unknown node "${e.from}"`,
ref: `${e.from}->${e.to}`,
});
}
if (!ids.has(e.to)) {
errors.push({
code: 'edge-to-unknown-node',
message: `Edge to unknown node "${e.to}"`,
ref: `${e.from}->${e.to}`,
});
}
}
// Cycle detection on dependency edges (the only kind that constrains order).
const cycleNode = findDependencyCycle(graph);
if (cycleNode) {
errors.push({
code: 'cycle',
message: `Dependency cycle detected involving node "${cycleNode}"`,
ref: cycleNode,
});
}
return { ok: errors.length === 0, errors, warnings };
}
// ---------------------------------------------------------------------------
// Topo sort
// ---------------------------------------------------------------------------
/**
* Linearise the graph into a frame play order.
*
* Algorithm:
* 1. Build dependency adjacency (only "dependency" edges constrain order).
* 2. Kahn topological sort; ties broken by sequence-edge order, else by
* original node array order.
*
* Returns node ids in playback order. Throws on cycle (callers should validate
* first; this is a defensive throw).
*/
export function topoSort(graph: ContentGraph): string[] {
const indeg = new Map<string, number>();
const deps = new Map<string, string[]>(); // from -> to (unblocks)
const nodeOrder = new Map<string, number>();
graph.nodes.forEach((n, i) => {
indeg.set(n.id, 0);
deps.set(n.id, []);
nodeOrder.set(n.id, i);
});
for (const e of graph.edges) {
if (e.kind !== 'dependency') continue;
if (!indeg.has(e.from) || !indeg.has(e.to)) continue;
deps.get(e.from)!.push(e.to);
indeg.set(e.to, (indeg.get(e.to) ?? 0) + 1);
}
// Sequence edges as a soft preference: if A->B (sequence) and both indeg=0,
// prefer A before B.
const seqAfter = new Map<string, Set<string>>(); // node -> nodes that should come after
for (const e of graph.edges) {
if (e.kind !== 'sequence') continue;
if (!indeg.has(e.from) || !indeg.has(e.to)) continue;
if (!seqAfter.has(e.from)) seqAfter.set(e.from, new Set());
seqAfter.get(e.from)!.add(e.to);
}
const ready: string[] = [];
for (const [id, d] of indeg) if (d === 0) ready.push(id);
// Stable sort by original node order so output is deterministic
ready.sort((a, b) => (nodeOrder.get(a) ?? 0) - (nodeOrder.get(b) ?? 0));
const out: string[] = [];
while (ready.length > 0) {
// Pick the ready node that:
// 1. is NOT a "sequence successor" of any other ready node, and
// 2. earliest in original order among the survivors.
let pickIdx = 0;
for (let i = 0; i < ready.length; i++) {
const cand = ready[i]!;
const blockedBySequence = ready.some(
(other) => other !== cand && seqAfter.get(other)?.has(cand),
);
if (!blockedBySequence) {
pickIdx = i;
break;
}
}
const next = ready.splice(pickIdx, 1)[0]!;
out.push(next);
for (const succ of deps.get(next) ?? []) {
indeg.set(succ, (indeg.get(succ) ?? 1) - 1);
if (indeg.get(succ) === 0) {
// Insert maintaining original-order stability
const ord = nodeOrder.get(succ) ?? 0;
let insertAt = ready.length;
for (let i = 0; i < ready.length; i++) {
if ((nodeOrder.get(ready[i]!) ?? 0) > ord) {
insertAt = i;
break;
}
}
ready.splice(insertAt, 0, succ);
}
}
}
if (out.length !== graph.nodes.length) {
throw new Error(
`topoSort: cycle detected (sorted ${out.length} of ${graph.nodes.length} nodes)`,
);
}
return out;
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function findDependencyCycle(graph: ContentGraph): string | null {
const adj = new Map<string, string[]>();
for (const n of graph.nodes) adj.set(n.id, []);
for (const e of graph.edges) {
if (e.kind !== 'dependency') continue;
if (!adj.has(e.from) || !adj.has(e.to)) continue;
adj.get(e.from)!.push(e.to);
}
const WHITE = 0,
GRAY = 1,
BLACK = 2;
const color = new Map<string, number>();
for (const id of adj.keys()) color.set(id, WHITE);
const stack: { id: string; iter: Iterator<string> }[] = [];
for (const start of adj.keys()) {
if (color.get(start) !== WHITE) continue;
color.set(start, GRAY);
stack.push({ id: start, iter: adj.get(start)![Symbol.iterator]() });
while (stack.length > 0) {
const top = stack[stack.length - 1]!;
const next = top.iter.next();
if (next.done) {
color.set(top.id, BLACK);
stack.pop();
} else {
const c = color.get(next.value);
if (c === GRAY) return next.value;
if (c === WHITE) {
color.set(next.value, GRAY);
stack.push({ id: next.value, iter: adj.get(next.value)![Symbol.iterator]() });
}
}
}
}
return null;
}
/**
* Look up a node by id. Returns undefined if missing — callers handle.
*/
export function getNode(graph: ContentGraph, id: string): Node | undefined {
return graph.nodes.find((n) => n.id === id);
}
/**
* Default per-frame duration when a node doesn't set one.
*/
export const DEFAULT_FRAME_DURATION_SEC = 3;
/**
* Compute total video duration by summing per-frame durations along the
* topo-sorted play order.
*/
export function totalDurationSec(graph: ContentGraph): number {
const order = topoSort(graph);
let total = 0;
for (const id of order) {
const n = getNode(graph, id);
total += n?.durationSec ?? DEFAULT_FRAME_DURATION_SEC;
}
return total;
}
packages/content-graph/tsconfig.json
+10
View File
@@ -0,0 +1,10 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "dist",
"composite": false,
"noEmit": false
},
"include": ["src/**/*"]
}
packages/core/package.json
+34
View File
@@ -0,0 +1,34 @@
{
"name": "@html-video/core",
"version": "0.1.0",
"description": "html-video core: types, registry, asset store, storyboard orchestrator",
"license": "Apache-2.0",
"type": "module",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
".": {
"import": "./dist/index.js",
"types": "./dist/index.d.ts"
},
"./types": {
"import": "./dist/types/index.js",
"types": "./dist/types/index.d.ts"
}
},
"files": ["dist", "README.md"],
"scripts": {
"build": "tsc -p tsconfig.json",
"typecheck": "tsc -p tsconfig.json --noEmit",
"test": "node --test test/"
},
"dependencies": {
"@html-video/content-graph": "workspace:*",
"ajv": "^8.17.1",
"ajv-formats": "^3.0.1",
"yaml": "^2.7.0"
},
"devDependencies": {
"typescript": "^5.7.2"
}
}
packages/core/src/asset-store.ts
+172
View File
@@ -0,0 +1,172 @@
/**
* Content-addressed asset store, scoped per project.
* RFC-05 §文件存储.
*/
import { createHash } from 'node:crypto';
import { copyFile, mkdir, readFile, stat, writeFile } from 'node:fs/promises';
import { existsSync } from 'node:fs';
import { extname, join } from 'node:path';
import type { Asset, AssetType } from './types/index.js';
import { HtmlVideoError } from './errors.js';
export interface AssetStoreOptions {
projectRoot: string;
}
export class AssetStore {
private readonly projectsDir: string;
constructor(opts: AssetStoreOptions) {
this.projectsDir = join(opts.projectRoot, '.html-video', 'projects');
}
private projectDir(projectId: string): string {
return join(this.projectsDir, projectId);
}
private assetsDir(projectId: string): string {
return join(this.projectDir(projectId), 'assets');
}
static async computeId(filePath: string): Promise<string> {
const buf = await readFile(filePath);
return createHash('sha1').update(buf).digest('hex');
}
static computeInlineId(content: string): string {
return createHash('sha1').update(content).digest('hex');
}
static guessMime(filePath: string): { mime: string; type: AssetType } {
const ext = extname(filePath).toLowerCase();
const map: Record<string, { mime: string; type: AssetType }> = {
'.png': { mime: 'image/png', type: 'image' },
'.jpg': { mime: 'image/jpeg', type: 'image' },
'.jpeg': { mime: 'image/jpeg', type: 'image' },
'.webp': { mime: 'image/webp', type: 'image' },
'.gif': { mime: 'image/gif', type: 'image' },
'.svg': { mime: 'image/svg+xml', type: 'image' },
'.mp3': { mime: 'audio/mpeg', type: 'audio' },
'.wav': { mime: 'audio/wav', type: 'audio' },
'.aac': { mime: 'audio/aac', type: 'audio' },
'.m4a': { mime: 'audio/mp4', type: 'audio' },
'.mp4': { mime: 'video/mp4', type: 'video' },
'.webm': { mime: 'video/webm', type: 'video' },
'.mov': { mime: 'video/quicktime', type: 'video' },
'.csv': { mime: 'text/csv', type: 'data' },
'.json': { mime: 'application/json', type: 'data' },
'.tsv': { mime: 'text/tab-separated-values', type: 'data' },
'.txt': { mime: 'text/plain', type: 'text' },
'.md': { mime: 'text/markdown', type: 'text' },
};
return map[ext] ?? { mime: 'application/octet-stream', type: 'reference-link' };
}
async addFileAsset(
projectId: string,
sourcePath: string,
userTags: string[] = [],
userCaption?: string,
): Promise<Asset> {
if (!existsSync(sourcePath)) {
throw new HtmlVideoError('asset-not-found', `Source file not found: ${sourcePath}`);
}
const id = await AssetStore.computeId(sourcePath);
const { mime, type } = AssetStore.guessMime(sourcePath);
const ext = extname(sourcePath);
const dir = this.assetsDir(projectId);
await mkdir(dir, { recursive: true });
const destPath = join(dir, `${id}${ext}`);
if (!existsSync(destPath)) {
await copyFile(sourcePath, destPath);
}
const st = await stat(destPath);
const filename = sourcePath.split('/').pop() ?? sourcePath;
return {
id,
type,
path: destPath,
metadata: {
filename,
mimeType: mime,
sizeBytes: st.size,
...(userCaption !== undefined && { userCaption }),
},
userTags,
};
}
async addInlineAsset(
projectId: string,
content: string,
type: 'text' | 'data',
userTags: string[] = [],
userCaption?: string,
): Promise<Asset> {
const id = AssetStore.computeInlineId(content);
const dir = this.assetsDir(projectId);
await mkdir(dir, { recursive: true });
const ext = type === 'data' ? '.json' : '.txt';
const destPath = join(dir, `${id}${ext}`);
if (!existsSync(destPath)) {
await writeFile(destPath, content, 'utf8');
}
return {
id,
type,
path: destPath,
content,
metadata: {
filename: `inline${ext}`,
mimeType: type === 'data' ? 'application/json' : 'text/plain',
sizeBytes: Buffer.byteLength(content, 'utf8'),
...(userCaption !== undefined && { userCaption }),
},
userTags,
};
}
/**
* Store raw bytes (e.g. an MP3 returned by a generation API) as a
* content-addressed asset. The id is the sha1 of the bytes, so identical
* payloads dedupe; `ext` drives the mime/type via {@link guessMime}.
*/
async addBufferAsset(
projectId: string,
bytes: Buffer,
ext: string,
userTags: string[] = [],
userCaption?: string,
): Promise<Asset> {
if (bytes.length === 0) {
throw new HtmlVideoError('invalid-input', 'addBufferAsset: empty buffer');
}
const normExt = ext.startsWith('.') ? ext.toLowerCase() : `.${ext.toLowerCase()}`;
const id = createHash('sha1').update(bytes).digest('hex');
const { mime, type } = AssetStore.guessMime(`x${normExt}`);
const dir = this.assetsDir(projectId);
await mkdir(dir, { recursive: true });
const destPath = join(dir, `${id}${normExt}`);
if (!existsSync(destPath)) {
await writeFile(destPath, bytes);
}
return {
id,
type,
path: destPath,
metadata: {
filename: `${id}${normExt}`,
mimeType: mime,
sizeBytes: bytes.length,
...(userCaption !== undefined && { userCaption }),
},
userTags,
};
}
resolvePath(asset: Asset): string {
if (asset.path) return asset.path;
throw new HtmlVideoError('asset-not-found', `Asset ${asset.id} has no path`);
}
}
packages/core/src/errors.ts
+39
View File
@@ -0,0 +1,39 @@
/**
* Stable error codes per RFC-01 + RFC-05.
*/
export type ErrorCode =
| 'engine-not-installed'
| 'engine-not-registered'
| 'template-invalid'
| 'template-not-found'
| 'render-failed'
| 'render-timeout'
| 'output-corrupt'
| 'disk-full'
| 'cancelled'
| 'asset-not-found'
| 'project-not-found'
| 'invalid-input';
export class HtmlVideoError extends Error {
override readonly name = 'HtmlVideoError';
constructor(
public readonly code: ErrorCode,
message: string,
public readonly retryable = false,
public readonly context: Record<string, unknown> = {},
) {
super(message);
}
toJSON(): Record<string, unknown> {
return {
name: this.name,
code: this.code,
message: this.message,
retryable: this.retryable,
context: this.context,
};
}
}
packages/core/src/index.ts
+21
View File
@@ -0,0 +1,21 @@
/**
* @html-video/core — Public API surface.
*/
export * from './types/index.js';
export { HtmlVideoError } from './errors.js';
export type { ErrorCode } from './errors.js';
export { AssetStore } from './asset-store.js';
export type { AssetStoreOptions } from './asset-store.js';
export { EngineRegistry, TemplateRegistry, ProjectStore } from './registry.js';
export { ProjectOrchestrator } from './project.js';
export type {
CreateProjectInput,
ProjectOrchestratorDeps,
} from './project.js';
export {
resolveMinimaxCredentials,
generateTts,
generateMusic,
} from './minimax.js';
export type { MinimaxCredentials, MinimaxAudioResult } from './minimax.js';
packages/core/src/minimax.ts
+245
View File
@@ -0,0 +1,245 @@
/**
* @html-video/core — MiniMax audio provider.
*
* MiniMax exposes speech (`/t2a_v2`) and music (`/music_generation`) under the
* same host, the same Bearer key, and the same response shape — both wrap the
* payload in a `base_resp` envelope and return the audio as a hex string in
* `data.audio`. So one provider + one key covers both narration and music.
*
* The request/parse pattern is ported from open-design's `renderMinimaxTTS`
* (apps/daemon/src/media.ts): fetch → Bearer → check `base_resp.status_code`
* (an HTTP 200 can still be a logical failure) → `Buffer.from(hex, 'hex')`.
*
* Credentials are read from the environment so the studio works without any
* config file; a missing key yields `null` from {@link resolveMinimaxCredentials}
* and callers report it gracefully instead of throwing.
*/
import { HtmlVideoError } from './errors.js';
/** Default base URL. The old `api.minimaxi.chat` host is RETIRED server-side
* (issue #4). MiniMax now has two region-bound endpoints — international
* `api.minimax.io` and China `api.minimaxi.com` — and a key only authenticates
* against its own region. We default to international; override via
* OD_MINIMAX_BASE_URL / MINIMAX_BASE_URL (or the Studio Settings UI). */
const MINIMAX_DEFAULT_BASE_URL = 'https://api.minimax.io/v1';
/** Hard ceiling for a single MiniMax request. Music generation is slow but a
* request that hasn't returned in 2 minutes is hung, not slow. */
const MINIMAX_REQUEST_TIMEOUT_MS = 120_000;
/** Fast turbo speech tier (same default open-design ships). */
const MINIMAX_TTS_MODEL = 'speech-02-turbo';
/**
* Music model. We use music-1.5, NOT the newer music-2.6 family: 2.6's
* synchronous music_generation call never returns for our key (verified: 180s
* with no response), whereas music-1.5 returns audio synchronously in ~50s.
* Trade-off: 1.5 has no `is_instrumental` flag and REQUIRES a `lyrics` field,
* so for instrumental soundtracks we pass a minimal humming placeholder.
*/
const MINIMAX_MUSIC_MODEL = 'music-1.5';
export interface MinimaxCredentials {
apiKey: string;
baseUrl: string;
}
export interface MinimaxAudioResult {
/** Decoded audio bytes (MP3). */
bytes: Buffer;
/** File extension to store under. */
ext: '.mp3';
/** Human-readable note of what was produced (provider · model · size). */
providerNote: string;
/** Reported duration in seconds, if the API surfaced it. */
durationSec?: number;
}
/**
* Resolve MiniMax credentials from the environment. Returns `null` (not throw)
* when no key is set, so the studio can show a friendly "configure your key"
* message instead of a 500.
*
* Key precedence: OD_MINIMAX_API_KEY → MINIMAX_API_KEY
* Base precedence: OD_MINIMAX_BASE_URL → MINIMAX_BASE_URL → default
*/
export function resolveMinimaxCredentials(
env: NodeJS.ProcessEnv = process.env,
): MinimaxCredentials | null {
const apiKey = (env.OD_MINIMAX_API_KEY || env.MINIMAX_API_KEY || '').trim();
if (!apiKey) return null;
const baseUrl = (env.OD_MINIMAX_BASE_URL || env.MINIMAX_BASE_URL || MINIMAX_DEFAULT_BASE_URL)
.trim()
.replace(/\/$/, '');
return { apiKey, baseUrl };
}
/**
* Shared POST + decode for both MiniMax audio endpoints. Throws
* HtmlVideoError('render-failed', …) on transport / API / decode failure.
*/
async function postAndDecode(
endpoint: string,
body: unknown,
creds: MinimaxCredentials,
label: string,
signal?: AbortSignal,
): Promise<{ bytes: Buffer; extraInfo: Record<string, unknown> }> {
// MiniMax generation (esp. music) can take tens of seconds, but it must NOT
// hang forever — an unbounded fetch leaves the studio's SSE stream stuck on
// "generating…" with no failure event, which reads to the user as "the button
// does nothing". Cap it; if the caller passed its own signal, respect that.
const timeoutSignal = AbortSignal.timeout(MINIMAX_REQUEST_TIMEOUT_MS);
const effectiveSignal = signal
? (AbortSignal.any ? AbortSignal.any([signal, timeoutSignal]) : signal)
: timeoutSignal;
let resp: Response;
try {
resp = await fetch(`${creds.baseUrl}/${endpoint}`, {
method: 'POST',
headers: {
authorization: `Bearer ${creds.apiKey}`,
'content-type': 'application/json',
},
body: JSON.stringify(body),
signal: effectiveSignal,
});
} catch (e) {
const isTimeout = e instanceof Error && (e.name === 'TimeoutError' || e.name === 'AbortError');
const msg = e instanceof Error ? e.message : String(e);
throw new HtmlVideoError(
'render-failed',
isTimeout
? `minimax ${label} timed out after ${Math.round(MINIMAX_REQUEST_TIMEOUT_MS / 1000)}s (the API did not respond — try again, or check OD_MINIMAX_BASE_URL)`
: `minimax ${label} request failed: ${msg} (check the API region — international is api.minimax.io, China is api.minimaxi.com; a key only works against its own region)`,
true,
);
}
const respText = await resp.text();
if (!resp.ok) {
throw new HtmlVideoError(
'render-failed',
`minimax ${label} ${resp.status}: ${truncate(respText, 240)}`,
resp.status >= 500,
);
}
let data: {
base_resp?: { status_code?: number; status_msg?: string };
data?: { audio?: unknown };
extra_info?: Record<string, unknown>;
};
try {
data = JSON.parse(respText);
} catch {
throw new HtmlVideoError('render-failed', `minimax ${label} non-JSON: ${truncate(respText, 200)}`);
}
// MiniMax wraps every response in base_resp; an HTTP 200 can still be a
// logical failure (auth / params), surfaced via a non-zero status_code.
if (data.base_resp && data.base_resp.status_code !== 0) {
const code = data.base_resp.status_code;
const hint = code === 1004 || code === 1008 ? ' (auth / insufficient balance — check the API key)' : '';
throw new HtmlVideoError(
'render-failed',
`minimax ${label} api error ${code}: ${data.base_resp.status_msg || 'unknown'}${hint}`,
);
}
const hex = data.data?.audio;
if (typeof hex !== 'string' || !hex) {
throw new HtmlVideoError('render-failed', `minimax ${label} response missing data.audio`);
}
const bytes = Buffer.from(hex, 'hex');
if (bytes.length === 0) {
throw new HtmlVideoError('render-failed', `minimax ${label} decoded zero bytes`);
}
return { bytes, extraInfo: data.extra_info ?? {} };
}
/**
* Generate spoken narration via MiniMax TTS (`/t2a_v2`).
* Defaults to a neutral Mandarin male voice that reads both zh + en well.
*/
export async function generateTts(opts: {
text: string;
voiceId?: string;
languageBoost?: string;
speed?: number;
vol?: number;
pitch?: number;
creds: MinimaxCredentials;
signal?: AbortSignal;
}): Promise<MinimaxAudioResult> {
const text = (opts.text || '').trim();
if (!text) {
throw new HtmlVideoError('invalid-input', 'narration text is empty');
}
const voiceId = (opts.voiceId || '').trim() || 'male-qn-qingse';
const languageBoost = (opts.languageBoost || '').trim();
const body = {
model: MINIMAX_TTS_MODEL,
text,
stream: false,
...(languageBoost ? { language_boost: languageBoost } : {}),
voice_setting: {
voice_id: voiceId,
speed: opts.speed ?? 1.0,
vol: opts.vol ?? 1.0,
pitch: opts.pitch ?? 0,
},
audio_setting: { sample_rate: 32000, format: 'mp3' },
};
const { bytes, extraInfo } = await postAndDecode('t2a_v2', body, opts.creds, 'tts', opts.signal);
const audioLen = typeof extraInfo.audio_length === 'number' ? extraInfo.audio_length : undefined;
const durationSec = audioLen ? Math.round(audioLen / 100) / 10 : undefined;
return {
bytes,
ext: '.mp3',
providerNote: `minimax/${MINIMAX_TTS_MODEL} · ${voiceId} · ${durationSec ?? '?'}s · ${bytes.length} bytes`,
durationSec,
};
}
/**
* Generate background music via MiniMax (`/music_generation`).
* Instrumental-only by default (a video soundtrack rarely wants vocals).
*/
export async function generateMusic(opts: {
prompt: string;
instrumental?: boolean;
creds: MinimaxCredentials;
signal?: AbortSignal;
}): Promise<MinimaxAudioResult> {
const prompt = (opts.prompt || '').trim();
if (!prompt) {
throw new HtmlVideoError('invalid-input', 'music prompt is empty');
}
const instrumental = opts.instrumental ?? true;
// music-1.5 requires a non-empty `lyrics` field and has no is_instrumental
// flag. For an instrumental soundtrack we feed a minimal hummed placeholder
// so the model produces a melody without foregrounded vocals; otherwise let
// the prompt double as a loose lyrical brief.
const lyrics = instrumental ? '[Intro]\nooh ooh\n[Hook]\nla la la' : prompt;
const body = {
model: MINIMAX_MUSIC_MODEL,
prompt,
lyrics,
audio_setting: { sample_rate: 44100, bitrate: 256000, format: 'mp3' },
output_format: 'hex',
};
const { bytes } = await postAndDecode('music_generation', body, opts.creds, 'music', opts.signal);
return {
bytes,
ext: '.mp3',
providerNote: `minimax/${MINIMAX_MUSIC_MODEL} · ${instrumental ? 'instrumental' : 'with-vocals'} · ${bytes.length} bytes`,
};
}
function truncate(s: string, n: number): string {
return s.length > n ? `${s.slice(0, n)}…` : s;
}
packages/core/src/project.ts
+954
View File
@@ -0,0 +1,954 @@
/**
* Project orchestrator: 单模板单视频工作流（RFC-05）。
* - createProject
* - addAsset / removeAsset
* - setTemplate / setVariable / setVariables
* - renderPreviewHtml: 调 EngineAdapter.renderToHtml() → HTML for iframe
* - exportMp4: 调 EngineAdapter.render() → MP4 file
*/
import { randomUUID } from 'node:crypto';
import { join, basename } from 'node:path';
import type {
Asset,
EngineId,
FrameRecord,
Project,
ProjectStatus,
TemplateMetadata,
TemplateRef,
} from './types/index.js';
import {
type ContentGraph,
validate as validateGraph,
topoSort,
DEFAULT_FRAME_DURATION_SEC,
} from '@html-video/content-graph';
import { HtmlVideoError } from './errors.js';
import type { AssetStore } from './asset-store.js';
import type { EngineRegistry, ProjectStore, TemplateRegistry } from './registry.js';
export interface CreateProjectInput {
name: string;
intent?: string;
preferences?: Project['preferences'];
}
export interface ProjectOrchestratorDeps {
projectRoot: string;
engines: EngineRegistry;
templates: TemplateRegistry;
projects: ProjectStore;
assets: AssetStore;
}
export class ProjectOrchestrator {
constructor(private readonly deps: ProjectOrchestratorDeps) {}
// ---------------- CRUD ----------------
async create(input: CreateProjectInput): Promise<Project> {
const id = `proj_${randomUUID().slice(0, 12)}`;
const now = new Date().toISOString();
const project: Project = {
id,
name: input.name,
...(input.intent !== undefined && { intent: input.intent }),
assets: [],
templateId: null,
variables: {},
preferences: input.preferences ?? {},
status: 'draft',
createdAt: now,
updatedAt: now,
};
await this.deps.projects.save(project);
return project;
}
async list(): Promise<Project[]> {
return this.deps.projects.list();
}
async load(id: string): Promise<Project> {
return this.deps.projects.load(id);
}
async remove(id: string): Promise<void> {
return this.deps.projects.remove(id);
}
// ---------------- Asset ops ----------------
async addFileAsset(projectId: string, sourcePath: string, userCaption?: string): Promise<Project> {
await this.deps.projects.ensureDir(projectId);
const project = await this.deps.projects.load(projectId);
const asset = await this.deps.assets.addFileAsset(projectId, sourcePath, [], userCaption);
if (!project.assets.find((a) => a.id === asset.id)) {
project.assets.push(asset);
}
project.status = downgradeStatus(project.status, 'draft');
await this.deps.projects.save(project);
return project;
}
async addInlineAsset(
projectId: string,
content: string,
type: 'text' | 'data',
userCaption?: string,
): Promise<Project> {
await this.deps.projects.ensureDir(projectId);
const project = await this.deps.projects.load(projectId);
const asset = await this.deps.assets.addInlineAsset(projectId, content, type, [], userCaption);
if (!project.assets.find((a) => a.id === asset.id)) {
project.assets.push(asset);
}
project.status = downgradeStatus(project.status, 'draft');
await this.deps.projects.save(project);
return project;
}
/**
* Store generated audio bytes (MP3 from MiniMax) as a project asset and
* return the created Asset so the caller can reference it in `soundtrack`.
* Unlike addFileAsset, this does NOT downgrade status — attaching a
* soundtrack to an already-previewed video shouldn't invalidate the render.
*/
async addBufferAsset(
projectId: string,
bytes: Buffer,
ext: string,
userCaption?: string,
): Promise<{ project: Project; asset: Asset }> {
await this.deps.projects.ensureDir(projectId);
const project = await this.deps.projects.load(projectId);
const asset = await this.deps.assets.addBufferAsset(projectId, bytes, ext, [], userCaption);
if (!project.assets.find((a) => a.id === asset.id)) {
project.assets.push(asset);
}
await this.deps.projects.save(project);
return { project, asset };
}
async removeAsset(projectId: string, assetId: string): Promise<Project> {
const project = await this.deps.projects.load(projectId);
project.assets = project.assets.filter((a) => a.id !== assetId);
await this.deps.projects.save(project);
return project;
}
// ---------------- Template / variables ----------------
async setTemplate(projectId: string, templateId: string | null): Promise<Project> {
const project = await this.deps.projects.load(projectId);
if (templateId !== null && !this.deps.templates.has(templateId)) {
throw new HtmlVideoError('template-not-found', `Template ${templateId} not found`);
}
project.templateId = templateId;
// v0.3: variables are no longer the user-facing surface. Reset on every
// template change so old keys don't bleed through into the new context.
project.variables = {};
project.status = downgradeStatus(project.status, 'draft');
await this.deps.projects.save(project);
return project;
}
async setVariables(projectId: string, vars: Record<string, unknown>): Promise<Project> {
const project = await this.deps.projects.load(projectId);
project.variables = vars;
project.status = downgradeStatus(project.status, 'draft');
await this.deps.projects.save(project);
return project;
}
async setVariable(projectId: string, key: string, value: unknown): Promise<Project> {
const project = await this.deps.projects.load(projectId);
project.variables = { ...project.variables, [key]: value };
project.status = downgradeStatus(project.status, 'draft');
await this.deps.projects.save(project);
return project;
}
async setAgent(projectId: string, agentId: string | null, agentModel?: string | null): Promise<Project> {
const project = await this.deps.projects.load(projectId);
project.agentId = agentId;
// Only touch the model when explicitly provided; switching agent clears a
// stale model unless a new one is given.
if (agentModel !== undefined) project.agentModel = agentModel;
else project.agentModel = null;
await this.deps.projects.save(project);
return project;
}
/**
* v0.3 chat-to-HTML: write raw HTML produced by an agent into the project's preview slot.
* Single-frame fast-path. Clears any prior multi-frame graph state.
*/
async writePreviewHtmlRaw(projectId: string, html: string): Promise<{ project: Project; htmlPath: string }> {
const project = await this.deps.projects.load(projectId);
const projectDir = await this.deps.projects.ensureDir(projectId);
const { writeFile } = await import('node:fs/promises');
const { join } = await import('node:path');
const htmlPath = join(projectDir, 'preview.html');
await writeFile(htmlPath, html, 'utf8');
project.lastPreviewHtmlPath = htmlPath;
// v0.8.2: only treat this as a "supersedes the storyboard" event for
// truly fresh single-frame projects (no frames yet). For projects that
// already have a storyboard, the single-frame raw write is treated as
// an in-place inline edit on the active preview file — frames[] /
// contentGraphPath are preserved so the user doesn't lose their
// storyboard if they happen to use a single-frame iteration on a
// multi-frame project. (Frame-specific in-place edits should go
// through writeFrameHtml instead.)
if ((project.frames?.length ?? 0) === 0) {
project.frames = [];
delete project.contentGraphPath;
}
if (project.status === 'draft') project.status = 'previewed';
await this.deps.projects.save(project);
return { project, htmlPath };
}
// ---------------- v0.8: ContentGraph + multi-frame ----------------
/**
* Persist a content graph alongside the project. Validates first, throws
* on cycles / unknown edges / etc.
*/
async writeContentGraph(
projectId: string,
graph: ContentGraph,
opts: { preserveFrames?: boolean } = {},
): Promise<{ project: Project; graphPath: string }> {
const result = validateGraph(graph);
if (!result.ok) {
throw new HtmlVideoError(
'invalid-input',
`ContentGraph invalid: ${result.errors.map((e) => e.message).join('; ')}`,
);
}
const project = await this.deps.projects.load(projectId);
const projectDir = await this.deps.projects.ensureDir(projectId);
const { writeFile, mkdir } = await import('node:fs/promises');
const { join } = await import('node:path');
const graphPath = join(projectDir, 'content-graph.json');
await writeFile(graphPath, JSON.stringify(graph, null, 2), 'utf8');
project.contentGraphPath = graphPath;
await mkdir(join(projectDir, 'frames'), { recursive: true });
if (opts.preserveFrames) {
// Editing an existing storyboard's metadata (e.g. re-pacing durations) —
// keep the rendered frames, just sync each frame's durationSec from the
// graph so export uses the new timing.
const byId = new Map(graph.nodes.map((n) => [n.id, n.durationSec]));
project.frames = (project.frames ?? []).map((f) => ({
...f,
durationSec: byId.get(f.graphNodeId) ?? f.durationSec,
}));
} else {
// Fresh graph → agent will re-emit per-frame HTML; drop stale frames.
project.frames = [];
if (project.status !== 'rendered') project.status = 'draft';
}
await this.deps.projects.save(project);
return { project, graphPath };
}
/**
* Read the persisted content graph. Returns null if none.
*/
async readContentGraph(projectId: string): Promise<ContentGraph | null> {
const project = await this.deps.projects.load(projectId);
if (!project.contentGraphPath) return null;
const { readFile } = await import('node:fs/promises');
const { existsSync } = await import('node:fs');
if (!existsSync(project.contentGraphPath)) return null;
return JSON.parse(await readFile(project.contentGraphPath, 'utf8')) as ContentGraph;
}
/**
* Write one frame's HTML to disk. Updates the project's frames[] list,
* keeping play-order consistent with the graph's topo sort.
*
* Frame filenames follow `<order>-<nodeId>.html` for visual debuggability.
*/
async writeFrameHtml(
projectId: string,
graphNodeId: string,
html: string,
): Promise<{ project: Project; frame: FrameRecord }> {
const project = await this.deps.projects.load(projectId);
const graph = await this.readContentGraph(projectId);
if (!graph) {
throw new HtmlVideoError(
'invalid-input',
'Cannot write frame: project has no content graph yet',
);
}
const order = topoSort(graph);
const idx = order.indexOf(graphNodeId);
if (idx === -1) {
throw new HtmlVideoError(
'invalid-input',
`Graph node "${graphNodeId}" not found in content graph`,
);
}
const node = graph.nodes.find((n) => n.id === graphNodeId)!;
const projectDir = await this.deps.projects.ensureDir(projectId);
const { writeFile, mkdir } = await import('node:fs/promises');
const { join } = await import('node:path');
const framesDir = join(projectDir, 'frames');
await mkdir(framesDir, { recursive: true });
const safeId = graphNodeId.replace(/[^a-z0-9_-]/gi, '_');
const filename = `${String(idx + 1).padStart(2, '0')}-${safeId}.html`;
const htmlPath = join(framesDir, filename);
await writeFile(htmlPath, html, 'utf8');
const frame: FrameRecord = {
graphNodeId,
htmlPath,
durationSec: node.durationSec ?? DEFAULT_FRAME_DURATION_SEC,
order: idx,
};
project.frames = (project.frames ?? []).filter((f) => f.graphNodeId !== graphNodeId);
project.frames.push(frame);
project.frames.sort((a, b) => a.order - b.order);
// First frame becomes the project preview when no single-frame HTML exists.
if (project.frames[0]?.graphNodeId === graphNodeId) {
project.lastPreviewHtmlPath = htmlPath;
}
if (project.status === 'draft') project.status = 'previewed';
await this.deps.projects.save(project);
return { project, frame };
}
// ---------------- Render: preview HTML / export MP4 ----------------
async renderPreviewHtml(projectId: string): Promise<{ project: Project; htmlPath: string }> {
const project = await this.deps.projects.load(projectId);
if (!project.templateId) {
throw new HtmlVideoError('invalid-input', 'Project has no template selected');
}
const tmpl = this.deps.templates.get(project.templateId);
const adapter = this.deps.engines.get(tmpl.engine);
if (!adapter.renderToHtml) {
throw new HtmlVideoError(
'render-failed',
`Engine ${tmpl.engine} adapter does not support renderToHtml()`,
);
}
const projectDir = await this.deps.projects.ensureDir(projectId);
const out = await adapter.renderToHtml(
{
template: templateRefFromMeta(tmpl),
variables: project.variables,
config: {
format: 'mp4',
resolution: project.preferences.resolution ?? { width: 1920, height: 1080 },
fps: project.preferences.fps ?? 60,
duration: 'auto',
outputPath: join(projectDir, 'output.mp4'),
},
},
{ workDir: projectDir },
);
project.lastPreviewHtmlPath = out.htmlPath;
project.lastPreviewPosterPath = out.posterPath;
if (project.status === 'draft') project.status = 'previewed';
await this.deps.projects.save(project);
return { project, htmlPath: out.htmlPath };
}
async exportMp4(args: {
projectId: string;
outputPath?: string;
onProgress?: (pct: number, stage: string) => void;
signal?: AbortSignal;
}): Promise<{ project: Project; outputPath: string }> {
const project = await this.deps.projects.load(args.projectId);
const projectDir = await this.deps.projects.ensureDir(project.id);
// Unique per-export filename so repeated exports of the SAME project don't
// overwrite each other (different projects already have separate dirs).
// output.mp4 stays as a stable "latest" alias updated after each export.
const stamp = new Date().toISOString().replace(/[:.]/g, '-').replace('T', '_').slice(0, 19);
const outputPath = args.outputPath ?? join(projectDir, `output-${stamp}.mp4`);
// v0.8: multi-frame path. If the project has frames[] from a content graph,
// render each frame's HTML to a per-frame MP4, then ffmpeg concat them.
if (project.frames && project.frames.length > 0) {
const ordered = [...project.frames].sort((a, b) => a.order - b.order);
const tmpl = project.templateId ? this.deps.templates.get(project.templateId) : null;
const projectEngine = tmpl?.engine ?? 'hyperframes';
const frameMp4s: string[] = [];
// Mixed engines across frames → the per-frame MP4s may carry different
// h264 params (hyperframes' libx264 vs Remotion's encoder), so a stream
// -c copy concat can stutter/corrupt. Re-encode the join in that case.
const enginesUsed = new Set(ordered.map((f) => f.engine ?? projectEngine));
const reencode = enginesUsed.size > 1;
for (let i = 0; i < ordered.length; i++) {
const f = ordered[i]!;
const frameOut = join(projectDir, 'frames', `${String(i + 1).padStart(2, '0')}.mp4`);
const { engine: frameEngine, templateRef } = this.resolveFrameTemplateRef(f, projectEngine);
const adapter = this.deps.engines.get(frameEngine);
await adapter.render(
{
template: templateRef,
// Native data templates read `data` from variables; bridge/hyperframes
// ignore it. Frame's own data (when enhanced) overrides project vars.
variables: f.data !== undefined ? { ...project.variables, data: f.data } : project.variables,
config: {
format: 'mp4',
resolution: project.preferences.resolution ?? { width: 1920, height: 1080 },
fps: project.preferences.fps ?? 60,
duration: f.durationSec,
// The user set per-frame length on the format card — honor it as a
// hard cap so one runaway animation can't stretch a 4s frame to ~30s.
durationMode: 'explicit',
outputPath: frameOut,
},
},
{
workDir: projectDir,
...(args.onProgress !== undefined && {
onProgress: (pct, stage) =>
args.onProgress!((i + pct / 100) / ordered.length * 100, `frame ${i + 1}/${ordered.length}: ${stage}`),
}),
...(args.signal !== undefined && { signal: args.signal }),
},
);
frameMp4s.push(frameOut);
}
await concatFramesWithFfmpeg(frameMp4s, outputPath, projectDir, {
reencode,
fps: project.preferences.fps ?? 60,
});
const totalDur = ordered.reduce((s, f) => s + (f.durationSec || 0), 0);
await this.applySoundtrack(project, outputPath, totalDur, args.onProgress);
project.lastOutputMp4Path = outputPath;
recordExport(project, outputPath);
project.status = 'rendered';
await this.deps.projects.save(project);
return { project, outputPath };
}
// Single-frame fast path (v0.7 behaviour).
if (!project.templateId) {
throw new HtmlVideoError('invalid-input', 'Project has no template selected');
}
const tmpl = this.deps.templates.get(project.templateId);
const adapter = this.deps.engines.get(tmpl.engine);
await adapter.render(
{
template: templateRefFromMeta(tmpl),
variables: project.variables,
config: {
format: 'mp4',
resolution: project.preferences.resolution ?? { width: 1920, height: 1080 },
fps: project.preferences.fps ?? 60,
duration: 'auto',
outputPath,
},
},
{
workDir: projectDir,
...(args.onProgress !== undefined && { onProgress: args.onProgress }),
...(args.signal !== undefined && { signal: args.signal }),
},
);
await this.applySoundtrack(project, outputPath, undefined, args.onProgress);
project.lastOutputMp4Path = outputPath;
recordExport(project, outputPath);
project.status = 'rendered';
await this.deps.projects.save(project);
return { project, outputPath };
}
/**
* Resolve which engine + TemplateRef render a single frame. A frame that the
* user has enhanced (engine='remotion' + nativeTemplateId) renders via the
* native template's .tsx entry; otherwise it's the classic per-frame HTML on
* the project's engine (hyperframes). The base `htmlPath` is always retained
* on the frame so un-enhancing is non-destructive. (RFC-08/09)
*/
private resolveFrameTemplateRef(
f: FrameRecord,
projectEngine: EngineId,
): { engine: EngineId; templateRef: TemplateRef } {
if (f.engine === 'remotion' && f.nativeTemplateId) {
const nt = this.deps.templates.get(f.nativeTemplateId);
if (!nt.native?.compositionId) {
throw new HtmlVideoError(
'template-invalid',
`Native template "${f.nativeTemplateId}" has no native.compositionId in its metadata`,
);
}
if (!nt.__dir) {
throw new HtmlVideoError(
'template-invalid',
`Native template "${f.nativeTemplateId}" has no __dir; was it loaded via TemplateRegistry?`,
);
}
return {
engine: 'remotion',
templateRef: {
id: `frame-${f.graphNodeId}`,
engine: 'remotion',
sourcePath: join(nt.__dir, nt.source_entry),
mode: 'native',
nativeCompositionId: nt.native.compositionId,
},
};
}
const engine = f.engine ?? projectEngine;
return {
engine,
templateRef: { id: `frame-${f.graphNodeId}`, engine, sourcePath: f.htmlPath },
};
}
/**
* Enhance one data frame with a native engine template (the user-initiated
* "motion enhancement" — RFC-08/09). Snapshots the source DataNode's `data`
* onto the frame and points it at the native template. Asserts the node is a
* `data` node and that its data fits the native template's expected shape, so
* export doesn't later render NaN bars. The frame's `htmlPath` is untouched,
* so {@link unenhanceFrame} fully reverts it.
*/
async enhanceFrameNative(
projectId: string,
graphNodeId: string,
nativeTemplateId: string,
): Promise<{ project: Project; frame: FrameRecord }> {
const project = await this.deps.projects.load(projectId);
const graph = await this.readContentGraph(projectId);
if (!graph) {
throw new HtmlVideoError('invalid-input', 'Project has no content graph');
}
const node = graph.nodes.find((n) => n.id === graphNodeId);
if (!node) {
throw new HtmlVideoError('invalid-input', `Graph node "${graphNodeId}" not found`);
}
if (node.kind !== 'data') {
throw new HtmlVideoError(
'invalid-input',
`Frame "${graphNodeId}" is a ${node.kind} node; native data enhancement only applies to data frames`,
);
}
const tmpl = this.deps.templates.get(nativeTemplateId); // throws if unknown
if (tmpl.engine !== 'remotion' || !tmpl.native?.compositionId) {
throw new HtmlVideoError(
'invalid-input',
`Template "${nativeTemplateId}" is not a native Remotion template`,
);
}
const data = normalizeRollupData((node as { data?: unknown }).data);
const frame = (project.frames ?? []).find((f) => f.graphNodeId === graphNodeId);
if (!frame) {
throw new HtmlVideoError(
'invalid-input',
`Frame "${graphNodeId}" has not been rendered yet (no FrameRecord)`,
);
}
frame.engine = 'remotion';
frame.nativeTemplateId = nativeTemplateId;
frame.data = data;
await this.deps.projects.save(project);
return { project, frame };
}
/**
* Revert a frame's native enhancement back to its base hyperframes HTML.
* Clears the three enhance fields; `htmlPath` was never touched. (RFC-08/09)
*/
async unenhanceFrame(
projectId: string,
graphNodeId: string,
): Promise<{ project: Project; frame: FrameRecord }> {
const project = await this.deps.projects.load(projectId);
const frame = (project.frames ?? []).find((f) => f.graphNodeId === graphNodeId);
if (!frame) {
throw new HtmlVideoError('invalid-input', `Frame "${graphNodeId}" not found`);
}
delete frame.engine;
delete frame.nativeTemplateId;
delete frame.data;
delete frame.previewMp4Path; // stop advertising a now-stale preview video
await this.deps.projects.save(project);
return { project, frame };
}
/**
* Render a single (enhanced) frame to a short MP4 for studio preview. A native
* frame has no HTML to show in the iframe strip, so the studio renders it on
* its own and plays the result as a <video>. Reuses {@link resolveFrameTemplateRef}
* — the same per-frame engine/template resolution exportMp4 uses — so the
* preview is pixel-identical to what the final export will stitch in.
*
* Writes to `frames/<order>.preview.mp4` (distinct from export's `frames/NN.mp4`
* so the two never overwrite each other). No soundtrack mux — a per-frame
* preview is silent and faster. Sets `frame.previewMp4Path` and saves (bumping
* `updatedAt`, which the studio uses as the <video> cache-bust token).
*/
async renderFrameNativePreview(args: {
projectId: string;
graphNodeId: string;
onProgress?: (pct: number, stage: string) => void;
signal?: AbortSignal;
}): Promise<{ project: Project; frame: FrameRecord; previewPath: string }> {
const project = await this.deps.projects.load(args.projectId);
const projectDir = await this.deps.projects.ensureDir(project.id);
const frame = (project.frames ?? []).find((f) => f.graphNodeId === args.graphNodeId);
if (!frame) {
throw new HtmlVideoError('invalid-input', `Frame "${args.graphNodeId}" not found`);
}
const tmpl = project.templateId ? this.deps.templates.get(project.templateId) : null;
const projectEngine = tmpl?.engine ?? 'hyperframes';
const { engine, templateRef } = this.resolveFrameTemplateRef(frame, projectEngine);
const adapter = this.deps.engines.get(engine);
const previewPath = join(projectDir, 'frames', `${String(frame.order + 1).padStart(2, '0')}.preview.mp4`);
await adapter.render(
{
template: templateRef,
variables: frame.data !== undefined ? { ...project.variables, data: frame.data } : project.variables,
config: {
format: 'mp4',
resolution: project.preferences.resolution ?? { width: 1920, height: 1080 },
fps: project.preferences.fps ?? 60,
duration: frame.durationSec,
durationMode: 'explicit',
outputPath: previewPath,
},
},
{
workDir: projectDir,
...(args.onProgress !== undefined && { onProgress: args.onProgress }),
...(args.signal !== undefined && { signal: args.signal }),
},
);
frame.previewMp4Path = previewPath;
await this.deps.projects.save(project);
return { project, frame, previewPath };
}
/**
* If the project has a soundtrack (music and/or narration), mux it into the
* just-rendered video at `outputPath`. Renders to a temp file then renames
* over the original. No-op when there's no soundtrack. Audio generation
* never depends on ffmpeg — only this export-time mux does.
*/
private async applySoundtrack(
project: Project,
outputPath: string,
videoDurationSec: number | undefined,
onProgress?: (pct: number, stage: string) => void,
): Promise<void> {
const st = project.soundtrack;
if (!st || (!st.musicAssetId && !st.narrationAssetId)) return;
const findPath = (id?: string): string | undefined =>
id ? project.assets.find((a) => a.id === id)?.path : undefined;
const musicPath = findPath(st.musicAssetId);
const narrationPath = findPath(st.narrationAssetId);
if (!musicPath && !narrationPath) return; // referenced assets are gone
onProgress?.(99, 'mixing audio');
const { rename } = await import('node:fs/promises');
const tmpOut = `${outputPath}.muxed.mp4`;
// MiniMax music is a fixed ~50s clip regardless of request; `-shortest`
// already trims it to the video length, but a hard cut sounds abrupt.
// Default a gentle fade-out (≤ a third of the clip, capped 1.5s) when the
// user hasn't set one and we know the video length.
const defaultFadeOut =
musicPath && videoDurationSec && videoDurationSec > 2
? Math.min(1.5, videoDurationSec / 3)
: 0;
const fadeOutSec = st.fadeOutSec ?? defaultFadeOut;
await muxAudioWithFfmpeg({
videoPath: outputPath,
outputPath: tmpOut,
...(musicPath !== undefined && { musicPath }),
...(narrationPath !== undefined && { narrationPath }),
...(st.musicVolumeDb !== undefined && { musicVolumeDb: st.musicVolumeDb }),
...(st.narrationVolumeDb !== undefined && { narrationVolumeDb: st.narrationVolumeDb }),
...(st.fadeInSec !== undefined && { fadeInSec: st.fadeInSec }),
...(fadeOutSec > 0 && { fadeOutSec }),
...(videoDurationSec !== undefined && { videoDurationSec }),
});
await rename(tmpOut, outputPath);
}
}
// ---------------------------------------------------------------------------
// ffmpeg concat helper
// ---------------------------------------------------------------------------
/**
* Coerce a content-graph DataNode's free-form `data` into the shape the native
* frame-data-rollup template expects ({ title?, unit?, items: {label,value}[] }).
* DataNode.data is `unknown`, so an enhanced frame could otherwise feed NaN bars
* to the renderer. Accepts either the already-shaped object or a bare array of
* {label,value}. Throws a clear error rather than rendering garbage.
*/
function normalizeRollupData(raw: unknown): { title?: string; unit?: string; items: { label: string; value: number }[] } {
const asItems = (arr: unknown): { label: string; value: number }[] => {
if (!Array.isArray(arr)) {
throw new HtmlVideoError(
'invalid-input',
'Data frame has no `items` array to animate; expected {items:[{label,value}]}',
);
}
const items = arr
.map((it) => {
const o = (it ?? {}) as Record<string, unknown>;
const label = String(o.label ?? o.name ?? '');
const value = Number(o.value ?? o.y ?? o.count);
return { label, value };
})
.filter((it) => it.label !== '' && Number.isFinite(it.value));
if (items.length === 0) {
throw new HtmlVideoError(
'invalid-input',
'Data frame items had no usable {label, numeric value} pairs',
);
}
return items;
};
if (Array.isArray(raw)) return { items: asItems(raw) };
const o = (raw ?? {}) as Record<string, unknown>;
const out: { title?: string; unit?: string; items: { label: string; value: number }[] } = {
items: asItems(o.items),
};
if (typeof o.title === 'string') out.title = o.title;
if (typeof o.unit === 'string') out.unit = o.unit;
return out;
}
/**
* Concatenate per-frame MP4 files into a single output with ffmpeg.
*
* Two strategies:
* - Single-engine (default): the concat **demuxer** with `-c copy`. All frames
* came from one engine so their h264 streams are byte-compatible — fast, no
* re-encode.
* - Mixed-engine (`opts.reencode`): a hyperframes frame next to a native
* Remotion frame can differ in profile/GOP/**timebase**. The concat demuxer
* assumes continuous timestamps across segments and mis-accumulates the
* Remotion segment's PTS, ballooning the total duration. So we feed each
* segment as an independent input and join with the concat **filter**, which
* rebuilds a clean timeline, then re-encode to a uniform h264.
*
* Requires `ffmpeg` on PATH. Throws with a friendly hint if missing.
*/
async function concatFramesWithFfmpeg(
frameMp4s: string[],
outputPath: string,
workDir: string,
opts: { reencode?: boolean; fps?: number } = {},
): Promise<void> {
if (frameMp4s.length === 0) {
throw new HtmlVideoError('render-failed', 'No frames to concat');
}
const { writeFile } = await import('node:fs/promises');
const { join } = await import('node:path');
const { spawn } = await import('node:child_process');
const fps = opts.fps ?? 60;
let ffmpegArgs: string[];
if (opts.reencode) {
// concat FILTER: independent `-i` per segment + filter rebuilds the timeline.
const n = frameMp4s.length;
const inputs = frameMp4s.flatMap((p) => ['-i', p]);
const filter = `${frameMp4s.map((_, i) => `[${i}:v]`).join('')}concat=n=${n}:v=1:a=0[v]`;
ffmpegArgs = [
'-y',
...inputs,
'-filter_complex', filter,
'-map', '[v]',
'-c:v', 'libx264',
'-pix_fmt', 'yuv420p',
'-r', String(fps),
'-movflags', '+faststart',
outputPath,
];
} else {
// concat DEMUXER + stream copy: needs the list file.
const listPath = join(workDir, 'frames', 'concat.txt');
const list = frameMp4s.map((p) => `file '${p.replace(/'/g, "'\\''")}'`).join('\n');
await writeFile(listPath, list, 'utf8');
ffmpegArgs = ['-y', '-f', 'concat', '-safe', '0', '-i', listPath, '-c', 'copy', outputPath];
}
await new Promise<void>((resolveFn, reject) => {
const proc = spawn('ffmpeg', ffmpegArgs, { stdio: ['ignore', 'pipe', 'pipe'] });
let stderr = '';
proc.stderr.on('data', (chunk: Buffer) => {
stderr += chunk.toString('utf8');
});
proc.on('error', (err: NodeJS.ErrnoException) => {
if (err.code === 'ENOENT') {
reject(
new HtmlVideoError(
'render-failed',
'ffmpeg not found on PATH. Install with `brew install ffmpeg` (macOS) or your platform equivalent.',
),
);
} else {
reject(err);
}
});
proc.on('exit', (code: number | null) => {
if (code === 0) resolveFn();
else
reject(
new HtmlVideoError(
'render-failed',
`ffmpeg concat exited with code ${code}: ${stderr.slice(-2000)}`,
),
);
});
});
}
/**
* Mix a background-music track and/or a narration track into a (silent) video
* file, writing the result to `outputPath`. Video is stream-copied (no
* re-encode); audio is encoded to AAC. Music is ducked under narration via a
* volume offset, optional fade in/out is applied to the music, and `-shortest`
* keeps the result aligned to the video length.
*
* `videoPath` and `outputPath` must differ. Throws HtmlVideoError on ffmpeg
* failure; a missing ffmpeg yields the same friendly hint as concat.
*/
async function muxAudioWithFfmpeg(args: {
videoPath: string;
outputPath: string;
musicPath?: string;
narrationPath?: string;
musicVolumeDb?: number;
narrationVolumeDb?: number;
fadeInSec?: number;
fadeOutSec?: number;
videoDurationSec?: number;
}): Promise<void> {
const { spawn } = await import('node:child_process');
const hasMusic = !!args.musicPath;
const hasNarration = !!args.narrationPath;
if (!hasMusic && !hasNarration) return; // nothing to mix
const musicVol = args.musicVolumeDb ?? -18;
const narrVol = args.narrationVolumeDb ?? 0;
const fadeIn = args.fadeInSec ?? 0;
const fadeOut = args.fadeOutSec ?? 0;
// Inputs: [0] video, then music / narration in order.
const inputs: string[] = ['-i', args.videoPath];
let musicIdx = -1;
let narrIdx = -1;
let next = 1;
if (hasMusic) { inputs.push('-i', args.musicPath!); musicIdx = next++; }
if (hasNarration) { inputs.push('-i', args.narrationPath!); narrIdx = next++; }
// Build a filter graph producing a single [aout] label.
const filters: string[] = [];
const mixLabels: string[] = [];
if (hasMusic) {
let chain = `[${musicIdx}:a]volume=${musicVol}dB`;
if (fadeIn > 0) chain += `,afade=t=in:st=0:d=${fadeIn}`;
// Fade-out only when we know where the end is.
if (fadeOut > 0 && args.videoDurationSec && args.videoDurationSec > fadeOut) {
chain += `,afade=t=out:st=${(args.videoDurationSec - fadeOut).toFixed(2)}:d=${fadeOut}`;
}
chain += '[bg]';
filters.push(chain);
mixLabels.push('[bg]');
}
if (hasNarration) {
filters.push(`[${narrIdx}:a]volume=${narrVol}dB[vo]`);
mixLabels.push('[vo]');
}
if (mixLabels.length === 2) {
filters.push(`${mixLabels[0]}${mixLabels[1]}amix=inputs=2:duration=longest:dropout_transition=0[aout]`);
} else {
// single source → relabel to [aout]
filters.push(`${mixLabels[0]}anull[aout]`);
}
const ffArgs = [
'-y',
...inputs,
'-filter_complex', filters.join(';'),
'-map', '0:v',
'-map', '[aout]',
'-c:v', 'copy',
'-c:a', 'aac',
'-b:a', '192k',
'-shortest',
args.outputPath,
];
await new Promise<void>((resolveFn, reject) => {
const proc = spawn('ffmpeg', ffArgs, { stdio: ['ignore', 'pipe', 'pipe'] });
let stderr = '';
proc.stderr.on('data', (chunk: Buffer) => { stderr += chunk.toString('utf8'); });
proc.on('error', (err: NodeJS.ErrnoException) => {
if (err.code === 'ENOENT') {
reject(new HtmlVideoError(
'render-failed',
'ffmpeg not found on PATH. Install with `brew install ffmpeg` (macOS) or your platform equivalent.',
));
} else {
reject(err);
}
});
proc.on('exit', (code: number | null) => {
if (code === 0) resolveFn();
else reject(new HtmlVideoError('render-failed', `ffmpeg audio mux exited with code ${code}: ${stderr.slice(-2000)}`));
});
});
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
/** Append this export to the project's history (newest last, de-duped by path,
* capped so it doesn't grow unbounded). */
function recordExport(project: Project, outputPath: string): void {
const list = (project.exports ?? []).filter((e) => e.path !== outputPath);
list.push({ path: outputPath, filename: basename(outputPath), createdAt: new Date().toISOString() });
// Keep the most recent 20.
project.exports = list.slice(-20);
}
function templateRefFromMeta(meta: TemplateMetadata) {
if (!meta.__dir) {
throw new HtmlVideoError(
'template-invalid',
`Template ${meta.id} has no __dir set; was it loaded via TemplateRegistry?`,
);
}
return {
id: meta.id,
engine: meta.engine,
sourcePath: join(meta.__dir, meta.source_entry),
};
}
function downgradeStatus(current: ProjectStatus, target: ProjectStatus): ProjectStatus {
// After any modification, status should not be more advanced than 'draft'/given target.
// 'rendered' / 'previewed' get demoted back to 'draft' on any meaningful change.
if (target === 'draft') return 'draft';
return current;
}
packages/core/src/registry.ts
+217
View File
@@ -0,0 +1,217 @@
/**
* Registries for engine adapters, templates, and projects.
* RFC-05: Storyboard removed; Project takes its place.
*/
import { mkdir, readFile, readdir, writeFile, rm } from 'node:fs/promises';
import { existsSync } from 'node:fs';
import { join } from 'node:path';
import { parse as parseYaml } from 'yaml';
import type {
EngineAdapter,
EngineId,
Project,
TemplateMetadata,
} from './types/index.js';
import { HtmlVideoError } from './errors.js';
// ---------------------------------------------------------------------------
// EngineRegistry
// ---------------------------------------------------------------------------
export class EngineRegistry {
private adapters = new Map<EngineId, EngineAdapter>();
register(adapter: EngineAdapter): void {
this.adapters.set(adapter.id, adapter);
}
get(id: EngineId): EngineAdapter {
const a = this.adapters.get(id);
if (!a) {
throw new HtmlVideoError(
'engine-not-registered',
`Engine "${id}" is not registered. Did you forget to install @html-video/adapter-${id}?`,
);
}
return a;
}
list(): EngineAdapter[] {
return [...this.adapters.values()];
}
has(id: EngineId): boolean {
return this.adapters.has(id);
}
}
// ---------------------------------------------------------------------------
// TemplateRegistry
// ---------------------------------------------------------------------------
export class TemplateRegistry {
private templates = new Map<string, TemplateMetadata>();
async scan(rootDir: string): Promise<TemplateMetadata[]> {
if (!existsSync(rootDir)) return [];
const entries = await readdir(rootDir, { withFileTypes: true });
const found: TemplateMetadata[] = [];
for (const entry of entries) {
if (!entry.isDirectory()) continue;
const dir = join(rootDir, entry.name);
const yamlPath = join(dir, 'template.html-video.yaml');
if (!existsSync(yamlPath)) continue;
const raw = await readFile(yamlPath, 'utf8');
const meta = parseYaml(raw) as TemplateMetadata;
meta.__dir = dir;
this.templates.set(meta.id, meta);
found.push(meta);
}
return found;
}
get(id: string): TemplateMetadata {
const t = this.templates.get(id);
if (!t) {
throw new HtmlVideoError('template-not-found', `Template "${id}" not found`);
}
return t;
}
has(id: string): boolean {
return this.templates.has(id);
}
list(): TemplateMetadata[] {
return [...this.templates.values()];
}
search(opts: {
intent?: string;
aspect?: string;
licenseAllow?: string[];
enginesAvailable?: EngineId[];
top?: number;
}): { template: TemplateMetadata; score: number; reason: string }[] {
const top = opts.top ?? 5;
const intentLower = (opts.intent ?? '').toLowerCase();
const intentTokens = intentLower.split(/\W+/).filter((s) => s.length > 2);
const ranked: { template: TemplateMetadata; score: number; reason: string }[] = [];
for (const t of this.templates.values()) {
const reasonParts: string[] = [];
let score = 0;
const haystack = [
...t.tags,
...t.best_for,
t.name,
t.description,
t.category,
t.subcategory ?? '',
]
.join(' ')
.toLowerCase();
const matched = intentTokens.filter((tok) => haystack.includes(tok));
if (matched.length > 0) {
score += matched.length * 0.2;
reasonParts.push(`matched ${matched.length} intent tokens`);
}
if (opts.aspect) {
if (t.output.resolution.supported_aspects.includes(opts.aspect)) {
score += 0.15;
reasonParts.push(`aspect ${opts.aspect} supported`);
} else {
score -= 0.1;
}
}
if (opts.licenseAllow && !opts.licenseAllow.includes(t.license.spdx)) {
continue;
}
reasonParts.push(`license ${t.license.spdx} ok`);
if (opts.enginesAvailable && !opts.enginesAvailable.includes(t.engine)) {
continue;
}
score = Math.max(0, Math.min(1, score));
ranked.push({
template: t,
score,
reason: reasonParts.join('; '),
});
}
ranked.sort((a, b) => b.score - a.score);
return ranked.slice(0, top);
}
}
// ---------------------------------------------------------------------------
// ProjectStore — JSON-on-disk persistence
// ---------------------------------------------------------------------------
export class ProjectStore {
constructor(private projectRoot: string) {}
private dir(): string {
return join(this.projectRoot, '.html-video', 'projects');
}
private projectDir(id: string): string {
return join(this.dir(), id);
}
private path(id: string): string {
return join(this.projectDir(id), 'project.json');
}
/** Ensure project directory exists; returns its absolute path. */
async ensureDir(id: string): Promise<string> {
const dir = this.projectDir(id);
await mkdir(join(dir, 'assets'), { recursive: true });
return dir;
}
async save(project: Project): Promise<void> {
await this.ensureDir(project.id);
project.updatedAt = new Date().toISOString();
await writeFile(this.path(project.id), JSON.stringify(project, null, 2), 'utf8');
}
async load(id: string): Promise<Project> {
const p = this.path(id);
if (!existsSync(p)) {
throw new HtmlVideoError('project-not-found', `Project ${id} not found`);
}
return JSON.parse(await readFile(p, 'utf8')) as Project;
}
async list(): Promise<Project[]> {
const d = this.dir();
if (!existsSync(d)) return [];
const ids = await readdir(d);
const out: Project[] = [];
for (const id of ids) {
try {
out.push(await this.load(id));
} catch {
// skip corrupt
}
}
out.sort((a, b) => b.updatedAt.localeCompare(a.updatedAt));
return out;
}
async remove(id: string): Promise<void> {
const dir = this.projectDir(id);
if (existsSync(dir)) {
await rm(dir, { recursive: true, force: true });
}
}
}
packages/core/src/types/index.ts
+439
View File
@@ -0,0 +1,439 @@
/**
* @html-video/core type definitions
* Implements RFC-01 (engine adapter) + RFC-02 (template metadata) + RFC-05 (project-centric workflow).
* See research/2026-05-{26,27}-spec-{01,02,05}-*.md.
*
* NOTE: Storyboard / Scene types from RFC-04 were removed in v0.1
* after Joey's product clarification — see RFC-05.
*/
// ============================================================================
// RFC-01: Engine Adapter
// ============================================================================
export type EngineId = string;
export type Paradigm =
| 'html-css-gsap'
| 'react-tsx'
| 'ts-generator'
| 'json-scene'
| 'imperative-canvas';
export type OutputFormat = 'mp4' | 'webm' | 'webm-alpha' | 'gif' | 'png-sequence' | 'apng';
export type RenderTarget = 'local-chromium' | 'local-canvas' | 'lambda' | 'cloud-run';
export type LicensingTier = 'free-osi' | 'commercial-restricted' | 'unknown';
export interface RenderSpeedHint {
resolution: string;
durationSec: number;
fps: number;
estimatedRenderSec: number;
}
export interface EngineCapabilities {
paradigms: Paradigm[];
outputFormats: OutputFormat[];
maxResolution: { width: number; height: number };
alpha: boolean;
audio: 'none' | 'single' | 'multi';
subtitles: ('none' | 'burn-in' | 'sidecar')[];
renderTarget: RenderTarget[];
licensing: LicensingTier;
renderSpeedHint?: RenderSpeedHint;
bestFor: string[];
weaknesses: string[];
}
export interface ValidationError {
code: string;
message: string;
fix?: string;
}
export interface ValidationResult {
ok: boolean;
errors: ValidationError[];
warnings: ValidationError[];
}
export interface RenderConfig {
format: OutputFormat;
resolution: { width: number; height: number };
fps: number;
duration: number | 'auto';
/**
* How to treat `duration`. 'explicit' = the user set a per-frame length; it is
* a hard cap, do NOT extend the recording to fit a longer animation. 'auto'
* (default) = a fallback the renderer may extend so an opening animation isn't
* cut mid-play. Multi-frame export sets 'explicit' (the format card collected a
* real per-frame value); single-frame fast preview leaves it 'auto'.
*/
durationMode?: 'explicit' | 'auto';
outputPath: string;
alpha?: boolean;
quality?: number | 'low' | 'medium' | 'high' | 'lossless';
audio?: { path: string; volumeDb?: number }[];
}
export interface RenderInput {
template: TemplateRef;
variables: Record<string, unknown>;
config: RenderConfig;
}
export interface RenderContext {
workDir: string;
onProgress?: (pct: number, stage: string) => void;
signal?: AbortSignal;
env?: Record<string, string>;
}
export interface RenderOutput {
outputPath: string;
meta: {
durationSec: number;
fileSizeBytes: number;
actualResolution: { width: number; height: number };
fps: number;
renderedFrames: number;
renderWallClockSec: number;
engineVersion: string;
};
diagnostics: string[];
}
export interface PreviewContext {
workDir: string;
hostname?: string;
port?: number;
}
export interface PreviewHandle {
url: string;
port: number;
close(): Promise<void>;
}
export interface NativeTemplateRef {
nativeId: string;
path: string;
hints?: { name?: string; description?: string; bestFor?: string[] };
}
export interface HtmlSceneOutput {
htmlPath: string;
referencedAssets: { assetId: string; usagePath: string }[];
posterPath: string;
durationSec: number;
}
export interface EngineAdapter {
id: EngineId;
name: string;
upstreamVersion: string;
capabilities: EngineCapabilities;
validate(template: TemplateRef): ValidationResult;
render(input: RenderInput, ctx: RenderContext): Promise<RenderOutput>;
preview?(template: TemplateRef, ctx: PreviewContext): Promise<PreviewHandle>;
renderToHtml?(input: RenderInput, ctx: RenderContext): Promise<HtmlSceneOutput>;
listNativeTemplates?(): Promise<NativeTemplateRef[]>;
}
// ============================================================================
// RFC-02: Template Metadata
// ============================================================================
export type TemplateCategory =
| 'data-viz'
| 'social-shorts'
| 'product-demo'
| 'explainer'
| 'marketing'
| 'intro-outro'
| 'ambient'
| 'documentary'
| 'presentation'
| 'transition';
export interface OutputCapabilities {
formats: OutputFormat[];
default_format: OutputFormat;
resolution: {
default: { width: number; height: number };
supported_aspects: string[];
};
fps: { default: number; supported: number[] };
duration: {
type: 'variable' | 'fixed';
min_sec: number;
max_sec: number;
};
alpha: boolean;
audio: { supported: boolean; expected_inputs?: string[] };
}
export interface LicenseInfo {
spdx: string;
attribution_required: boolean;
redistribution_allowed: boolean;
commercial_use: boolean;
notes?: string | null;
}
export interface AssetAttribution {
name: string;
license: string;
author?: string;
url?: string;
}
export interface ChangelogEntry {
version: string;
date: string;
notes: string;
}
export interface PerformanceRef {
duration_sec: number;
render_wall_clock_sec: number;
machine: string;
}
/**
* Three-layer provenance (RFC-07). Records where a template's design actually
* came from so the studio can surface honest attribution:
* - origin — L1: the real-world design inspiration (a studio, person, or
* movement). `name: 'none'` / `kind: 'none'` when there is no
* specific upstream source (e.g. an original skill preset).
* - via_skill — L2: the open-source skill we actually transformed from; its
* license governs redistribution. author = the real copyright
* holder verified against the upstream LICENSE.
* - transformation — L3: what html-video changed (free-text).
*/
export interface ProvenanceOrigin {
name: string;
kind?: 'studio' | 'person' | 'movement' | 'none';
reference?: string;
}
export interface ProvenanceViaSkill {
name: string;
author?: string;
url?: string;
license?: string;
source_file?: string;
}
export interface Provenance {
origin?: ProvenanceOrigin;
via_skill?: ProvenanceViaSkill;
transformation?: string;
}
export interface TemplateMetadata {
spec_version: 1;
id: string;
name: string;
description: string;
engine: EngineId;
engine_version: string;
source_entry: string;
/**
* Native-engine templates (RFC-08 Phase 2): `source_entry` is a bundle entry
* (e.g. a Remotion `entry.ts` calling `registerRoot`) rather than an HTML
* file. `compositionId` is the `<Composition id>` the adapter selects. Absent
* for the classic HTML+CSS+GSAP templates (the 27 hyperframes ones).
*/
native?: { compositionId: string };
category: TemplateCategory;
subcategory?: string;
tags: string[];
best_for: string[];
not_for?: string[];
output: OutputCapabilities;
inputs: { schema: object; examples: object[] };
license: LicenseInfo;
/** Three-layer design attribution (RFC-07). See {@link Provenance}. */
provenance?: Provenance;
assets_attribution?: AssetAttribution[];
author: { name: string; url?: string; contact?: string };
maintainers?: { github: string }[];
contributing?: { url: string };
version: string;
changelog?: ChangelogEntry[];
preview: { poster: string; loop?: string; thumbnail?: string };
performance?: { reference_render: PerformanceRef };
share_optimized_for?: string[];
/** Internal: filesystem location of the template directory (set by registry) */
__dir?: string;
}
export interface TemplateRef {
id: string;
engine: EngineId;
/** Bridge mode: an HTML file. Native mode: a Remotion .tsx bundle entry. */
sourcePath: string;
variables?: Record<string, unknown>;
/**
* How the engine should consume `sourcePath`. Absent ⇒ 'bridge' (the legacy
* path: render the given HTML). 'native' ⇒ the engine bundles `sourcePath` as
* its own component entry and renders `nativeCompositionId` directly (e.g. a
* Remotion React-tsx data-animation template fed real data via `variables`).
* Only the Remotion adapter honors 'native' today (RFC-08 Phase 2).
*/
mode?: 'bridge' | 'native';
/** Native mode only: the Remotion `<Composition id>` to select after bundling. */
nativeCompositionId?: string;
}
// ============================================================================
// RFC-05: Project-centric workflow
// ============================================================================
export type AssetType = 'image' | 'text' | 'data' | 'audio' | 'video' | 'reference-link';
export interface Asset {
id: string;
type: AssetType;
path?: string;
content?: string;
metadata: {
filename?: string;
mimeType?: string;
sizeBytes?: number;
width?: number;
height?: number;
durationSec?: number;
userCaption?: string;
};
userTags: string[];
}
export interface UserPreferences {
aspect?: string;
durationTargetSec?: number;
format?: 'mp4' | 'webm';
resolution?: { width: number; height: number };
fps?: number;
mood?: string;
brandColors?: string[];
fontFamilies?: string[];
language?: string;
commercial?: boolean;
}
export type ProjectStatus = 'draft' | 'previewed' | 'rendered';
/**
* v0.8: a single rendered HTML frame in a multi-frame project.
* Maps 1:1 to a node in the project's contentGraph (graphNodeId).
*/
export interface FrameRecord {
/** Stable id, mirrors the graph node id */
graphNodeId: string;
/** Absolute path to the rendered HTML file (e.g. .../frames/01-intro.html) */
htmlPath: string;
/** Playback duration for this frame, seconds */
durationSec: number;
/** Optional poster image (first-frame thumbnail) */
posterPath?: string;
/** 0-based index in topo-sorted play order */
order: number;
/**
* Per-frame engine override (RFC-08/09). Absent ⇒ inherit the project's
* template engine (hyperframes). Set to 'remotion' when the user explicitly
* opts this frame into a native motion-enhancement. The base render at
* `htmlPath` is always retained so toggling enhancement off is non-destructive.
*/
engine?: EngineId;
/**
* When enhanced: which native template renders this frame (e.g.
* 'frame-data-rollup'). Resolved against the TemplateRegistry at export time.
*/
nativeTemplateId?: string;
/**
* When enhanced: snapshot of the source content-graph DataNode's `data`,
* bound at enhance time and fed to the native template as inputProps. Kept on
* the frame so export is self-contained and doesn't re-read the graph.
*/
data?: unknown;
/**
* When enhanced: path to a short single-frame MP4 the studio renders so the
* user can preview the native animation before a full export (a native frame
* has no HTML to load in an iframe). Cleared on unenhance. Separate from the
* export loop's `frames/NN.mp4` so the two lifecycles don't collide.
*/
previewMp4Path?: string;
}
/**
* v0.9: project-level soundtrack — one background music track + one narration
* track mixed into the exported MP4. Both reference an entry in `assets[]`
* (type 'audio'); this struct only holds the ids + mix preferences, so the
* audio bytes live in the normal asset store. Per-frame audio is a v2 concern.
*/
export interface ProjectSoundtrack {
/** asset.id of the background-music track (type 'audio'), if generated */
musicAssetId?: string;
/** asset.id of the narration / voiceover track, if generated */
narrationAssetId?: string;
/** Background-music gain in dB applied at mux time (default -18, pushed under voice) */
musicVolumeDb?: number;
/** Narration gain in dB (default 0) */
narrationVolumeDb?: number;
/** Last music style prompt used — kept so the UI can show / re-run it */
musicPrompt?: string;
/** Last narration text used (the stitched full script) */
narrationText?: string;
/** Per-frame narration: { [graphNodeId]: line }. The UI edits/shows narration
* per selected frame; narrationText is these stitched in frame order. */
narrationByFrame?: Record<string, string>;
/** Optional music fade-in seconds at the start of the video */
fadeInSec?: number;
/** Optional music fade-out seconds at the end of the video */
fadeOutSec?: number;
}
export interface Project {
id: string;
name: string;
intent?: string;
assets: Asset[];
templateId: string | null;
/** Agent runtime to use (detected agent id, e.g. "claude" / "cursor-agent"). null = default first available */
agentId?: string | null;
/** Model id for agents that support model selection (e.g. AMR: deepseek-v4-flash,
* claude-opus-4.8…). null = agent's default. */
agentModel?: string | null;
/**
* Free-form variables (RFC-02 inputs.schema compatible).
* v0.3+: deprecated as the user-facing primary surface — agents now produce HTML directly.
* Kept for adapter render() backward compatibility (engine still expects vars).
*/
variables: Record<string, unknown>;
preferences: UserPreferences;
status: ProjectStatus;
/** Path to the latest agent-generated HTML (v0.3 chat-to-HTML pipeline; single-frame fast path) */
lastPreviewHtmlPath?: string;
lastPreviewPosterPath?: string;
lastOutputMp4Path?: string;
/** Export history — every MP4 exported for this project, newest last. Each
* export writes a uniquely-named file so older ones aren't overwritten. */
exports?: Array<{ path: string; createdAt: string; filename: string }>;
/**
* v0.8: path to content-graph.json for multi-frame projects.
* Absent for single-frame fast-path projects.
*/
contentGraphPath?: string;
/**
* v0.8: rendered frame sequence in topo-sorted play order.
* Empty for single-frame fast-path projects.
*/
frames?: FrameRecord[];
/** v0.9: optional background music + narration mixed into the export. */
soundtrack?: ProjectSoundtrack;
createdAt: string;
updatedAt: string;
}
packages/core/tsconfig.json
+10
View File
@@ -0,0 +1,10 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "dist",
"composite": false,
"noEmit": false
},
"include": ["src/**/*"]
}
packages/project-studio/package.json
+15
View File
@@ -0,0 +1,15 @@
{
"name": "@html-video/project-studio",
"version": "0.1.0",
"description": "Browser studio UI for html-video — three-pane projects / assets / template+preview",
"license": "Apache-2.0",
"type": "module",
"files": ["public", "README.md"],
"exports": {
"./public": "./public/index.html"
},
"scripts": {
"build": "echo 'project-studio is static; nothing to build'",
"typecheck": "echo 'project-studio is static html'"
}
}
packages/project-studio/public/agent-icons/aider.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.5 KiB

packages/project-studio/public/agent-icons/amr.svg
+8
View File
@@ -0,0 +1,8 @@
<svg width="61" height="61" viewBox="0 0 61 61" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M59.8711 30.5C59.8711 56.0969 55.968 60 30.3711 60C4.7742 60 0.871094 56.0969 0.871094 30.5C0.871094 4.9031 4.7742 1 30.3711 1C55.968 1 59.8711 4.9031 59.8711 30.5Z" fill="#87EA5C"/>
<path d="M34.5271 26.9915C34.4226 27.6369 33.6061 27.8574 33.1978 27.3504L20.4888 11.5716C19.9658 10.9223 20.7318 10.0242 21.4533 10.4408L30.951 15.9243L36.3612 6.55362C36.7806 5.82706 37.889 6.21903 37.7551 7.04658L34.5271 26.9915Z" fill="#237F00"/>
<path d="M34.9157 33.3782C34.2696 33.4783 33.8076 32.7699 34.1636 32.2249L45.2428 15.262C45.6988 14.564 46.7896 15.0149 46.6164 15.8299L44.3362 26.5572L54.9201 28.8069C55.7407 28.9813 55.7104 30.1565 54.882 30.2849L34.9157 33.3782Z" fill="#237F00"/>
<path d="M28.9603 35.7227C28.6654 35.1391 29.1964 34.4809 29.8248 34.651L49.3811 39.9461C50.1858 40.1641 50.094 41.3408 49.2654 41.4279L38.3585 42.5743L39.4896 53.3353C39.5773 54.1697 38.4502 54.504 38.0721 53.7558L28.9603 35.7227Z" fill="#237F00"/>
<path d="M24.8915 30.7822C25.3554 30.3214 26.1455 30.623 26.1779 31.2732L27.1852 51.5086C27.2266 52.3413 26.079 52.6177 25.7402 51.8565L21.2795 41.8377L11.3946 46.2387C10.6282 46.5799 9.96197 45.6114 10.5567 45.0206L24.8915 30.7822Z" fill="#237F00"/>
<path d="M28.3305 25.3842C28.9121 25.683 28.8694 26.5277 28.261 26.7594L9.32726 33.9705C8.54806 34.2672 7.93066 33.2612 8.54982 32.7037L16.6999 25.3653L9.45967 17.3243C8.8983 16.7008 9.61357 15.7679 10.3592 16.1509L28.3305 25.3842Z" fill="#237F00"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.5 KiB

packages/project-studio/public/agent-icons/anthropic.svg
+3
View File
@@ -0,0 +1,3 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 46 32" fill="none">
<path fill="#d97757" d="M32.73 0h-6.46L38.18 32h6.46L32.73 0ZM12.91 0 1 32h6.59l2.43-6.71h12.46L24.91 32h6.59L19.59 0h-6.68Zm-.55 19.31 4.08-11.26 4.08 11.26h-8.16Z"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 250 B

packages/project-studio/public/agent-icons/claude.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="#d97757" fill-rule="evenodd" d="M20.998 10.949H24v3.102h-3v3.028h-1.487V20H18v-2.921h-1.487V20H15v-2.921H9V20H7.488v-2.921H6V20H4.487v-2.921H3V14.05H0v-3.1h3V5h17.998zM6 10.949h1.488V8.102H6zm10.51 0H18V8.102h-1.49z" clip-rule="evenodd"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 316 B

packages/project-studio/public/agent-icons/codex.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="#fff" d="M19.503 0H4.496A4.496 4.496 0 0 0 0 4.496v15.007A4.496 4.496 0 0 0 4.496 24h15.007A4.496 4.496 0 0 0 24 19.503V4.496A4.496 4.496 0 0 0 19.503 0"/><path fill="url(#a)" d="M9.064 3.344a4.6 4.6 0 0 1 2.285-.312q1.5.173 2.673 1.275.016.015.037.021a.1.1 0 0 0 .043 0 4.55 4.55 0 0 1 3.046.275l.047.022.116.057a4.58 4.58 0 0 1 2.188 2.399q.313.765.315 1.595a4.2 4.2 0 0 1-.134 1.223.12.12 0 0 0 .03.115q.89.91 1.183 2.17.433 2.138-.887 3.854l-.136.166a4.55 4.55 0 0 1-2.201 1.388.12.12 0 0 0-.081.076c-.191.551-.383 1.023-.74 1.494-.9 1.187-2.222 1.846-3.711 1.838q-1.78-.009-3.157-1.302a.11.11 0 0 0-.105-.024c-.388.125-.78.143-1.204.138a4.44 4.44 0 0 1-1.945-.466 4.54 4.54 0 0 1-1.61-1.335c-.152-.202-.303-.392-.414-.617a6 6 0 0 1-.37-.961 4.6 4.6 0 0 1-.014-2.298.1.1 0 0 0 .006-.056.1.1 0 0 0-.027-.048 4.5 4.5 0 0 1-1.034-1.651 3.9 3.9 0 0 1-.251-1.192 5.2 5.2 0 0 1 .141-1.6Q3.659 7.92 5.086 6.97q.318-.212.601-.33a6 6 0 0 1 .646-.227.1.1 0 0 0 .065-.066 4.5 4.5 0 0 1 .829-1.615 4.54 4.54 0 0 1 1.837-1.388m3.482 10.565a.637.637 0 0 0 0 1.272h3.636a.637.637 0 1 0 0-1.272zM8.462 9.23a.637.637 0 0 0-1.106.631l1.272 2.224-1.266 2.136a.636.636 0 1 0 1.095.649l1.454-2.455a.64.64 0 0 0 .005-.64z"/><defs><linearGradient id="a" x1="12" x2="12" y1="3" y2="21" gradientUnits="userSpaceOnUse"><stop stop-color="#b1a7ff"/><stop offset=".5" stop-color="#7a9dff"/><stop offset="1" stop-color="#3941ff"/></linearGradient></defs></svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.5 KiB

packages/project-studio/public/agent-icons/copilot.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="#8534F3" d="M23.922 16.992c-.861 1.495-5.859 5.023-11.922 5.023-6.063 0-11.061-3.528-11.922-5.023A.641.641 0 0 1 0 16.736v-2.869a.841.841 0 0 1 .053-.22c.372-.935 1.347-2.292 2.605-2.656.167-.429.414-1.055.644-1.517a10.195 10.195 0 0 1-.052-1.086c0-1.331.282-2.499 1.132-3.368.397-.406.89-.717 1.474-.952 1.399-1.136 3.392-2.093 6.122-2.093 2.731 0 4.767.957 6.166 2.093.584.235 1.077.546 1.474.952.85.869 1.132 2.037 1.132 3.368 0 .368-.014.733-.052 1.086.23.462.477 1.088.644 1.517 1.258.364 2.233 1.721 2.605 2.656a.832.832 0 0 1 .053.22v2.869a.641.641 0 0 1-.078.256ZM12.172 11h-.344a4.323 4.323 0 0 1-.355.508C10.703 12.455 9.555 13 7.965 13c-1.725 0-2.989-.359-3.782-1.259a2.005 2.005 0 0 1-.085-.104L4 11.741v6.585c1.435.779 4.514 2.179 8 2.179 3.486 0 6.565-1.4 8-2.179v-6.585l-.098-.104s-.033.045-.085.104c-.793.9-2.057 1.259-3.782 1.259-1.59 0-2.738-.545-3.508-1.492a4.323 4.323 0 0 1-.355-.508h-.016.016Zm.641-2.935c.136 1.057.403 1.913.878 2.497.442.544 1.134.938 2.344.938 1.573 0 2.292-.337 2.657-.751.384-.435.558-1.15.558-2.361 0-1.14-.243-1.847-.705-2.319-.477-.488-1.319-.862-2.824-1.025-1.487-.161-2.192.138-2.533.529-.269.307-.437.808-.438 1.578v.021c0 .265.021.562.063.893Zm-1.626 0c.042-.331.063-.628.063-.894v-.02c-.001-.77-.169-1.271-.438-1.578-.341-.391-1.046-.69-2.533-.529-1.505.163-2.347.537-2.824 1.025-.462.472-.705 1.179-.705 2.319 0 1.211.175 1.926.558 2.361.365.414 1.084.751 2.657.751 1.21 0 1.902-.394 2.344-.938.475-.584.742-1.44.878-2.497Z"/><path fill="#8534F3" d="M14.5 14.25a1 1 0 0 1 1 1v2a1 1 0 0 1-2 0v-2a1 1 0 0 1 1-1Zm-5 0a1 1 0 0 1 1 1v2a1 1 0 0 1-2 0v-2a1 1 0 0 1 1-1Z"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.7 KiB

packages/project-studio/public/agent-icons/cursor-agent.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="#1c1b1a" fill-rule="evenodd" viewBox="0 0 24 24"><path d="M22.106 5.68 12.5.135a1 1 0 0 0-.998 0L1.893 5.68a.84.84 0 0 0-.419.726v11.186c0 .3.16.577.42.727l9.607 5.547a1 1 0 0 0 .998 0l9.608-5.547a.84.84 0 0 0 .42-.727V6.407a.84.84 0 0 0-.42-.726zm-.603 1.176L12.228 22.92c-.063.108-.228.064-.228-.061V12.34a.59.59 0 0 0-.295-.51l-9.11-5.26c-.107-.062-.063-.228.062-.228h18.55c.264 0 .428.286.296.514"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 455 B

packages/project-studio/public/agent-icons/gemini.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="url(#a)" d="M0 4.391A4.39 4.39 0 0 1 4.391 0h15.217A4.39 4.39 0 0 1 24 4.391v15.217A4.39 4.39 0 0 1 19.608 24H4.391A4.39 4.39 0 0 1 0 19.608z"/><path fill="#1e1e2e" fill-rule="evenodd" d="M19.74 1.444a2.816 2.816 0 0 1 2.816 2.816v15.48a2.816 2.816 0 0 1-2.816 2.816H4.26a2.816 2.816 0 0 1-2.816-2.816V4.26A2.816 2.816 0 0 1 4.26 1.444zM7.236 8.564l7.752 3.728-7.752 3.727v2.802l9.557-4.596v-3.866L7.236 5.763z" clip-rule="evenodd"/><defs><linearGradient id="a" x1="24" x2="0" y1="6.587" y2="16.494" gradientUnits="userSpaceOnUse"><stop stop-color="#ee4d5d"/><stop offset=".328" stop-color="#b381dd"/><stop offset=".476" stop-color="#207cfe"/></linearGradient></defs></svg>
Side by Side

After

Width:  |  Height:  |  Size: 745 B

packages/project-studio/public/agent-icons/grok.svg
+15
View File
@@ -0,0 +1,15 @@
<!--
Geometric "X" silhouette evoking xAI's mathematical-double-struck X
brand mark (Unicode U+1D54F 𝕏). Drawn as simple straight strokes so
AgentIcon's mono mask path can paint it in the active theme color
alongside hermes / mimo / pi / kilo / cursor-agent. The fill is set
to a baked dark color so any non-masked render path still produces
a legible mark instead of black-on-black.
-->
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1024 1024" fill="#1c1b1a">
<path d="M 158 158 L 432 432 L 158 432 L 158 158 Z" />
<path d="M 866 158 L 866 432 L 592 432 L 866 158 Z" />
<path d="M 432 592 L 158 592 L 158 866 L 432 592 Z" />
<path d="M 592 592 L 866 592 L 866 866 L 592 592 Z" />
<path d="M 432 432 L 592 432 L 592 592 L 432 592 L 432 432 Z" />
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 785 B

packages/project-studio/public/agent-icons/hermes.svg
+1
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

packages/project-studio/public/agent-icons/opencode.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="#1c1b1a" fill-rule="evenodd" viewBox="0 0 24 24"><path d="M16 6H8v12h8zm4 16H4V2h16z"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 139 B

packages/project-studio/public/agent-icons/qoder.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><circle cx="12" cy="12" r="10" fill="#22c55e"/><path fill="#fff" d="M12 5a7 7 0 1 0 0 14 7 7 0 0 0 0-14Zm0 2.5a4.5 4.5 0 1 1 0 9 4.5 4.5 0 0 1 0-9Z"/><path fill="#fff" d="m15.5 15.5 3 3" stroke="#fff" stroke-width="2" stroke-linecap="round"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 309 B

packages/project-studio/public/agent-icons/qwen.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="url(#a)" d="M12.604 1.34q.59 1.035 1.174 2.075a.18.18 0 0 0 .157.091h5.552q.26 0 .446.327l1.454 2.57c.19.337.24.478.024.837q-.39.646-.76 1.3l-.367.658c-.106.196-.223.28-.04.512l2.652 4.637c.172.301.111.494-.043.77q-.656 1.177-1.335 2.34c-.159.272-.352.375-.68.37a43 43 0 0 0-2.327.016.1.1 0 0 0-.081.05 575 575 0 0 1-2.705 4.74c-.169.293-.38.363-.725.364q-1.495.005-3.017.002a.54.54 0 0 1-.465-.271l-1.335-2.323a.09.09 0 0 0-.083-.049H4.982a1.8 1.8 0 0 1-.805-.092l-1.603-2.77a.54.54 0 0 1-.002-.54l1.207-2.12a.2.2 0 0 0 0-.197 551 551 0 0 1-1.875-3.272l-.79-1.395c-.16-.31-.173-.496.095-.965q.697-1.22 1.387-2.436c.132-.234.304-.334.584-.335a338 338 0 0 1 2.589-.001.12.12 0 0 0 .107-.063l2.806-4.895a.49.49 0 0 1 .422-.246c.524-.001 1.053 0 1.583-.006L11.704 1c.341-.003.724.032.9.34m-3.432.403a.06.06 0 0 0-.052.03L6.254 6.788a.16.16 0 0 1-.135.078H3.253q-.084 0-.041.074l5.81 10.156q.037.062-.034.063l-2.795.015a.22.22 0 0 0-.2.116l-1.32 2.31q-.066.117.068.118l5.716.008q.068 0 .104.061l1.403 2.454q.069.122.139 0l5.006-8.76.783-1.382a.055.055 0 0 1 .096 0l1.424 2.53a.12.12 0 0 0 .107.062l2.763-.02a.04.04 0 0 0 .035-.02.04.04 0 0 0 0-.04l-2.9-5.086a.11.11 0 0 1 0-.113l.293-.507 1.12-1.977q.036-.062-.035-.062H9.2q-.088 0-.043-.077l1.434-2.505a.11.11 0 0 0 0-.114L9.225 1.774a.06.06 0 0 0-.053-.031m6.29 8.02q.07 0 .034.06l-.832 1.465-2.613 4.585a.06.06 0 0 1-.05.029.06.06 0 0 1-.05-.029L8.498 9.841q-.03-.051.028-.054l.216-.012 6.722-.012z"/><defs><linearGradient id="a" x1="0%" x2="100%" y1="0%" y2="0%"><stop offset="0%" stop-color="#6336e7" stop-opacity=".84"/><stop offset="100%" stop-color="#6f69f7" stop-opacity=".84"/></linearGradient></defs></svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.7 KiB

packages/project-studio/public/app.js
+3257
View File
File diff suppressed because it is too large Load Diff
packages/project-studio/public/i18n.js
+594
View File
@@ -0,0 +1,594 @@
/**
* Tiny i18n for the studio. No build step, no framework.
*
* Usage:
* import { t, setLocale, getLocale, AVAILABLE_LOCALES } from './i18n.js';
* t('toolbar.export_mp4')
* setLocale('zh')
*
* Locale resolution order:
* 1. localStorage hv.studio.locale
* 2. navigator.language prefix ("zh-CN" → "zh")
* 3. DEFAULT_LOCALE = "en"
*
* Strings missing in the active locale fall back to en, then the key.
*/
export const DEFAULT_LOCALE = 'en';
export const AVAILABLE_LOCALES = ['en', 'zh'];
const DICT = {
en: {
'app.empty_pick_create': 'Pick or create a project',
'app.empty_subtitle':
'Each project = one HTML video. Choose a template to see the visual baseline, chat with your agent to drive the content, edit per-frame text in the middle column, see the result on the right.',
'app.no_project': 'no project',
'sidebar.projects': 'Projects',
'sidebar.new': '+ New',
'sidebar.collapse': 'Collapse sidebar',
'sidebar.empty_list': 'no projects yet',
'sidebar.menu.rename': '✎ Rename',
'sidebar.menu.delete': '🗑 Delete',
'sidebar.rename_prompt': 'New project name',
'sidebar.delete_confirm': 'Delete "{name}"? This cannot be undone.',
'toolbar.template': 'Template',
'toolbar.template_pick': 'Optional · Pick template',
'toolbar.agent': 'Agent',
'toolbar.model': 'Model',
'toolbar.agent_none': '— none —',
'toolbar.agent_ready': '● ready',
'toolbar.agent_install': '○ install',
'toolbar.export_mp4': 'Export MP4',
'composer.placeholder.no_project': 'Pick a project first…',
'composer.placeholder.detecting_agents': 'Describe the video while we check for agents…',
'composer.placeholder.no_agent': 'Install Claude Code (claude CLI) to enable chat…',
'composer.placeholder.focus':
'Edit only this frame (click ✕ on the chip above to release)…',
'composer.placeholder.no_template':
'Describe a video, or paste an article link / GitHub repo to build one from it.',
'composer.placeholder.with_template': 'Describe the video — content, names, data… or paste an article link / GitHub repo.',
'composer.hint': 'Cmd / Ctrl + Enter · drag / paste files · drop a design.md / frame.md to lock brand + motion',
'composer.send': 'Send',
'composer.attach': 'Attach file',
'composer.focus_chip': 'Editing only frame {order} {fid}',
'composer.focus_clear': 'Clear focus',
'chat.empty.title': 'Send a message to start',
'chat.empty.body':
'Tell the agent what you want — a single brand card, a multi-frame teaser, a data poster — and it will scaffold the HTML.',
'chat.summary.form_submitted': '📋 Form submitted',
'chat.summary.confirm_generate': '✓ Generate',
'chat.summary.confirm_edit': '✏️ Edit',
'chat.thinking': 'agent thinking',
'chat.still_generating': '⏳ This project is still generating in the background — its result will appear here when done (reload preview to refresh).',
'chat.placeholder.gen_html': '📄 *generating HTML…*',
'chat.placeholder.plan_graph': '🧭 *planning storyboard…*',
'chat.empty_reply':
'⚠️ The agent returned an empty reply. Try rephrasing your request — e.g. tell it the brand / topic / 1-2 concrete details, or which kind of frame you want first.',
'preview.placeholder.pick_project': 'Pick a project first.',
'preview.placeholder.pick_template':
'Send a chat to generate the first HTML.<br>Or pick a template up top for a quick start.',
'preview.edit_text_on': '✓ Done editing',
'preview.edit_text_off': '✎ Edit text',
'preview.edit_text_title': 'Click any text in the preview to edit',
'preview.edit_text_done_title': 'Finish editing',
'preview.reload': '↻ Reload preview',
'preview.no_hv_text':
'This frame has no editable text (HTML missing data-hv-text).',
'frames.label': 'Frames',
'frames.view_graph': 'View graph',
'frames.enhance': '⚡ Remotion',
'frames.enhance_hint': 'Render this data frame natively with Remotion (numbers roll, bars grow)',
'frames.enhanced_revert': '⚡ Remotion ✓ (revert)',
'frames.enhancing': '⚡ {pct}%…',
'enhance.done': '⚡ Frame rendered with Remotion',
'enhance.failed': '⚡ Remotion failed: {message}',
'text_pane.title': 'Frame text',
'text_pane.no_project': 'No project.',
'text_pane.empty_with_frames':
'No editable text on this frame. Switch to another frame, or click ✎ Edit text on the canvas.',
'text_pane.empty_no_frames':
'No editable text yet. Send a chat to generate the first version of the HTML, then per-frame text fields appear here.',
'text_pane.collapse': 'Collapse panel',
'text_pane.save_state.idle': '—',
'text_pane.save_state.typing': 'typing…',
'text_pane.save_state.saving': 'saving…',
'text_pane.save_state.saved': 'saved',
'text_pane.save_state.error': 'error',
'export.starting': '⏵ Starting MP4 export…',
'export.button_running': '⏵ {pct}% · {stage}',
'export.done_seconds': '✓ MP4 exported · {seconds}',
'export.done_no_seconds': '✓ MP4 exported',
'export.failed': '⚠️ Export failed: {message}',
'export.stream_interrupted': 'Export stream interrupted: {message}',
'export.failed_short': 'Export failed: {message}',
'export.title': '🎬 MP4 ready',
'export.reveal': 'Reveal in Finder',
'export.copy_path': 'Copy path',
'export.copied': 'Path copied',
'export.copy_failed': 'Copy failed: {message}',
'export.reveal_failed': 'Open failed: {message}',
'soundtrack.title': '🎵 Add background music & narration',
'soundtrack.summary_sub': 'AI music + voiceover, mixed into your export',
'soundtrack.optional': 'optional',
'soundtrack.hint': 'Background music + narration, mixed into the MP4 on export.',
'soundtrack.music_label': 'Background music',
'soundtrack.music_placeholder': 'Pick a style above, or describe your own — genre, mood, tempo',
'soundtrack.preset_energetic': 'Energetic',
'soundtrack.preset_calm': 'Calm',
'soundtrack.preset_tech': 'Tech',
'soundtrack.preset_narrative': 'Narrative',
'soundtrack.preset_minimal': 'Minimal',
'soundtrack.preset_epic': 'Epic',
'soundtrack.narration_label': 'Narration / voiceover',
'soundtrack.step_write': 'Write the script (text)',
'soundtrack.step_voice': 'Synthesize the voice (audio)',
'soundtrack.editing_frame': '· editing frame {n}/{total}',
'soundtrack.draft_frame': '✨ Draft this frame',
'soundtrack.draft_all': '✨ Draft all frames',
'soundtrack.gen_music': '🎵 Generate music',
'soundtrack.gen_narration': '🎙 Synthesize voiceover',
'soundtrack.empty_music': 'Pick or describe a music style first.',
'soundtrack.empty_narration': 'Add narration text first (✨ to draft it).',
'soundtrack.frame_word': 'Frame',
'soundtrack.total_word': 'Total',
'soundtrack.drafting': 'Drafting…',
'soundtrack.draft_need_frames': 'Generate the video first',
'soundtrack.draft_failed': '⚠️ Draft failed: {message}',
'soundtrack.narration_placeholder': 'One line per frame — click ✨ to draft from the video (leave empty for none)',
'soundtrack.voice_label': 'Voice',
'soundtrack.voice_male_warm': 'Male · Warm',
'soundtrack.voice_male_pro': 'Male · Professional',
'soundtrack.voice_male_deep': 'Male · Deep',
'soundtrack.voice_female_anchor': 'Female · Anchor',
'soundtrack.voice_female_mature': 'Female · Mature',
'soundtrack.voice_female_sweet': 'Female · Sweet',
'soundtrack.fit_durations': '⇄ Fit timing to narration',
'soundtrack.fit_hint': 'Re-pace each frame by how much narration it has',
'soundtrack.fitting': 'Fitting…',
'soundtrack.fitted': '✓ Frame timing fit to narration · {sec}s total',
'soundtrack.fit_failed': 'Could not fit timing',
'soundtrack.music_volume': 'Music volume',
'soundtrack.narration_volume': 'Narration volume',
'soundtrack.generate': 'Generate soundtrack',
'soundtrack.generating': 'Generating…',
'soundtrack.clear': 'Clear',
'soundtrack.starting': '⏵ Generating soundtrack…',
'soundtrack.progress_music': '⏵ Generating background music…',
'soundtrack.progress_narration': '⏵ Generating narration…',
'soundtrack.done': '✓ Soundtrack ready — it will be mixed in on export',
'soundtrack.failed': '⚠️ Soundtrack failed: {message}',
'soundtrack.music_ready': 'Music',
'soundtrack.narration_ready': 'Narration',
'soundtrack.empty': 'Enter a music prompt and/or narration text first.',
'graph.title': 'Content graph',
'graph.download': '⬇ Download JSON',
'graph.close': '✕',
'graph.empty': '(no graph for this project)',
'graph.error': 'error loading graph: {message}',
'gallery.title': 'Pick a template',
'gallery.close': '✕',
'modal.new.title': 'New project',
'modal.new.name_label': 'Name',
'modal.new.name_placeholder': 'e.g. nexu-io launch teaser',
'modal.new.intent_label': 'Intent (optional)',
'modal.new.intent_placeholder': 'A one-line description of what this video is about',
'modal.new.cancel': 'Cancel',
'modal.new.create': 'Create',
'modal.new.name_required': 'Name is required',
'modal.new.created': 'Created "{name}"',
'modal.new.failed': 'Failed to create project',
'language.label': 'Language',
'settings.title': 'Settings',
'settings.tab.agent': 'Agent',
'settings.tab.audio': 'Audio',
'settings.tab.language': 'Language',
'settings.tab.about': 'About',
'settings.audio.title': 'Audio · MiniMax',
'settings.audio.subtitle': 'API key for soundtrack generation (background music + narration).',
'settings.audio.loading': 'Checking…',
'settings.audio.api_key': 'API key',
'settings.audio.api_key_placeholder': 'Paste your MiniMax API key',
'settings.audio.region': 'API region',
'settings.audio.region_intl': '🌐 International · api.minimax.io',
'settings.audio.region_cn': '🇨🇳 China · api.minimaxi.com',
'settings.audio.base_url': 'Base URL',
'settings.audio.save': 'Save',
'settings.audio.clear': 'Clear',
'settings.audio.configured': '✓ Configured · {key} · from {source}',
'settings.audio.not_configured': 'No MiniMax key configured yet.',
'settings.audio.source_config': 'Settings',
'settings.audio.source_env': 'environment',
'settings.audio.saving': 'Saving…',
'settings.audio.saved': '✓ Saved',
'settings.audio.save_failed': 'Save failed: {message}',
'settings.audio.need_key': 'Enter an API key first.',
'settings.audio.hint': 'Stored locally in .html-video/media-config.json. Pick the region that matches your key — an api.minimax.io (International) key will NOT work against api.minimaxi.com (China), and vice-versa. The old api.minimaxi.chat host is retired.',
'settings.agent.title': 'Agent',
'settings.agent.subtitle': 'Pick the runtime that turns your chat into HTML.',
'settings.agent.mode.local': 'Local CLI',
'settings.agent.mode.byok': 'BYOK (API)',
'settings.agent.detected': 'Detected agents ({count})',
'settings.agent.test': 'Test',
'settings.agent.testing': 'Testing…',
'settings.agent.test_ok': 'OK · {ms}ms · {bytes}B',
'settings.agent.test_fail': 'Failed: {message}',
'settings.agent.empty_reply': 'Failed: agent returned an empty reply',
'settings.agent.use': 'Use',
'settings.agent.in_use': 'In use',
'settings.agent.unavailable': 'Not installed',
'agent.sign_in': 'Sign in',
'agent.signing_in': 'Signing in…',
'agent.signed_in': '✓ Signed in to AMR',
'agent.sign_in_failed': 'Sign-in failed',
'agent.recommended': 'Recommended · one login, many models',
'settings.agent.byok.intro': 'Use your own Anthropic / OpenRouter API key. Reads from environment:',
'settings.agent.byok.env_key': 'ANTHROPIC_API_KEY or ANTHROPIC_AUTH_TOKEN',
'settings.agent.byok.env_base': 'ANTHROPIC_BASE_URL (optional, defaults to api.anthropic.com)',
'settings.agent.rescan': '↻ Rescan',
'settings.agent.rescanned': 'Rescanned',
'settings.language.title': 'Language',
'settings.language.subtitle': 'Studio interface language. Re-renders instantly.',
'settings.language.en': 'English',
'settings.language.zh': '中文',
'settings.language.en_sub': 'EN',
'settings.language.zh_sub': 'ZH-CN',
'settings.about.title': 'About',
'settings.about.subtitle': 'html-video — open-source HTML→Video meta-layer for coding agents.',
'settings.about.version': 'Version',
'settings.about.repo': 'Repo',
'settings.about.discord': 'Discord',
'settings.about.license': 'License',
'settings.about.related': 'Related',
'toolbar.settings': 'Settings',
'tpl_preview.cancel': 'Cancel',
'tpl_preview.use': 'Use this template',
'tpl_preview.replace_confirm': 'Replace current template with "{name}"? Existing preview content stays put — the agent can rebuild on next chat.',
'tpl_preview.applied': 'Template: {name}',
'tpl_preview.fps_dur': '{fps}fps · {duration}s · {aspect}',
'tpl_preview.source_skill': 'Adapted from',
'tpl_preview.source_origin': 'Design lineage',
'tpl_preview.source_license': 'License',
},
zh: {
'app.empty_pick_create': '挑一个项目或新建',
'app.empty_subtitle':
'每个项目 = 一个 HTML 视频。挑一个模板看视觉基线、跟 agent 聊驱动内容、在中间栏改逐帧文字、右边看效果。',
'app.no_project': '未选项目',
'sidebar.projects': '项目',
'sidebar.new': '+ 新建',
'sidebar.collapse': '收起侧栏',
'sidebar.empty_list': '还没有项目',
'sidebar.menu.rename': '✎ 重命名',
'sidebar.menu.delete': '🗑 删除',
'sidebar.rename_prompt': '新项目名',
'sidebar.delete_confirm': '删除 "{name}"？此操作不可撤销。',
'toolbar.template': '模板',
'toolbar.template_pick': '可选 · 挑模板',
'toolbar.agent': 'Agent',
'toolbar.model': '模型',
'toolbar.agent_none': '— 无 —',
'toolbar.agent_ready': '● 就绪',
'toolbar.agent_install': '○ 待装',
'toolbar.export_mp4': '导出 MP4',
'composer.placeholder.no_project': '先选一个项目…',
'composer.placeholder.detecting_agents': '描述视频（正在探测 agent）…',
'composer.placeholder.no_agent': '装 claude CLI 后即可聊天…',
'composer.placeholder.focus': '只修改这一帧的内容（点掉上方芯片可恢复整片）…',
'composer.placeholder.no_template': '描述视频，或粘一个文章链接 / GitHub repo 据此生成。',
'composer.placeholder.with_template': '描述视频 — 内容、名字、数据…或粘文章链接 / GitHub repo。',
'composer.hint': 'Cmd / Ctrl + Enter · 拖拽 / 粘贴文件 · 拖入 design.md / frame.md 锁定品牌+动效',
'composer.send': '发送',
'composer.attach': '附加文件',
'composer.focus_chip': '仅修改第 {order} 帧 {fid}',
'composer.focus_clear': '清除',
'chat.empty.title': '发条消息开始',
'chat.empty.body':
'告诉 agent 想做什么 — 单帧标题卡、多帧预告片、数据大字报 — 它会搭出 HTML。',
'chat.summary.form_submitted': '📋 提交了表单',
'chat.summary.confirm_generate': '✓ 确认生成',
'chat.summary.confirm_edit': '✏️ 改一下',
'chat.thinking': 'agent 思考中',
'chat.still_generating': '⏳ 这个项目仍在后台生成中 —— 完成后结果会出现在这里（点重载预览可刷新）。',
'chat.placeholder.gen_html': '📄 *正在生成 HTML…*',
'chat.placeholder.plan_graph': '🧭 *规划故事板…*',
'chat.empty_reply':
'⚠️ Agent 返回为空。试着重新表述 — 比如告诉它品牌 / 主题 / 1-2 个具体点，或者你想要什么类型的帧。',
'preview.placeholder.pick_project': '先选一个项目。',
'preview.placeholder.pick_template':
'发一条消息让 agent 生成第一版 HTML。<br>或上方挑一个模板快开。',
'preview.edit_text_on': '✓ 完成编辑',
'preview.edit_text_off': '✎ 编辑文字',
'preview.edit_text_title': '点画面里的文字直接修改',
'preview.edit_text_done_title': '完成编辑',
'preview.reload': '↻ 重载预览',
'preview.no_hv_text': '当前帧没有可编辑的文字（HTML 缺 data-hv-text 标签）。',
'frames.label': '分镜',
'frames.view_graph': '看图谱',
'frames.enhance': '⚡ Remotion',
'frames.enhance_hint': '用原生 Remotion 渲染这个数据帧（数字滚动、柱子生长）',
'frames.enhanced_revert': '⚡ Remotion ✓（还原）',
'frames.enhancing': '⚡ {pct}%…',
'enhance.done': '⚡ 该帧已用 Remotion 渲染',
'enhance.failed': '⚡ Remotion 失败：{message}',
'text_pane.title': '帧文字',
'text_pane.no_project': '无项目。',
'text_pane.empty_with_frames':
'当前帧没有可编辑文字。切到别的帧，或在画面里点 ✎ 编辑文字。',
'text_pane.empty_no_frames':
'还没有可编辑的字段。发一条消息生成第一版 HTML，逐帧字段会出现在这里。',
'text_pane.collapse': '收起面板',
'text_pane.save_state.idle': '—',
'text_pane.save_state.typing': '输入中…',
'text_pane.save_state.saving': '保存中…',
'text_pane.save_state.saved': '已保存',
'text_pane.save_state.error': '错误',
'export.starting': '⏵ 开始导出 MP4…',
'export.button_running': '⏵ {pct}% · {stage}',
'export.done_seconds': '✓ MP4 已导出 · {seconds}',
'export.done_no_seconds': '✓ MP4 已导出',
'export.failed': '⚠️ 导出失败：{message}',
'export.stream_interrupted': '导出流中断：{message}',
'export.failed_short': '导出失败：{message}',
'export.title': '🎬 MP4 已就绪',
'export.reveal': '在 Finder 中显示',
'export.copy_path': '复制路径',
'export.copied': '已复制路径',
'export.copy_failed': '复制失败：{message}',
'export.reveal_failed': '打开失败：{message}',
'soundtrack.title': '🎵 添加背景音乐和配音',
'soundtrack.summary_sub': 'AI 配乐 + 旁白，导出时自动混入',
'soundtrack.optional': '可选',
'soundtrack.hint': '背景音乐 + 旁白，导出时混入 MP4。',
'soundtrack.music_label': '背景音乐',
'soundtrack.music_placeholder': '选上面的风格，或自己描述 —— 风格、情绪、节奏',
'soundtrack.preset_energetic': '动感',
'soundtrack.preset_calm': '舒缓',
'soundtrack.preset_tech': '科技',
'soundtrack.preset_narrative': '叙事',
'soundtrack.preset_minimal': '极简',
'soundtrack.preset_epic': '史诗',
'soundtrack.narration_label': '旁白 / 配音',
'soundtrack.step_write': '第一步 · 写旁白文字',
'soundtrack.step_voice': '第二步 · 合成配音（音频）',
'soundtrack.editing_frame': '· 正在编辑第 {n}/{total} 帧',
'soundtrack.draft_frame': '✨ AI 起草本页',
'soundtrack.draft_all': '✨ AI 起草全部',
'soundtrack.gen_music': '🎵 生成配乐',
'soundtrack.gen_narration': '🎙 合成配音',
'soundtrack.empty_music': '请先选或描述一个音乐风格。',
'soundtrack.empty_narration': '请先填旁白文字（点 ✨ 可让 AI 起草）。',
'soundtrack.frame_word': '画面',
'soundtrack.total_word': '总时长',
'soundtrack.drafting': '生成中…',
'soundtrack.draft_need_frames': '请先生成视频',
'soundtrack.draft_failed': '⚠️ 生成失败：{message}',
'soundtrack.narration_placeholder': '每帧一句——点 ✨ 根据视频生成（留空则不加）',
'soundtrack.voice_label': '音色',
'soundtrack.voice_male_warm': '男声 · 温暖',
'soundtrack.voice_male_pro': '男声 · 专业',
'soundtrack.voice_male_deep': '男声 · 低沉',
'soundtrack.voice_female_anchor': '女声 · 播音',
'soundtrack.voice_female_mature': '女声 · 御姐',
'soundtrack.voice_female_sweet': '女声 · 甜美',
'soundtrack.fit_durations': '⇄ 时长适配配音',
'soundtrack.fit_hint': '按每帧旁白长短重新分配各帧时长',
'soundtrack.fitting': '适配中…',
'soundtrack.fitted': '✓ 帧时长已适配配音 · 共 {sec}s',
'soundtrack.fit_failed': '适配失败',
'soundtrack.music_volume': '音乐音量',
'soundtrack.narration_volume': '旁白音量',
'soundtrack.generate': '生成配乐',
'soundtrack.generating': '生成中…',
'soundtrack.clear': '清除',
'soundtrack.starting': '⏵ 正在生成配乐…',
'soundtrack.progress_music': '⏵ 正在生成背景音乐…',
'soundtrack.progress_narration': '⏵ 正在生成旁白…',
'soundtrack.done': '✓ 配乐就绪 —— 导出时会自动混入',
'soundtrack.failed': '⚠️ 配乐失败：{message}',
'soundtrack.music_ready': '音乐',
'soundtrack.narration_ready': '旁白',
'soundtrack.empty': '请先填写音乐提示词或旁白文字。',
'graph.title': '内容图谱',
'graph.download': '⬇ 下载 JSON',
'graph.close': '✕',
'graph.empty': '（项目没有图谱）',
'graph.error': '加载图谱失败：{message}',
'gallery.title': '挑一个模板',
'gallery.close': '✕',
'modal.new.title': '新建项目',
'modal.new.name_label': '名称',
'modal.new.name_placeholder': '例如：nexu-io 发布预告',
'modal.new.intent_label': '意图（可选）',
'modal.new.intent_placeholder': '一句话说说这个视频在讲什么',
'modal.new.cancel': '取消',
'modal.new.create': '创建',
'modal.new.name_required': '名称不能空',
'modal.new.created': '已创建 "{name}"',
'modal.new.failed': '创建项目失败',
'language.label': '语言',
'settings.title': '设置',
'settings.tab.agent': 'Agent',
'settings.tab.audio': '音频',
'settings.tab.language': '界面语言',
'settings.tab.about': '关于',
'settings.audio.title': '音频 · MiniMax',
'settings.audio.subtitle': '配乐生成（背景音乐 + 旁白）所需的 API key。',
'settings.audio.loading': '检查中…',
'settings.audio.api_key': 'API key',
'settings.audio.api_key_placeholder': '粘贴你的 MiniMax API key',
'settings.audio.region': 'API 区域',
'settings.audio.region_intl': '🌐 国际 · api.minimax.io',
'settings.audio.region_cn': '🇨🇳 国内 · api.minimaxi.com',
'settings.audio.base_url': 'Base URL',
'settings.audio.save': '保存',
'settings.audio.clear': '清除',
'settings.audio.configured': '✓ 已配置 · {key} · 来自{source}',
'settings.audio.not_configured': '尚未配置 MiniMax key。',
'settings.audio.source_config': '设置',
'settings.audio.source_env': '环境变量',
'settings.audio.saving': '保存中…',
'settings.audio.saved': '✓ 已保存',
'settings.audio.save_failed': '保存失败：{message}',
'settings.audio.need_key': '请先填写 API key。',
'settings.audio.hint': '保存在本地 .html-video/media-config.json。请按你的 key 选对区域——国际版 api.minimax.io 的 key 在国内版 api.minimaxi.com 上无法使用，反之亦然；旧的 api.minimaxi.chat 域名已停用。',
'settings.agent.title': 'Agent',
'settings.agent.subtitle': '选一个运行时把你的对话翻成 HTML。',
'settings.agent.mode.local': '本机 CLI',
'settings.agent.mode.byok': 'BYOK (API)',
'settings.agent.detected': '已检测到的 agent（{count}）',
'settings.agent.test': '测试',
'settings.agent.testing': '测试中…',
'settings.agent.test_ok': '通过 · {ms}ms · {bytes}B',
'settings.agent.test_fail': '失败：{message}',
'settings.agent.empty_reply': '失败：agent 返回为空',
'settings.agent.use': '使用',
'settings.agent.in_use': '当前',
'settings.agent.unavailable': '未安装',
'agent.sign_in': '登录',
'agent.signing_in': '登录中…',
'agent.signed_in': '✓ 已登录 AMR',
'agent.sign_in_failed': '登录失败',
'agent.recommended': '推荐 · 一次登录，多种模型',
'settings.agent.byok.intro': '直连 Anthropic / OpenRouter API。从环境变量读：',
'settings.agent.byok.env_key': 'ANTHROPIC_API_KEY 或 ANTHROPIC_AUTH_TOKEN',
'settings.agent.byok.env_base': 'ANTHROPIC_BASE_URL（可选，默认 api.anthropic.com）',
'settings.agent.rescan': '↻ 重新扫描',
'settings.agent.rescanned': '已重新扫描',
'settings.language.title': '界面语言',
'settings.language.subtitle': 'Studio 界面语言，切换立即生效。',
'settings.language.en': 'English',
'settings.language.zh': '中文',
'settings.language.en_sub': 'EN',
'settings.language.zh_sub': 'ZH-CN',
'settings.about.title': '关于',
'settings.about.subtitle': 'html-video — 开源的 HTML→视频 meta-layer，为 coding agent 设计。',
'settings.about.version': '版本',
'settings.about.repo': '代码仓库',
'settings.about.discord': 'Discord',
'settings.about.license': '许可',
'settings.about.related': '相关项目',
'toolbar.settings': '设置',
'tpl_preview.cancel': '取消',
'tpl_preview.use': '使用此模板',
'tpl_preview.replace_confirm': '把当前模板替换为 "{name}"？现有预览不会被覆盖，下一轮 chat 时 agent 会按新模板重写。',
'tpl_preview.applied': '已切换模板：{name}',
'tpl_preview.fps_dur': '{fps}fps · {duration}秒 · {aspect}',
'tpl_preview.source_skill': '改编自',
'tpl_preview.source_origin': '设计渊源',
'tpl_preview.source_license': '许可证',
},
};
const STORAGE_KEY = 'hv.studio.locale';
let _locale = resolveInitialLocale();
function resolveInitialLocale() {
try {
const stored = localStorage.getItem(STORAGE_KEY);
if (stored && AVAILABLE_LOCALES.includes(stored)) return stored;
} catch {
/* localStorage unavailable */
}
// Default is English regardless of nav.language. Joey explicitly asked.
return DEFAULT_LOCALE;
}
export function getLocale() {
return _locale;
}
export function setLocale(loc) {
if (!AVAILABLE_LOCALES.includes(loc)) return;
_locale = loc;
try { localStorage.setItem(STORAGE_KEY, loc); } catch {}
// Notify listeners (the studio app re-renders).
document.dispatchEvent(new CustomEvent('hv-locale-change', { detail: { locale: loc } }));
}
/**
* Apply i18n to static DOM elements. Markers:
* data-i18n="key" → textContent
* data-i18n-attr="placeholder:key,title:key2" → set those attrs
* data-i18n-html="key" → innerHTML (caution: only for trusted keys)
*
* Call once after DOMContentLoaded and also on every locale change.
*/
export function applyDomI18n(root) {
const r = root || document;
r.querySelectorAll('[data-i18n]').forEach((el) => {
el.textContent = t(el.dataset.i18n);
});
r.querySelectorAll('[data-i18n-html]').forEach((el) => {
el.innerHTML = t(el.dataset.i18nHtml);
});
r.querySelectorAll('[data-i18n-attr]').forEach((el) => {
const pairs = (el.dataset.i18nAttr || '').split(',').map((s) => s.trim()).filter(Boolean);
for (const pair of pairs) {
const [attr, key] = pair.split(':').map((s) => s.trim());
if (attr && key) el.setAttribute(attr, t(key));
}
});
}
document.addEventListener('hv-locale-change', () => applyDomI18n());
document.addEventListener('DOMContentLoaded', () => applyDomI18n());
/**
* Translate a key. `params` is a plain object whose keys substitute
* `{key}` placeholders in the resolved string.
*/
export function t(key, params) {
const dict = DICT[_locale] ?? DICT[DEFAULT_LOCALE];
let s = dict[key];
if (s === undefined) {
// Fall back to English, then to the key itself.
s = DICT[DEFAULT_LOCALE][key] ?? key;
}
if (params) {
for (const [k, v] of Object.entries(params)) {
s = s.replaceAll(`{${k}}`, String(v));
}
}
return s;
}
packages/project-studio/public/index.html
+1197
View File
File diff suppressed because it is too large Load Diff
packages/project-studio/public/tokens.css
+84
View File
@@ -0,0 +1,84 @@
/* Design tokens — refreshed to the Hyperframes (heygen.com/hyperframes) visual
* language: warm paper-white surfaces, mint-green accent, warm-grey hairline
* borders, tight radii, near-shadowless, mono-forward "code editor" feel.
* Only token VALUES changed — the studio layout/structure is untouched. */
:root {
/* Surfaces — paper white + warm off-white panels (hyperframes #fff / #f6f5f1 / #f0efe9) */
--bg: #f6f5f1;
--bg-panel: #ffffff;
--bg-subtle: #f0efe9;
--bg-elevated: #ffffff;
/* Text — near-black on paper (hyperframes #1a1a1a / #6b6b6b / #aaa) */
--text: #1a1a1a;
--text-muted: #6b6b6b;
--text-faint: #aaaaaa;
/* Accent — mint green (hyperframes primary CTA #3ce6ac on dark text) */
--accent: #3ce6ac;
--accent-hover: #2fd49c;
--accent-soft: #d6f7ec;
--accent-fg: #0b0b0b;
/* Border — warm grey hairline (hyperframes #e0dfdb) */
--border: #e0dfdb;
--border-strong: #d0cfc9;
/* Status — keep the hyperframes traffic-light accents available */
--green: #16a34a;
--green-bg: #e3f5e8;
--blue: #2348b8;
--blue-bg: #e6ecfb;
--red: #e5484d;
--red-bg: #fbe7e7;
--amber: #b26200;
--amber-bg: #faedc8;
/* Selected (user chat bubbles) — sits on the dark/ink end, not the mint */
--selected: #1a1a1a;
--selected-fg: #ffffff;
/* Radii — hyperframes buttons 10-12px, pills 20px */
--radius-sm: 8px;
--radius: 12px;
--radius-lg: 16px;
--radius-pill: 999px;
/* Shadows — near-shadowless paper look; keep tiny lifts only */
--shadow-xs: 0 1px 1px rgba(20, 18, 11, 0.03);
--shadow-sm: 0 1px 3px rgba(20, 18, 11, 0.05);
--shadow-md: 0 4px 16px rgba(20, 18, 11, 0.07);
--shadow-lg: 0 18px 48px rgba(20, 18, 11, 0.14);
/* Typography — Inter (≈ TT Norms Pro) + JetBrains Mono (≈ TT Norms Pro Mono) */
--font-sans: 'Inter', -apple-system, 'Segoe UI', system-ui, sans-serif;
--font-mono: 'JetBrains Mono', 'SF Mono', Menlo, Consolas, monospace;
--font-serif: 'Source Serif 4', Georgia, serif;
}
/* Dark mode — the hyperframes ink end (#0a0a0a) with the same mint accent. */
@media (prefers-color-scheme: dark) {
:root {
--bg: #0a0a0a;
--bg-panel: #141414;
--bg-subtle: #1c1c1c;
--bg-elevated: #1c1c1c;
--text: #f4f4f2;
--text-muted: #9a9a96;
--text-faint: #555552;
--accent: #3ce6ac;
--accent-hover: #54ecb9;
--accent-soft: #103a2c;
--accent-fg: #0b0b0b;
--border: #2a2a28;
--border-strong: #3a3a36;
--green-bg: #103a23;
--blue-bg: #16213f;
--red-bg: #3a1a1b;
--amber-bg: #3a2c0d;
}
}
packages/runtime/package.json
+30
View File
@@ -0,0 +1,30 @@
{
"name": "@html-video/runtime",
"version": "0.1.0",
"description": "Local coding agent runtime adapter (forked-from-OD slim variant)",
"license": "Apache-2.0",
"type": "module",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
".": {
"import": "./dist/index.js",
"types": "./dist/index.d.ts"
}
},
"files": [
"dist",
"README.md"
],
"scripts": {
"build": "tsc -p tsconfig.json",
"typecheck": "tsc -p tsconfig.json --noEmit",
"test": "npm run build && node --test --experimental-strip-types \"test/**/*.test.ts\" \"test/**/*.test.mjs\""
},
"devDependencies": {
"typescript": "^5.7.2"
},
"dependencies": {
"@powerformer/vela-cli": "^0.0.16"
}
}
packages/runtime/src/acp-client.ts
+195
View File
@@ -0,0 +1,195 @@
/**
* Minimal ACP (Agent Client Protocol) JSON-RPC client over child-process stdio.
*
* Used for AMR (the `vela agent run --runtime opencode` runtime), which speaks
* ACP rather than printing to stdout. This is a slim, single-prompt client —
* just enough to drive one turn and stream the text back; it does NOT implement
* the full OD daemon ACP surface (permissions, terminals, MCP, multi-turn).
*
* Handshake (all newline-delimited JSON-RPC 2.0 on stdin/stdout):
* → initialize { protocolVersion, clientCapabilities, clientInfo }
* → session/new { cwd, mcpServers: [] }
* → session/set_model { sessionId, modelId } (only when model !== 'default')
* → session/prompt { sessionId, prompt: [{type:'text', text}] }
* ← session/update notifications: agent_message_chunk.content.text = output
* ← result for the prompt request id = turn complete
*
* Distilled from open-design/apps/daemon/src/acp.ts (the production version).
*/
import { spawn as cpSpawn } from 'node:child_process';
import { resolve as resolvePath } from 'node:path';
import type { AgentEvent } from './types.js';
const ACP_PROTOCOL_VERSION = 1;
const DEFAULT_STAGE_TIMEOUT_MS = 120_000;
export interface RunAcpOptions {
/** Resolved absolute path to the agent binary (e.g. vela). */
bin: string;
/** Argv after the binary, e.g. ['agent','run','--runtime','opencode']. */
args: string[];
prompt: string;
cwd: string;
/** Model id for session/set_model. Omit / 'default' → skip set_model. */
model?: string;
env?: Record<string, string>;
onEvent: (e: AgentEvent) => void;
signal: AbortSignal;
clientName?: string;
clientVersion?: string;
}
type JsonRpcId = number;
export async function runAcpAgent(opts: RunAcpOptions): Promise<{ exitCode: number }> {
const cwd = resolvePath(opts.cwd || process.cwd());
const child = cpSpawn(opts.bin, opts.args, {
cwd,
env: { ...process.env, ...(opts.env ?? {}) },
stdio: ['pipe', 'pipe', 'pipe'],
});
if (!child.stdin || !child.stdout) {
opts.onEvent({ type: 'error', message: 'ACP: child has no stdio pipes' });
return { exitCode: -1 };
}
let stderrBuf = '';
child.stderr?.on('data', (c: Buffer) => { stderrBuf += c.toString('utf8'); });
return await new Promise<{ exitCode: number }>((done) => {
let settled = false;
let nextId = 1;
let sessionId: string | null = null;
let initId: JsonRpcId | null = null;
let newSessionId: JsonRpcId | null = null;
let setModelId: JsonRpcId | null = null;
let promptId: JsonRpcId | null = null;
let buf = '';
let stageTimer: ReturnType<typeof setTimeout> | null = null;
const writeRpc = (id: JsonRpcId, method: string, params: unknown) => {
child.stdin!.write(`${JSON.stringify({ jsonrpc: '2.0', id, method, params })}\n`);
};
const armTimer = (label: string) => {
if (stageTimer) clearTimeout(stageTimer);
stageTimer = setTimeout(() => finish(-1, `ACP timed out waiting for ${label}`), DEFAULT_STAGE_TIMEOUT_MS);
};
const finish = (code: number, errMsg?: string) => {
if (settled) return;
settled = true;
if (stageTimer) clearTimeout(stageTimer);
if (errMsg) {
const tail = stderrBuf.trim().slice(-400);
opts.onEvent({ type: 'error', message: tail ? `${errMsg} — ${tail}` : errMsg });
}
try { child.kill('SIGTERM'); } catch { /* already gone */ }
done({ exitCode: code });
};
opts.signal.addEventListener('abort', () => finish(-1));
child.on('error', (err) => finish(-1, `ACP spawn failed: ${err.message}`));
child.on('exit', (code) => {
// Normal completion path resolves on the prompt result; an exit before
// that is an error (often: not logged in, surfaced on stderr).
if (!settled) finish(code ?? -1, `vela exited (code ${code ?? 'null'}) before the turn completed`);
});
const handle = (msg: Record<string, unknown>) => {
// --- responses to our requests (have an id + result/error) ---
if ('id' in msg && (('result' in msg) || ('error' in msg))) {
if (msg.error) {
const e = msg.error as { message?: string; code?: number };
return finish(-1, `ACP error${e.code ? ` ${e.code}` : ''}: ${e.message ?? 'unknown'}`);
}
const id = msg.id as JsonRpcId;
if (id === initId) {
// initialized → open a session
newSessionId = nextId++;
armTimer('session/new');
return writeRpc(newSessionId, 'session/new', { cwd, mcpServers: [] });
}
if (id === newSessionId) {
const result = msg.result as { sessionId?: string } | undefined;
sessionId = result?.sessionId ?? null;
if (!sessionId) return finish(-1, 'ACP: session/new returned no sessionId');
if (opts.model && opts.model !== 'default') {
setModelId = nextId++;
armTimer('session/set_model');
return writeRpc(setModelId, 'session/set_model', { sessionId, modelId: opts.model });
}
return sendPrompt();
}
if (id === setModelId) {
return sendPrompt();
}
if (id === promptId) {
// Turn complete.
opts.onEvent({ type: 'message_end', reason: 'ok' });
return finish(0);
}
return;
}
// --- notifications (method, no response expected) ---
if (msg.method === 'session/update') {
const params = msg.params as { update?: Record<string, unknown> } | undefined;
const update = params?.update;
if (update?.sessionUpdate === 'agent_message_chunk') {
const content = update.content as { text?: string } | undefined;
if (content?.text) {
armTimer('session/prompt stream');
opts.onEvent({ type: 'text', chunk: content.text });
}
}
return;
}
// --- requests FROM the agent (permission etc.) — auto-deny/ignore ---
if (msg.method === 'session/request_permission' && 'id' in msg) {
// We run non-interactively; reject so the agent proceeds with defaults
// rather than hanging. (OD picks an allow option; for our read-only
// text turn, declining is safe and avoids file mutations.)
const params = msg.params as { options?: Array<{ optionId?: string; kind?: string }> } | undefined;
const allow = params?.options?.find((o) => /allow|accept|yes/i.test(o.kind ?? o.optionId ?? ''));
const optionId = allow?.optionId ?? params?.options?.[0]?.optionId;
if (optionId) child.stdin!.write(`${JSON.stringify({ jsonrpc: '2.0', id: msg.id, result: { outcome: { outcome: 'selected', optionId } } })}\n`);
return;
}
};
const sendPrompt = () => {
promptId = nextId++;
armTimer('session/prompt');
writeRpc(promptId, 'session/prompt', {
sessionId,
prompt: [{ type: 'text', text: opts.prompt }],
});
};
child.stdout!.on('data', (chunk: Buffer) => {
buf += chunk.toString('utf8');
let nl: number;
while ((nl = buf.indexOf('\n')) >= 0) {
const line = buf.slice(0, nl).trim();
buf = buf.slice(nl + 1);
if (!line) continue;
let msg: Record<string, unknown>;
try { msg = JSON.parse(line); } catch { continue; } // ignore non-JSON noise
try { handle(msg); } catch (e) { finish(-1, `ACP handler error: ${e instanceof Error ? e.message : e}`); }
}
});
// Kick off the handshake.
initId = nextId++;
armTimer('initialize');
writeRpc(initId, 'initialize', {
protocolVersion: ACP_PROTOCOL_VERSION,
clientCapabilities: { terminal: false },
clientInfo: { name: opts.clientName ?? 'html-video', version: opts.clientVersion ?? '0.1' },
});
});
}
packages/runtime/src/defs/aider.ts
+29
View File
@@ -0,0 +1,29 @@
import type { AgentDef } from '../types.js';
/**
* Aider def (`aider`, by Aider-AI — aider-ai/aider, Python pair-programmer).
*
* Aider does NOT read its prompt from stdin — the one-shot entry is the
* `-m`/`--message` argv flag (send one message, process, exit). So the prompt
* goes into argv (same shape as the hermes def), with promptViaStdin left off.
*
* Scripting flags:
* --yes-always auto-confirm "Add file to chat?" etc., else it can stall
* even with -m (long-standing behaviour).
* --no-pretty drop ANSI colour/markup so stdout is clean text.
* --no-stream emit the full reply at once rather than token-streaming.
* Default output is human-readable text (no JSON), so plain mode reads the
* fenced ```html``` block.
*/
export const aider: AgentDef = {
id: 'aider',
name: 'Aider',
bin: 'aider',
versionArgs: ['--version'],
buildArgs(prompt, _ctx) {
// argv-only prompt (Aider has no stdin prompt path).
return ['--yes-always', '--no-pretty', '--no-stream', '--message', prompt];
},
streamFormat: 'plain',
installUrl: 'https://aider.chat/docs/scripting.html',
};
packages/runtime/src/defs/amr.ts
+161
View File
@@ -0,0 +1,161 @@
import { execFile } from 'node:child_process';
import { homedir } from 'node:os';
import { join } from 'node:path';
import { readFileSync } from 'node:fs';
import { promisify } from 'node:util';
import type { AgentDef } from '../types.js';
const exec = promisify(execFile);
/**
* Open Design AMR (Vela) agent.
*
* AMR is the `vela` CLI's ACP stdio mode: `vela agent run --runtime opencode`
* starts a private OpenCode server and speaks ACP JSON-RPC over stdio. The user
* authenticates once in the Open Design app (`vela login`, browser OAuth) — we
* reuse that login state, never ask for an API key.
*
* Binary resolution (so every user can use AMR, not just OD installers):
* 1. PATH (`vela`)
* 2. Open Design.app bundle (reuse an existing OD install)
* 3. @powerformer/vela-cli npm package — ships per-platform `vela` binaries,
* so html-video bundles AMR support without any external install.
* ($OPEN_DESIGN_VELA_CLI_BIN overrides everything, matching OD's env knob.)
*
* Login state lives in ~/.vela/config.json (profile → runtimeKey/apiUrl/user);
* each user signs in with their OWN OD/AMR account (browser OAuth via
* `vela login`) — html-video never holds a key. We treat a profile with a
* runtimeKey + `vela whoami` success as "logged in".
*
* Stage 2 (this revision): bundled-binary distribution via vela-cli.
* The ACP JSON-RPC client (initialize → session/new → session/prompt → streamed
* session/update) lands in stage 4.
*/
const VELA_CLI_BIN_ENV = 'OPEN_DESIGN_VELA_CLI_BIN';
const VELA_BUNDLE_FALLBACKS = [
'/Applications/Open Design.app/Contents/Resources/open-design/bin/vela',
join(homedir(), 'Applications/Open Design.app/Contents/Resources/open-design/bin/vela'),
];
/** Resolve `vela` from the bundled @powerformer/vela-cli npm package (per-platform
* binaries). Returns an absolute path, or null if the package isn't installed /
* has no binary for this platform. */
async function resolveBundledVela(): Promise<string | null> {
const envOverride = process.env[VELA_CLI_BIN_ENV]?.trim();
if (envOverride) return envOverride;
try {
const mod = (await import('@powerformer/vela-cli')) as unknown as {
resolveVelaCliBin?: (opts?: { strict?: boolean }) => unknown;
};
if (typeof mod.resolveVelaCliBin !== 'function') return null;
const resolved = await Promise.resolve(mod.resolveVelaCliBin({ strict: false }));
if (typeof resolved === 'string') return resolved.trim() || null;
if (resolved && typeof resolved === 'object') {
const p = (resolved as { path?: unknown }).path;
if (typeof p === 'string') return p.trim() || null;
}
return null;
} catch {
return null;
}
}
interface VelaProfile {
runtimeKey?: string;
apiUrl?: string;
user?: { email?: string; plan?: string } | null;
}
/** Read ~/.vela/config.json and return the active profile (prod by default). */
export function readVelaProfile(): { name: string; profile: VelaProfile } | null {
try {
const raw = readFileSync(join(homedir(), '.vela', 'config.json'), 'utf8');
const cfg = JSON.parse(raw) as { profiles?: Record<string, VelaProfile> };
const profiles = cfg.profiles ?? {};
const name = process.env.VELA_PROFILE?.trim() || (profiles.prod ? 'prod' : Object.keys(profiles)[0] ?? '');
const profile = profiles[name];
return profile ? { name, profile } : null;
} catch {
return null;
}
}
export const amr: AgentDef = {
id: 'amr',
name: 'Open Design AMR',
bin: 'vela',
binFallbacks: VELA_BUNDLE_FALLBACKS,
resolveBinFallback: resolveBundledVela,
versionArgs: ['--version'],
// ACP stdio runtime: starts a private OpenCode server, talks JSON-RPC.
buildArgs: () => ['agent', 'run', '--runtime', 'opencode'],
streamFormat: 'acp-json-rpc',
// Identify html-video as the host so the vela CLI tags its command +
// model_request analytics with source=html_video (revenue attribution).
env: { AMR_CLIENT_SOURCE: 'html_video' },
// AMR home; ?source=html_video lets vela's home page_view attribute the visit
// to html-video (same host dimension as the model-spend attribution above).
installUrl: 'https://open-design.ai/amr?source=html_video',
// AMR rejects session/prompt until a model is set. Default to deepseek-v4-flash
// (the "Lower cost / Many models" official pick); overridable per-call later.
defaultModel: 'deepseek-v4-flash',
// Found on disk → confirm the user is actually logged in. AMR needs no API
// key, but it does need a live `vela login` session.
async extraDetect(resolvedBin: string) {
const prof = readVelaProfile();
if (!prof || !prof.profile.runtimeKey) {
return { available: false, hint: 'Sign in to AMR in the Open Design app first (vela login).' };
}
// whoami is the authoritative liveness check — the stored runtimeKey can
// expire even when config.json still looks populated.
try {
const { stdout } = await exec(resolvedBin, ['whoami'], { timeout: 6000 });
const out = stdout.trim();
if (/not logged in|run `?vela login`?/i.test(out)) {
return { available: false, hint: 'AMR session expired — re-run vela login in Open Design.' };
}
const email = prof.profile.user?.email;
const plan = prof.profile.user?.plan;
return { available: true, version: `AMR${email ? ` · ${email}` : ''}${plan ? ` (${plan})` : ''}` };
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
if (/not logged in/i.test(msg)) {
return { available: false, hint: 'AMR session expired — re-run vela login in Open Design.' };
}
return { available: false, hint: `vela whoami failed: ${msg.slice(0, 120)}` };
}
},
};
/** Preferred ordering — surface the cheap/fast default + flagships first. */
const AMR_MODEL_RANK: ReadonlyMap<string, number> = new Map(
['deepseek-v4-flash', 'deepseek-v4-pro', 'claude-opus-4.8', 'claude-sonnet-4.6', 'gpt-5.5', 'gemini-3.1-pro-preview']
.map((id, i) => [id, i]),
);
export interface AmrModel { id: string; label: string }
/**
* List the live AMR catalog via `vela model list`. Each line is
* `<model-id>\t<provider>`; the id is already the link-facing slug AMR accepts
* in session/set_model, so no normalization is needed. Ordered preferred-first.
*/
export async function listAmrModels(resolvedBin: string): Promise<AmrModel[]> {
const { stdout } = await exec(resolvedBin, ['model', 'list'], { timeout: 10_000, maxBuffer: 1024 * 1024 });
const seen = new Set<string>();
const models: AmrModel[] = [];
for (const line of String(stdout).split('\n')) {
const id = line.split('\t')[0]?.trim();
if (!id || id.startsWith('#') || seen.has(id)) continue;
seen.add(id);
models.push({ id, label: id });
}
return models.sort((a, b) => {
const ra = AMR_MODEL_RANK.get(a.id) ?? Number.MAX_SAFE_INTEGER;
const rb = AMR_MODEL_RANK.get(b.id) ?? Number.MAX_SAFE_INTEGER;
return ra - rb || a.id.localeCompare(b.id);
});
}
packages/runtime/src/defs/anthropic-api.ts
+151
View File
@@ -0,0 +1,151 @@
/**
* Anthropic Messages API (HTTP) agent.
*
* Bypasses the `claude --print` CLI entirely — that interface has a
* non-deterministic "silently return 1 byte" failure mode on long creative
* outputs (verified by hand on Joey's machine). Talking to the Messages API
* directly is reliable and stream-friendly.
*
* Auth resolution (first match wins):
* 1. ANTHROPIC_API_KEY (canonical)
* 2. ANTHROPIC_AUTH_TOKEN (Joey's OpenRouter routing setup)
*
* Base URL:
* ANTHROPIC_BASE_URL or default https://api.anthropic.com
* When OpenRouter is in use, ANTHROPIC_BASE_URL is set to
* https://openrouter.ai/api — the OpenRouter shim accepts the standard
* Anthropic Messages payload + bare model names (claude-sonnet-4-6 etc).
*
* Model: claude-sonnet-4-6 by default. Sonnet 4.6 strikes the best balance
* of speed / creativity / instruction-following for our HTML output;
* upgradable per-call later if needed.
*/
import type { AgentDef, AgentEvent } from '../types.js';
const DEFAULT_MODEL = 'claude-sonnet-4-6';
const DEFAULT_BASE = 'https://api.anthropic.com';
const HV_MODEL_ENV = 'HV_AGENT_MODEL';
function resolveAuth(): { token: string; baseUrl: string; model: string } | null {
const token = process.env.ANTHROPIC_API_KEY || process.env.ANTHROPIC_AUTH_TOKEN || '';
if (!token) return null;
const baseUrl = (process.env.ANTHROPIC_BASE_URL || DEFAULT_BASE).replace(/\/+$/, '');
const model = process.env[HV_MODEL_ENV] || DEFAULT_MODEL;
return { token, baseUrl, model };
}
export const anthropicApi: AgentDef = {
id: 'anthropic-api',
name: 'Anthropic API (direct)',
bin: 'anthropic-api',
versionArgs: [],
buildArgs: () => [],
streamFormat: 'plain',
kind: 'http',
installUrl: 'https://docs.claude.com/en/api/getting-started',
async httpProbe() {
const auth = resolveAuth();
if (!auth) {
return {
available: false,
hint: 'Set ANTHROPIC_API_KEY (or ANTHROPIC_AUTH_TOKEN). For OpenRouter routing also set ANTHROPIC_BASE_URL=https://openrouter.ai/api.',
};
}
return {
available: true,
version: `${auth.model} via ${new URL(auth.baseUrl).host}`,
};
},
async httpHandler(prompt, _ctx, onEvent, signal) {
const auth = resolveAuth();
if (!auth) {
onEvent({ type: 'error', message: 'No ANTHROPIC_API_KEY / ANTHROPIC_AUTH_TOKEN in env' });
return { exitCode: -1 };
}
// OpenRouter expects bearer tokens via Authorization. The Anthropic
// direct API uses x-api-key. We try x-api-key first (works on both)
// and let the host respond. OpenRouter accepts x-api-key too.
const headers: Record<string, string> = {
'content-type': 'application/json',
'anthropic-version': '2023-06-01',
'x-api-key': auth.token,
authorization: `Bearer ${auth.token}`,
};
const url = `${auth.baseUrl}/v1/messages`;
let res: Response;
try {
res = await fetch(url, {
method: 'POST',
headers,
signal,
body: JSON.stringify({
model: auth.model,
max_tokens: 16000,
stream: true,
messages: [{ role: 'user', content: prompt }],
}),
});
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
onEvent({ type: 'error', message: `fetch failed: ${msg}` });
return { exitCode: -1 };
}
if (!res.ok || !res.body) {
const body = await res.text().catch(() => '');
onEvent({
type: 'error',
message: `${res.status} ${res.statusText}${body ? `: ${body.slice(0, 400)}` : ''}`,
});
return { exitCode: -1 };
}
// Anthropic SSE format: event: <type>\ndata: <json>\n\n
// Events we care about:
// content_block_delta { delta: { type: 'text_delta', text } } → emit text
// message_stop → done
// error → emit error
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buf = '';
try {
while (true) {
const { done, value } = await reader.read();
if (done) break;
buf += decoder.decode(value, { stream: true });
const events = buf.split('\n\n');
buf = events.pop() ?? '';
for (const ev of events) {
let dataLine = '';
for (const line of ev.split('\n')) {
if (line.startsWith('data:')) dataLine = line.slice(5).trim();
}
if (!dataLine || dataLine === '[DONE]') continue;
try {
const parsed = JSON.parse(dataLine) as { type?: string; delta?: { type?: string; text?: string }; error?: { message?: string } };
if (parsed.type === 'content_block_delta' && parsed.delta?.type === 'text_delta' && parsed.delta.text) {
onEvent({ type: 'text', chunk: parsed.delta.text });
} else if (parsed.type === 'error' && parsed.error?.message) {
onEvent({ type: 'error', message: parsed.error.message });
}
// ignore message_start / content_block_start / message_delta / ping etc
} catch {
/* malformed line — skip */
}
}
}
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
// AbortError is expected when the user cancels; don't log it as a hard error
if (msg !== 'BodyStreamBuffer was aborted' && !msg.includes('aborted')) {
onEvent({ type: 'error', message: `stream read failed: ${msg}` });
}
return { exitCode: -1 };
}
return { exitCode: 0 };
},
};
packages/runtime/src/defs/claude.ts
+23
View File
@@ -0,0 +1,23 @@
import type { AgentDef } from '../types.js';
/**
* Claude Code CLI def (`claude`, by Anthropic).
* Slim version: prompt via stdin, plain text streamFormat.
* v0.2 will switch to stream-json when we wire interactive tool_result.
*/
export const claude: AgentDef = {
id: 'claude',
name: 'Claude Code',
bin: 'claude',
versionArgs: ['--version'],
buildArgs(_prompt, _ctx) {
// --print: non-interactive, write entire response to stdout then exit.
// No permission-mode: chat-to-HTML never edits files; the model only emits
// text + a fenced ```html``` block, which our extractor reads. Plan mode
// (which we tried in v0.3) suppressed actual content.
return ['--print'];
},
streamFormat: 'plain',
promptViaStdin: true,
installUrl: 'https://docs.claude.com/en/docs/claude-code/setup',
};
packages/runtime/src/defs/codex.ts
+25
View File
@@ -0,0 +1,25 @@
import type { AgentDef } from '../types.js';
/**
* Codex CLI def (`codex`, by OpenAI).
*
* Slim version matching html-video's text-first model: the studio reads the
* model's free-form output and extracts the fenced ```html``` block, so we run
* `codex exec` in plain (non-JSON) mode rather than `--json` — the latter emits
* NDJSON envelopes that the v0.1 spawn loop would dump verbatim into the chat.
*
* `--skip-git-repo-check` lets it run in the project dir without a git repo;
* prompt is piped via stdin (long HTML-generation prompts).
*/
export const codex: AgentDef = {
id: 'codex',
name: 'Codex CLI',
bin: 'codex',
versionArgs: ['--version'],
buildArgs(_prompt, _ctx) {
return ['exec', '--skip-git-repo-check'];
},
streamFormat: 'plain',
promptViaStdin: true,
installUrl: 'https://developers.openai.com/codex/cli',
};
packages/runtime/src/defs/copilot.ts
+29
View File
@@ -0,0 +1,29 @@
import type { AgentDef } from '../types.js';
/**
* GitHub Copilot CLI def (`copilot`, by GitHub).
*
* This is the standalone agentic `copilot` binary (npm `@github/copilot`,
* GA 2026-02), NOT the deprecated `gh copilot` gh-cli extension.
*
* Headless: `-p`/`--prompt` runs one prompt and exits; stdin is also accepted.
* We pipe the prompt via stdin and pass `-p` with no value to force prompt
* mode, plus `--allow-all-tools` so the agent doesn't stall waiting for a
* tool-approval prompt during a non-interactive run. Default stdout carries
* some session metadata; the studio extractor still finds the fenced
* ```html``` block, so plain mode is fine. (If metadata ever pollutes output
* we can add `-s`/silent — left off until confirmed against an installed bin.)
*/
export const copilot: AgentDef = {
id: 'copilot',
name: 'GitHub Copilot CLI',
bin: 'copilot',
versionArgs: ['--version'],
buildArgs(_prompt, _ctx) {
// Prompt via stdin; --allow-all-tools avoids interactive approval stalls.
return ['--allow-all-tools'];
},
streamFormat: 'plain',
promptViaStdin: true,
installUrl: 'https://github.com/github/copilot-cli',
};
packages/runtime/src/defs/cursor-agent.ts
+14
View File
@@ -0,0 +1,14 @@
import type { AgentDef } from '../types.js';
export const cursorAgent: AgentDef = {
id: 'cursor-agent',
name: 'Cursor Agent',
bin: 'cursor-agent',
versionArgs: ['--version'],
buildArgs(_prompt, _ctx) {
return ['--print'];
},
streamFormat: 'plain',
promptViaStdin: true,
installUrl: 'https://cursor.com/cli',
};
packages/runtime/src/defs/gemini.ts
+29
View File
@@ -0,0 +1,29 @@
import type { AgentDef } from '../types.js';
/**
* Gemini CLI def (`gemini`, by Google — google-gemini/gemini-cli).
*
* Headless usage: piping a prompt on stdin to a non-TTY `gemini` invocation
* drops it into one-shot mode (prints the answer, exits) — verified locally
* 2026-06-04 (a stdin-only run went straight to the auth check rather than
* opening the interactive TUI). Default stdout is free-form text
* (`--output-format text`), which our extractor reads to pull the fenced
* ```html``` block, so we stay in plain mode.
*
* No model / API key is set here: if the user hasn't configured GEMINI_API_KEY
* (or another auth method) the CLI prints a clear auth error to stdout, which
* surfaces in the studio chat as-is.
*/
export const gemini: AgentDef = {
id: 'gemini',
name: 'Gemini CLI',
bin: 'gemini',
versionArgs: ['--version'],
buildArgs(_prompt, _ctx) {
// Prompt arrives via stdin; no args needed for headless one-shot.
return [];
},
streamFormat: 'plain',
promptViaStdin: true,
installUrl: 'https://github.com/google-gemini/gemini-cli',
};
packages/runtime/src/defs/grok.ts
+33
View File
@@ -0,0 +1,33 @@
import type { AgentDef } from '../types.js';
/**
* Grok Build def (`grok`, by xAI — x.ai/cli; OD labels it "Grok Build (xAI)").
*
* This is xAI's OFFICIAL CLI, not the community `superagent-ai/grok-cli`.
* Confirmed locally against grok 0.1.212 (`grok --help`, 2026-06-04):
* -p, --single <PROMPT> "Single-turn prompt. Prints the response to
* stdout and exits" ← exactly our headless path
* --output-format <fmt> [default: plain] (plain | json | streaming-json)
* -v, --version
* Prompt is an argv value of `-p` (no stdin pipe option), so it goes into
* buildArgs and promptViaStdin stays off — same shape as hermes/aider.
*
* Default output is plain text, which the studio extractor reads for the
* fenced ```html``` block.
*
* Note: Grok Build is early beta and gated behind SuperGrok / X Premium+ —
* detection finds the bin, but an un-signed-in user gets an OAuth prompt on
* first run (surfaced in chat).
*/
export const grok: AgentDef = {
id: 'grok',
name: 'Grok Build (xAI)',
bin: 'grok',
versionArgs: ['--version'],
buildArgs(prompt, _ctx) {
// -p/--single: single-turn headless; prompt is the argv value.
return ['-p', prompt];
},
streamFormat: 'plain',
installUrl: 'https://x.ai/cli',
};
packages/runtime/src/defs/hermes.ts
+29
View File
@@ -0,0 +1,29 @@
/**
* Hermes (Python CLI by Hermes Agent project) as a local agent runtime.
*
* hermes chat -q "<prompt>" -Q
*
* `-Q` (quiet) prints just the response text to stdout, prefixed with one
* `session_id: ...` header line. We strip that line in the studio prompt
* code path so the user doesn't see it.
*
* Hermes default backend is OpenRouter — same provider Joey already uses,
* so any model-side outage that hits Anthropic also hits hermes via the
* same route. Useful as a parallel comparison agent rather than a
* fundamentally redundant one.
*/
import type { AgentDef } from '../types.js';
export const hermes: AgentDef = {
id: 'hermes',
name: 'Hermes (local CLI)',
bin: 'hermes',
versionArgs: ['version'],
buildArgs(prompt) {
// hermes -q reads its query from argv (not stdin). argv length on
// macOS is ~256KB which fits any reasonable prompt.
return ['chat', '-Q', '-q', prompt];
},
streamFormat: 'plain',
installUrl: 'https://hermes.agentinfo.dev',
};
packages/runtime/src/defs/opencode.ts
+32
View File
@@ -0,0 +1,32 @@
import type { AgentDef } from '../types.js';
/**
* OpenCode def (`opencode`, by SST — sst/opencode, npm `opencode-ai`).
*
* Unlike the `--print`-style CLIs, OpenCode's non-interactive entry is the
* `run` SUBCOMMAND, not a flag. `opencode run` takes the message as a
* positional arg or on stdin (non-TTY stdin is prepended to the prompt), runs
* the session, and exits when idle. We pipe the prompt via stdin.
*
* Output: default stdout is an event-stream of text (not JSON unless you pass
* `--format json`), so plain mode captures it and the extractor reads the
* fenced ```html``` block.
*
* Note: in non-interactive mode OpenCode tightens permissions by default
* (question/plan: deny). Our chat-to-HTML flow only needs the model to emit
* text + an html block — no file edits — so the default is fine. If a future
* mode needs the agent to write files, add `--dangerously-skip-permissions`.
*/
export const opencode: AgentDef = {
id: 'opencode',
name: 'OpenCode',
bin: 'opencode',
versionArgs: ['--version'],
buildArgs(_prompt, _ctx) {
// `run` is the headless subcommand; prompt arrives via stdin.
return ['run'];
},
streamFormat: 'plain',
promptViaStdin: true,
installUrl: 'https://opencode.ai/docs',
};
packages/runtime/src/defs/qoder.ts
+40
View File
@@ -0,0 +1,40 @@
import type { AgentDef } from '../types.js';
/**
* Qoder CLI def (`qodercli`, by Qoder — npm `@qoder-ai/qodercli`).
*
* Qoder ships two binaries: `qoder` (the IDE / Electron wrapper) and
* `qodercli` (the standalone coding agent CLI, installed via npm).
* html-video uses the latter — it's the headless-capable one.
*
* Headless contract (same shape as Claude Code / Codex / Qwen):
* echo "<prompt>" | qodercli -p - --permission-mode bypass_permissions
*
* -p - → non-interactive, read prompt from stdin, print to stdout.
* --permission-mode bypass_permissions → auto-approve all tool calls
* (file writes, shell, etc.) so the session never blocks on an
* interactive permission prompt the studio cannot answer.
*
* Prompt is passed via stdin (promptViaStdin: true) — avoids Windows shell
* escaping issues with backticks, quotes, and CJK chars in long prompts.
* Default stdout is plain text, so the extractor reads the fenced ```html``` block.
*
* Version probe: `qodercli -v` → e.g. "1.0.14".
*/
export const qoderCli: AgentDef = {
id: 'qoder-cli',
name: 'Qoder CLI',
bin: 'qodercli',
versionArgs: ['-v'],
buildArgs(_prompt, _ctx) {
// -p -: read prompt from stdin (avoids Windows shell escaping issues
// with backticks, quotes, and other special chars in long prompts).
// --permission-mode bypass_permissions: auto-approve tool calls.
// --dangerously-skip-permissions: belt-and-suspenders — ensures
// the sandbox layer also lifts write/edit restrictions.
return ['-p', '-', '--permission-mode', 'bypass_permissions', '--dangerously-skip-permissions'];
},
streamFormat: 'plain',
promptViaStdin: true,
installUrl: 'https://www.npmjs.com/package/@qoder-ai/qodercli',
};

Some files were not shown because too many files have changed in this diff Show More