chore: import upstream snapshot with attribution
regression / regression-shards (style-16-prod style-9-prod style-17-prod iframe-render-compat variables-prod mp4-h265-sdr, shard-4) (push) Has been cancelled
regression / regression-shards (style-4-prod style-11-prod style-2-prod animejs-adapter typegpu-adapter parallel-capture-regression, shard-5) (push) Has been cancelled
regression / regression-shards (style-7-prod style-8-prod style-10-prod css-spinner-render-compat webm-transparency mp4-h264-sdr webm-vp9, shard-3) (push) Has been cancelled
regression / regression-shards (sub-composition-video style-18-prod raf-ball-render-compat font-variant-numeric sub-comp-t0 sub-comp-id-selector, shard-7) (push) Has been cancelled
Windows render verification / Detect changes (push) Has been cancelled
Windows render verification / Preflight (lint + format) (push) Has been cancelled
Windows render verification / Render on windows-latest (push) Has been cancelled
Windows render verification / Tests on windows-latest (push) Has been cancelled
CI / Detect changes (push) Has been cancelled
CI / Build (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Fallow audit (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Typecheck (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Producer: integration tests (push) Has been cancelled
CI / Producer: unit tests (push) Has been cancelled
CI / File size check (push) Has been cancelled
CI / Test: skills (push) Has been cancelled
CI / Skills: manifest in sync (push) Has been cancelled
CI / CLI: npx shim (macos-latest) (push) Has been cancelled
CI / CLI: npx shim (ubuntu-latest) (push) Has been cancelled
CI / CLI: npx shim (windows-latest) (push) Has been cancelled
CI / SDK: unit + contract + smoke (push) Has been cancelled
CI / Test: runtime contract (push) Has been cancelled
CI / Studio: load smoke (push) Has been cancelled
CI / Smoke: global install (push) Has been cancelled
CI / CLI smoke (required) (push) Has been cancelled
CI / Semantic PR title (push) Has been cancelled
Player perf / Detect changes (push) Has been cancelled
Player perf / Preflight (lint + format) (push) Has been cancelled
Player perf / player-perf (push) Has been cancelled
Player perf / Perf: drift (push) Has been cancelled
Player perf / Perf: fps (push) Has been cancelled
Player perf / Perf: parity (push) Has been cancelled
Player perf / Perf: scrub (push) Has been cancelled
Player perf / Perf: load (push) Has been cancelled
preview-regression / Detect changes (push) Has been cancelled
preview-regression / Preflight (lint + format) (push) Has been cancelled
preview-regression / Preview parity (push) Has been cancelled
preview-regression / preview-regression (push) Has been cancelled
regression / regression (push) Has been cancelled
regression / Detect changes (push) Has been cancelled
regression / Preflight (lint + format) (push) Has been cancelled
regression / regression-shards (hdr-regression style-5-prod style-3-prod mov-prores, shard-1) (push) Has been cancelled
regression / regression-shards (overlay-montage-prod style-12-prod chat missing-host-comp-id png-sequence portrait-edge-bleed, shard-6) (push) Has been cancelled
regression / regression-shards (style-13-prod style-6-prod vignelli-stacking gsap-letters-render-compat audio-mux-parity, shard-8) (push) Has been cancelled
regression / regression-shards (style-15-prod hdr-hlg-regression style-1-prod many-cuts vfr-screen-recording render-symlinked-assets, shard-2) (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Docs / Validate docs (push) Has been cancelled
Sync skills to ClawHub / Publish changed skills (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:58:35 +08:00
commit 85453da49f
4031 changed files with 710987 additions and 0 deletions
@@ -0,0 +1,142 @@
# Design Panel QA Matrix
Campaign artifact for `docs/plans/2026-07-02-001-fix-studio-design-panel-inputs-plan.md`.
Environment: published CLI `hyperframes@0.7.26`, embedded mode (`npx hyperframes preview`
in a scaffolded `warm-grain` project outside the repo), Chrome via agent-browser.
## Step 0: demo-failure reproduction (baseline, pre-fix)
Reproduced. The demo symptom ("font size does nothing") is real, deterministic, and its
root cause is the **selection layer**, not the persist pipeline.
### S0.1 Master view: click on visible text selects the invisible top overlay
- Action: click the "Hyperframes" H1 (from `compositions/intro.html`, embedded in `index.html`).
- Selected instead: `.grain-texture` (`hf-0qtj`, label "Grain Texture"), the full-canvas grain
overlay on `data-track-index="100"`, even though its parent `#grain-overlay-comp` has
`pointer-events: none`.
- Panel then shows generic values (Size 16px) and a Text section with an empty Content field
for a div that contains no text.
- Committing Size 72px:
- signal a (disk): `index.html` changed, `<div style="font-size: 72px" ... class="grain-texture">`
- signal b (HTTP): `POST /file-mutations/patch-element/index.html` 200, `matched:true, changed:true`
- signal c (console/telemetry): nothing
- Visible effect: none (the styled element is an invisible overlay). This alone explains the
demo: every click lands on the overlay, every edit applies to it.
- Bucket: **selection/hit-testing** (new bucket; persist pipeline healthy in this leg).
### S0.2 Sub-composition view: hover finds the element, click cannot select it
- Setup: open `intro` in the sidebar (Master > intro breadcrumb), scrub to t=2s where the
title card is visible.
- Hover over the H1: teal highlight appears (hit-testing sees the element).
- Click (real mouse down/up at the text): no selection API call, no panel, no console error.
Reproduced with element-ref clicks and coordinate clicks.
- Bucket: **selection click-to-commit in sub-composition view**.
### S0.3 Stale selection target carried across composition switch
- After selecting `.grain-texture` in Master and switching the canvas to `compositions/intro.html`,
a subsequent click re-emitted the old target: `probe-element/compositions%2Fintro.html` with
`{hfId: hf-0qtj, selector: .grain-texture}` (an element that does not exist in intro.html),
followed by a selection PUT labeled "Grain Texture" with `sourceFile: compositions/intro.html`,
then `selection: null`.
- If a patch had been committed in that state it would have written to the wrong file or
silently no-oped (`matched:false`).
- Bucket: **selection state lifecycle across composition switches**.
### S0.4 Double-click on canvas element clears selection
- Double-click on the H1 in Master view: `PUT /selection {selection: null}`. No drill-down into
the sub-composition, no text editing mode. Users double-click text instinctively.
- Bucket: **selection UX** (candidate: intentional-but-hostile; confirm with maintainers).
### Working in this leg
- Persist pipeline end-to-end (patch-element → linkedom mutation → disk write → 200 with
`matched/changed`): healthy for the (wrong) selected element.
- Hover highlighting in both views.
- Panel rendering, section expansion, input commit on Enter.
### Instrumentation notes for the full matrix
- Fetch shim on `window.fetch` in the top document captures patch/probe/selection traffic
(studio app runs in the top document; composition renders in a shadow-DOM iframe).
- Element-ref clicks work for selection in Master view; sub-composition view needs
coordinate clicks (`mouse move/down/up`) and still fails to select (S0.2).
- Seek-slider `fill` does not move the playhead; use Play/pause or timeline clicks to scrub.
- GSAP warning noise in console: `GSAP target #a-roll not found` (from the warm-grain
captions comp; unrelated).
## Selection-layer fixes: embedded-mode re-verification (post-fix)
Environment: locally built CLI (commit with selection fixes), embedded mode, fixture copied
to a scratch dir outside the repo.
- Click on empty canvas over the invisible full-canvas overlay: selection resolves to
**null** (previously: selected the overlay). S0.1 fixed for real pointer input.
- Click on visible fixture text (`#qa-headline`): selects the H1 itself; panel shows real
values (Size 48px, weight 700, content "Static Headline") instead of overlay defaults.
- Font size commit 96px: `patch-element` 200 `matched:true, changed:true`; disk gains
`<h1 style="font-size: 96px" ...>`; preview renders 96px. The demo scenario works.
- Note for future automation: `agent-browser click @ref` on an element whose DOM box is
off-viewport can land on the sidebar "Select off-canvas element" helper buttons and
select programmatically, bypassing hit-testing. Use coordinate clicks on visible pixels
for selection tests.
## Full matrix (post selection + U3 fixes, embedded mode, locally built CLI)
Instrument: scripted agent-browser runner (`matrix-runner.mjs`, session scratchpad) + interactive
follow-ups. Signals per cell: patch/gsap-mutation HTTP response, disk content, computed style,
reload survival.
### Selection (click on canvas, Inspector enabled)
| Archetype | Result |
| ----------------------- | ------------------------------------------------------------------------------------------------- |
| Static text (h1) | selects the element itself |
| Multi-span child (span) | selects the span itself |
| GSAP-tweened box | selects the element |
| Keyframed box | selects the element |
| Image | selects the element (canEditStyles true) |
| Shape div | selects the element |
| Video | selects the element (visible only inside its clip window; hidden outside, correctly unselectable) |
| Runtime caption word | falls back to the parent host (runtime nodes cannot persist; by design) |
| Sub-composition child | selects the child with sourceFile pointing at the sub-composition file |
### Inputs (all persist to disk with matched:true/changed:true and survive reload)
- Text on h1: size, content, weight, line-height, letter-spacing, align, case, style. Span-self
size also works.
- Layout on shape: W, H, rotation persist as `tl.set(...)` in the GSAP script (designed manual-edit
path); z-index persists inline.
- 3D: rotationX persists via the `gsap-mutations` endpoint (ok:true, changed:true).
- GSAP-tweened element: Layout X persists as `gsap.set("#qa-tween-box", { x: 40 })` appended to the
script. Works; note: a load-time `gsap.set` on an element that also has an x tween is semantically
debatable (starting value shifts) — flag for maintainers, not a broken input.
- Timing: start persists as `data-start="0.20"` (normalized to 2 decimals).
- Video section (titled "Video", not "Media"): volume slider persists `data-volume="0.8"`;
object-fit select persists `object-fit: cover`.
- Transparency: opacity range persists `opacity: 0.8`; blend select persists
`mix-blend-mode: multiply`.
- Radius text input persists `border-radius: 24px`; Effects blur range persists `blur(4px)`;
Clip overflow select persists `overflow: hidden`.
- Sub-composition child: Text size persists to `compositions/qa-sub.html` (`font-size: 48px`).
### Confirmed bugs
- **U4 child text-field escaping** (persist-level, confirmed by the headless harness test
"documents U4 bug: child text-field style persists as escaped markup"): editing a child field of
a multi-field element serializes markup into a `text-content` op that the server escapes.
### Notes and paper cuts (not input bugs)
- Inspector defaults OFF on a fresh embedded-mode load; canvas clicks silently do nothing until it
is toggled on. Zero feedback for the user in that state.
- Fill color picker: opens with a hex input reflecting the current color; persist path verified
green by the headless harness (fill style op); scripted popup commit was flaky (focus-sensitive
popup), verified manually instead.
- Color grading section appears for img/video elements.
- Automation notes: media/timing cells must run with the playhead inside the clip window (a
data-start edit hides the element at t=0, which is correct but confuses naive re-runs); commit
fires on Enter/blur only when the draft differs from the last value.
+381
View File
@@ -0,0 +1,381 @@
#!/usr/bin/env node
// Design-panel e2e smoke: drives agent-browser against a running preview of the
// design-panel-qa fixture and asserts selection + one representative input per
// panel section persists to disk.
//
// Usage:
// 1. Copy fixtures/design-panel-qa to a scratch dir OUTSIDE the repo.
// 2. Start the CLI there: node <repo>/packages/cli/dist/cli.js preview --no-open
// 3. STUDIO_URL=http://localhost:3002 PROJECT_DIR=<scratch dir> node design-panel.mjs
//
// Requires the agent-browser CLI on PATH. Exits non-zero on any failed cell.
// Automation notes (learned the hard way, see design-panel-qa-matrix.md):
// - Inspector defaults OFF on fresh load; the script toggles it on.
// - Selection commits on real CDP mouse events at canvas coordinates; element-ref
// clicks on off-viewport nodes can hit sidebar helper buttons instead.
// - Commit fires on Enter/blur only when the draft differs from the last value.
// - Range inputs need input+change+pointerup; selects need change.
// - Panel sections are found by the app's own `data-panel-section` attribute, not
// by matching h3 display text, and specific fields are found by their sibling
// label span (or, where there's no label, by being the section's only input of
// that type) — not by guessing the fixture's current value. Both survive wording
// or fixture-default changes that would otherwise break this script silently.
import { execFileSync } from "node:child_process";
import { readFileSync } from "node:fs";
const STUDIO_URL = process.env.STUDIO_URL || "http://localhost:3002";
const PROJECT_DIR = process.env.PROJECT_DIR;
if (!PROJECT_DIR) {
console.error("PROJECT_DIR env var is required (scratch copy of the design-panel-qa fixture)");
process.exit(2);
}
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
let failures = 0;
function ab(...args) {
try {
return execFileSync("agent-browser", args, { encoding: "utf8", timeout: 30000 });
} catch (e) {
return "ERR: " + (e.stdout || "") + (e.stderr || e.message);
}
}
function abEval(code) {
const out = ab("eval", code).trim();
let v;
try {
v = JSON.parse(out);
} catch {
return out;
}
if (typeof v === "string") {
try {
return JSON.parse(v);
} catch {
return v;
}
}
return v;
}
function check(id, ok, detail) {
console.log(`${ok ? "PASS" : "FAIL"} ${id}${detail ? " " + detail : ""}`);
if (!ok) {
failures += 1;
console.log(` patchLog: ${JSON.stringify(abEval("window.__patchLog"))}`);
}
}
function disk(file, needle) {
try {
return readFileSync(`${PROJECT_DIR}/${file}`, "utf8").includes(needle);
} catch {
return false;
}
}
// The patch fetch resolving client-side doesn't guarantee the server's file
// write has landed yet (seen in practice on the very first commit of a run).
// Poll disk instead of asserting immediately after the in-browser signal.
async function waitForDisk(file, needle, timeout = 3000) {
const start = Date.now();
while (Date.now() - start < timeout) {
if (disk(file, needle)) return true;
await sleep(100);
}
return false;
}
// Polls a page-context boolean expression instead of sleeping a fixed duration:
// faster on a healthy run, and it fails loudly (returns false) rather than
// silently passing on a slow one.
async function waitFor(expr, { timeout = 8000, interval = 150 } = {}) {
const start = Date.now();
while (Date.now() - start < timeout) {
if (abEval(expr) === true) return true;
await sleep(interval);
}
return false;
}
const HELPERS = String.raw`
(() => {
window.__patchLog = window.__patchLog || [];
window.__qaFault = window.__qaFault || null;
if (!window.__qaShim) {
window.__qaShim = true;
const orig = window.fetch;
window.fetch = async function(...args) {
const url = typeof args[0] === 'string' ? args[0] : (args[0] && args[0].url) || '';
if (window.__qaFault && url.includes(window.__qaFault.match)) {
const fault = window.__qaFault;
window.__qaFault = null; // one-shot
window.__patchLog.push({ t: Date.now(), url, status: fault.status, req: null, resp: '(fault injected)' });
return new Response(JSON.stringify({ error: 'e2e injected fault' }), {
status: fault.status,
headers: { 'content-type': 'application/json' },
});
}
const isMut = url.includes('file-mutations') || url.includes('gsap-mutations') || (args[1] && args[1].method && args[1].method !== 'GET' && url.includes('/api/'));
let body = null;
if (isMut && args[1] && typeof args[1].body === 'string') body = args[1].body.slice(0, 1500);
const res = await orig.apply(this, args);
if (isMut) {
const clone = res.clone();
let respText = '';
try { respText = (await clone.text()).slice(0, 300); } catch {}
window.__patchLog.push({ t: Date.now(), url, status: res.status, req: body, resp: respText });
}
return res;
};
}
const qa = {};
qa.frame = () => {
const frames = [];
const collect = (root) => {
for (const f of root.querySelectorAll('iframe')) { try { if (f.contentDocument && f.contentDocument.querySelector('#design-panel-qa')) frames.push(f); } catch {} }
for (const el of root.querySelectorAll('*')) { if (el.shadowRoot) collect(el.shadowRoot); }
};
collect(document);
return frames.sort((a, b) => b.getBoundingClientRect().width - a.getBoundingClientRect().width)[0] || null;
};
qa.coords = (sel) => {
const f = qa.frame();
if (!f) return null;
const el = f.contentDocument.querySelector(sel);
if (!el) return null;
const r = el.getBoundingClientRect();
const fr = f.getBoundingClientRect();
const scale = fr.width / 1920;
const x = fr.x + (r.x + Math.min(r.width, 80) / 2) * scale;
const y = fr.y + (r.y + Math.min(r.height, 60) / 2) * scale;
if (x < fr.x || x > fr.x + fr.width || y < fr.y || y > fr.y + fr.height) return null;
return [Math.round(x), Math.round(y)];
};
qa.lastSel = () => {
const s = window.__patchLog.filter(p => p.url.includes('/selection')).slice(-1)[0];
if (!s) return null;
const req = s.req || '';
if (req.includes('"selection":null')) return { nullSel: true };
const m = (k) => { const r = new RegExp('"' + k + '":"([^"]*)"').exec(req); return r ? r[1] : null; };
return { label: m('label'), selector: m('selector'), hfId: m('hfId'), src: m('sourceFile') };
};
qa.clear = () => { window.__patchLog.length = 0; return 'cleared'; };
qa.section = (slug) => document.querySelector('[data-panel-section="' + slug + '"]');
qa.sectionInputs = (slug) => {
const section = qa.section(slug);
if (!section) return null;
return [...section.querySelectorAll('input, textarea, select')];
};
qa.ensureSection = (slug) => {
const section = qa.section(slug);
if (!section) return 'no section: ' + slug;
const inputs = qa.sectionInputs(slug);
if (inputs && inputs.length) return 'open';
const header = section.querySelector('button');
if (header) header.click();
return 'clicked';
};
// Walks up a few ancestor levels looking for a preceding <span> label — covers
// both a field whose label is a direct sibling of its input (MetricField) and
// one whose label sits beside the input's wrapper (a hand-rolled SliderControl
// row). More robust than hardcoding either shape.
qa.labelFor = (el) => {
let node = el;
for (let i = 0; i < 3 && node; i++) {
const sib = node.previousElementSibling;
if (sib && sib.tagName === 'SPAN' && sib.textContent.trim()) return sib.textContent.trim();
node = node.parentElement;
}
return null;
};
qa.pickByLabel = (slug, label) => {
const inputs = qa.sectionInputs(slug);
if (!inputs) return null;
return inputs.find((el) => qa.labelFor(el) === label) || null;
};
qa.pickByType = (slug, type) => {
const inputs = qa.sectionInputs(slug);
if (!inputs) return null;
const matches = inputs.filter((el) => el.type === type);
return matches.length === 1 ? matches[0] : null;
};
qa.setEl = (el, value) => {
if (!el) return { error: 'no matching field' };
const from = el.value;
if (el.tagName === 'SELECT') {
const setter = Object.getOwnPropertyDescriptor(HTMLSelectElement.prototype, 'value').set;
setter.call(el, value);
el.dispatchEvent(new Event('change', { bubbles: true }));
return { from, to: value };
}
el.focus();
const proto = el.tagName === 'TEXTAREA' ? HTMLTextAreaElement.prototype : HTMLInputElement.prototype;
const setter = Object.getOwnPropertyDescriptor(proto, 'value').set;
setter.call(el, value);
el.dispatchEvent(new Event('input', { bubbles: true }));
if (el.type === 'range') {
el.dispatchEvent(new Event('change', { bubbles: true }));
el.dispatchEvent(new PointerEvent('pointerup', { bubbles: true }));
} else {
el.dispatchEvent(new KeyboardEvent('keydown', { key: 'Enter', bubbles: true }));
el.blur();
}
return { from, to: value };
};
qa.enableInspector = () => {
if ([...document.querySelectorAll('h3, [class*="panel"]')].length && document.body.textContent.includes('Select an element')) return 'on';
const btn = [...document.querySelectorAll('button')].find(b => b.textContent.trim() === 'Inspector');
if (!btn) return 'no inspector button';
btn.click();
return 'toggled';
};
qa.injectFault = (match, status) => { window.__qaFault = { match, status: status || 500 }; return 'armed'; };
window.__qa = qa;
return 'qa ready';
})()`;
const FRAME_SIGNATURE_EXPR =
"(() => { const f = window.__qa.frame(); if (!f) return null; const r = f.getBoundingClientRect(); return Math.round(r.x*1000+r.y*100+r.width*10+r.height); })()";
// A prior commit can resize the property panel (opening it, or its content
// changing height), which shifts the preview frame's on-page position. Computing
// click coordinates mid-reflow silently clicks the wrong spot — the click still
// lands on the overlay, so it doesn't error, it just selects nothing (or the
// wrong element). Wait for two consecutive reads of the frame's rect to agree
// before trusting it, instead of guessing how long a reflow takes.
async function waitForStableFrame({ tries = 10, interval = 100 } = {}) {
let prev = abEval(FRAME_SIGNATURE_EXPR);
for (let i = 0; i < tries; i++) {
await sleep(interval);
const next = abEval(FRAME_SIGNATURE_EXPR);
if (next != null && next === prev) return true;
prev = next;
}
return false;
}
async function select(sel) {
abEval("window.__qa.clear()");
await waitForStableFrame();
const coords = abEval(`window.__qa.coords(${JSON.stringify(sel)})`);
if (!Array.isArray(coords)) return { error: "no coords" };
ab("mouse", "move", String(coords[0]), String(coords[1]));
ab("mouse", "down", "left");
await sleep(150); // deliberate gesture delay to simulate a real click, not an async wait
ab("mouse", "up", "left");
await waitFor("window.__qa.lastSel() !== null");
return abEval("window.__qa.lastSel()");
}
// `pick` locates the field to edit: byLabel("Size") finds the input beside a
// "Size" label; byType("range") finds the section's sole range input. Neither
// depends on knowing the fixture's current value ahead of time.
const byLabel = (label) => (slug) =>
`window.__qa.pickByLabel(${JSON.stringify(slug)}, ${JSON.stringify(label)})`;
const byType = (type) => (slug) =>
`window.__qa.pickByType(${JSON.stringify(slug)}, ${JSON.stringify(type)})`;
async function commit(sectionSlug, pick, value) {
abEval("window.__qa.clear()");
abEval(`window.__qa.ensureSection(${JSON.stringify(sectionSlug)})`);
await waitFor(`(window.__qa.sectionInputs(${JSON.stringify(sectionSlug)}) || []).length > 0`);
const r = abEval(`window.__qa.setEl(${pick(sectionSlug)}, ${JSON.stringify(value)})`);
await waitFor("window.__patchLog.length > 0");
return r;
}
async function openStudio(url) {
ab("open", url);
await waitFor("document.readyState === 'complete' && !!document.querySelector('button')", {
timeout: 15000,
});
abEval(HELPERS);
// "a button exists" fires on the app shell alone; wait for the actual preview
// iframe to mount before handing control back, or a caller's first frame()
// lookup races the composition load and comes back null.
await waitFor("!!window.__qa.frame()", { timeout: 15000 });
}
// fallow-ignore-next-line complexity
async function main() {
await openStudio(`${STUDIO_URL}/?v=e2e${Date.now()}`);
abEval("window.__qa.enableInspector()");
await waitFor("document.body.textContent.includes('Select an element')", { timeout: 5000 });
let s = await select("#qa-headline");
check("select.headline", s && s.selector === "#qa-headline", JSON.stringify(s));
let r = await commit("text", byLabel("Size"), "72px");
check(
"text.size",
await waitForDisk("index.html", "font-size: 72px"),
`set=${JSON.stringify(r)}`,
);
s = await select("#qa-shape");
check("select.shape", s && s.selector === "#qa-shape", JSON.stringify(s));
r = await commit("transparency", byType("range"), "80");
check(
"style.opacity",
await waitForDisk("index.html", "opacity: 0.8"),
`set=${JSON.stringify(r)}`,
);
r = await commit("radius", byLabel("All"), "24");
check(
"style.radius",
await waitForDisk("index.html", "border-radius: 24px"),
`set=${JSON.stringify(r)}`,
);
s = await select("#qa-video");
check("select.video", s && s.selector === "#qa-video", JSON.stringify(s));
r = await commit("video", byLabel("Volume"), "80");
check(
"media.volume",
await waitForDisk("index.html", 'data-volume="0.8"'),
`set=${JSON.stringify(r)}`,
);
s = await select("#qa-sub-title");
check(
"select.sub-title",
s && s.selector === "#qa-sub-title" && s.src === "compositions/qa-sub.html",
JSON.stringify(s),
);
r = await commit("text", byLabel("Size"), "48px");
check(
"sub.text-size",
await waitForDisk("compositions/qa-sub.html", "font-size: 48px"),
`set=${JSON.stringify(r)}`,
);
// Fault injection: the server rejects the patch — the panel must surface the
// rejection and the value already on disk (72px, from the first cell) must
// survive untouched, not silently take on the value that failed to persist.
s = await select("#qa-headline");
check("select.headline-again", s && s.selector === "#qa-headline", JSON.stringify(s));
abEval("window.__qa.injectFault('file-mutations/patch-element', 500)");
r = await commit("text", byLabel("Size"), "90px");
await waitFor('document.body.textContent.includes("Couldn\'t save")', { timeout: 4000 });
const toastShown = abEval('document.body.textContent.includes("Couldn\'t save")');
check("fault.toast-shown", toastShown === true, `set=${JSON.stringify(r)}`);
check(
"fault.no-persist",
!disk("index.html", "font-size: 90px"),
"rejected value must not reach disk",
);
check(
"fault.prior-value-survives",
disk("index.html", "font-size: 72px"),
"prior committed value must survive",
);
// Reload survival for the headline edit.
await openStudio(`${STUDIO_URL}/?v=r${Date.now()}`);
const survived = abEval(
`(() => { const f = window.__qa.frame(); const el = f && f.contentDocument.querySelector('#qa-headline'); return el ? f.contentWindow.getComputedStyle(el).fontSize : null; })()`,
);
check("reload.survival", survived === "72px", String(survived));
console.log(failures === 0 ? "ALL PASS" : `${failures} FAILURES`);
process.exit(failures === 0 ? 0 : 1);
}
main().catch((e) => {
console.error("RUNNER ERROR", e);
process.exit(1);
});
@@ -0,0 +1,9 @@
{
"$schema": "https://hyperframes.heygen.com/schema/hyperframes.json",
"registry": "https://raw.githubusercontent.com/heygen-com/hyperframes/main/registry",
"paths": {
"blocks": "compositions",
"components": "compositions/components",
"assets": "assets"
}
}
@@ -0,0 +1,248 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>Design Panel QA Fixture</title>
<script src="https://cdn.jsdelivr.net/npm/gsap@3.14.2/dist/gsap.min.js"></script>
<style>
body,
html {
margin: 0;
padding: 0;
width: 1920px;
height: 1080px;
overflow: hidden;
background: #101418;
font-family: Arial, Helvetica, sans-serif;
color: #f0f0f0;
}
#design-panel-qa {
position: relative;
width: 1920px;
height: 1080px;
overflow: hidden;
}
.zone {
position: absolute;
width: 560px;
height: 300px;
border: 2px dashed #3a4652;
padding: 12px;
box-sizing: border-box;
}
#qa-headline {
font-size: 48px;
margin: 0;
color: #ffd166;
}
#qa-multi span {
display: block;
}
.qa-line-a {
font-size: 32px;
color: #06d6a0;
}
.qa-line-b {
font-size: 24px;
font-weight: 700;
color: #ef476f;
}
#qa-tween-box,
#qa-keyframe-box {
width: 120px;
height: 120px;
background: #118ab2;
border-radius: 8px;
}
#qa-keyframe-box {
background: #9b5de5;
}
#qa-shape {
width: 200px;
height: 140px;
background: #ff9f1c;
border-radius: 16px;
}
#qa-caption-host span {
display: inline-block;
font-size: 28px;
margin-right: 10px;
color: #cdeafe;
}
#qa-video {
position: absolute;
left: 1330px;
top: 380px;
width: 320px;
height: 180px;
}
</style>
</head>
<body>
<div
id="design-panel-qa"
data-composition-id="design-panel-qa"
data-width="1920"
data-height="1080"
data-start="0"
data-duration="6"
data-fps="30"
>
<!-- media: framework-owned playback, direct child of composition root -->
<video
id="qa-video"
class="clip"
data-hf-id="qa-video"
data-start="0"
data-duration="2"
data-track-index="7"
data-volume="0.5"
src="assets/test.mp4"
muted
playsinline
></video>
<!-- plain static text -->
<div
id="qa-zone-headline"
class="clip zone"
data-hf-id="qa-zone-headline"
data-start="0"
data-duration="6"
data-track-index="1"
style="left: 40px; top: 40px"
>
<h1 id="qa-headline" data-hf-id="qa-headline">Static Headline</h1>
</div>
<!-- multi-text-field element: parent with styled child spans -->
<div
id="qa-zone-multi"
class="clip zone"
data-hf-id="qa-zone-multi"
data-start="0"
data-duration="6"
data-track-index="2"
style="left: 680px; top: 40px"
>
<div id="qa-multi" data-hf-id="qa-multi">
<span class="qa-line-a">First styled line</span>
<span class="qa-line-b">Second styled line</span>
</div>
</div>
<!-- GSAP-tweened element -->
<div
id="qa-zone-tween"
class="clip zone"
data-hf-id="qa-zone-tween"
data-start="0"
data-duration="6"
data-track-index="3"
style="left: 40px; top: 380px"
>
<div id="qa-tween-box" data-hf-id="qa-tween-box"></div>
</div>
<!-- keyframed element -->
<div
id="qa-zone-keyframe"
class="clip zone"
data-hf-id="qa-zone-keyframe"
data-start="0"
data-duration="6"
data-track-index="4"
style="left: 680px; top: 380px"
>
<div id="qa-keyframe-box" data-hf-id="qa-keyframe-box"></div>
</div>
<!-- image -->
<div
id="qa-zone-image"
class="clip zone"
data-hf-id="qa-zone-image"
data-start="0"
data-duration="6"
data-track-index="5"
style="left: 40px; top: 720px"
>
<img
id="qa-image"
data-hf-id="qa-image"
width="240"
height="180"
alt="QA test pattern"
src="data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNDAiIGhlaWdodD0iMTgwIj48cmVjdCB3aWR0aD0iMjQwIiBoZWlnaHQ9IjE4MCIgZmlsbD0iIzJhOWQ4ZiIvPjxjaXJjbGUgY3g9IjEyMCIgY3k9IjkwIiByPSI2MCIgZmlsbD0iI2U5YzQ2YSIvPjwvc3ZnPg=="
/>
</div>
<!-- plain shape -->
<div
id="qa-zone-shape"
class="clip zone"
data-hf-id="qa-zone-shape"
data-start="0"
data-duration="6"
data-track-index="6"
style="left: 680px; top: 720px"
>
<div id="qa-shape" data-hf-id="qa-shape"></div>
</div>
<!-- runtime-generated caption-like spans (no static hf-ids on the words) -->
<div
id="qa-zone-captions"
class="clip zone"
data-hf-id="qa-zone-captions"
data-start="0"
data-duration="6"
data-track-index="8"
style="left: 1320px; top: 720px"
>
<div id="qa-caption-host" data-hf-id="qa-caption-host"></div>
</div>
<!-- sub-composition host + editable child -->
<div
id="qa-sub-layer"
class="clip"
data-hf-id="qa-sub-layer"
data-composition-id="qa-sub"
data-composition-src="compositions/qa-sub.html"
data-start="0"
data-duration="6"
data-track-index="9"
style="position: absolute; left: 1320px; top: 40px; width: 560px; height: 300px"
></div>
</div>
<script>
// Runtime-generated caption words: deterministic, built synchronously at load.
(function () {
const host = document.getElementById("qa-caption-host");
for (const word of ["Generated", "caption", "words", "here"]) {
const span = document.createElement("span");
span.textContent = word;
host.appendChild(span);
}
})();
window.__timelines = window.__timelines || {};
const tl = gsap.timeline({ paused: true });
tl.to("#qa-tween-box", { x: 300, rotation: 90, duration: 2, ease: "power1.inOut" }, 0);
tl.to(
"#qa-keyframe-box",
{
keyframes: [
{ x: 0, y: 0, duration: 1 },
{ x: 250, y: 40, duration: 1 },
{ x: 0, y: 0, duration: 1 },
],
},
0,
);
window.__timelines["design-panel-qa"] = tl;
</script>
</body>
</html>
@@ -0,0 +1,4 @@
{
"name": "design-panel-qa-fixture",
"private": true
}
+88
View File
@@ -0,0 +1,88 @@
#!/bin/bash
# E2E test helpers for agent-browser that work reliably with React components.
# Uses JS-based clicks (element.click()) instead of pointer events to avoid
# stale ref issues after iframe reloads.
# Click a button by its exact text content
click_button() {
local text="$1"
agent-browser eval --stdin <<EVALEOF
(function() {
var btns = document.querySelectorAll('button');
for (var i = 0; i < btns.length; i++) {
if (btns[i].textContent.trim() === '${text}') {
btns[i].click();
return 'clicked: ${text}';
}
}
return 'not found: ${text}';
})()
EVALEOF
}
# Click a button whose text contains a substring
click_button_contains() {
local text="$1"
agent-browser eval --stdin <<EVALEOF
(function() {
var btns = document.querySelectorAll('button');
for (var i = 0; i < btns.length; i++) {
if (btns[i].textContent.includes('${text}')) {
btns[i].click();
return 'clicked: ' + btns[i].textContent.trim().substring(0, 60);
}
}
return 'not found: ${text}';
})()
EVALEOF
}
# Set an input value by label text and commit via Enter
set_input() {
local label="$1"
local value="$2"
agent-browser eval --stdin <<EVALEOF
(function() {
var labels = document.querySelectorAll('label, span');
for (var i = 0; i < labels.length; i++) {
if (labels[i].textContent.trim() === '${label}') {
var input = labels[i].closest('[class]')?.querySelector('input');
if (!input) continue;
var nativeSetter = Object.getOwnPropertyDescriptor(window.HTMLInputElement.prototype, 'value').set;
nativeSetter.call(input, '${value}');
input.dispatchEvent(new Event('input', {bubbles: true}));
input.dispatchEvent(new KeyboardEvent('keydown', {key: 'Enter', code: 'Enter', bubbles: true}));
return 'set ${label} = ${value}';
}
}
return 'not found: ${label}';
})()
EVALEOF
}
# Select from a combobox/select by label and option value
select_option() {
local label="$1"
local value="$2"
agent-browser eval --stdin <<EVALEOF
(function() {
var selects = document.querySelectorAll('select');
for (var i = 0; i < selects.length; i++) {
var lbl = selects[i].closest('[class]')?.querySelector('label, span');
if (lbl && lbl.textContent.trim() === '${label}') {
selects[i].value = '${value}';
selects[i].dispatchEvent(new Event('change', {bubbles: true}));
return 'selected ${label} = ${value}';
}
}
return 'not found: ${label}';
})()
EVALEOF
}
# Wait for text to appear in the page
wait_for_text() {
local text="$1"
local timeout="${2:-10000}"
agent-browser wait --text "${text}" --timeout "${timeout}" 2>/dev/null || echo "timeout waiting for: ${text}"
}