85453da49f
regression / regression-shards (style-16-prod style-9-prod style-17-prod iframe-render-compat variables-prod mp4-h265-sdr, shard-4) (push) Has been cancelled
regression / regression-shards (style-4-prod style-11-prod style-2-prod animejs-adapter typegpu-adapter parallel-capture-regression, shard-5) (push) Has been cancelled
regression / regression-shards (style-7-prod style-8-prod style-10-prod css-spinner-render-compat webm-transparency mp4-h264-sdr webm-vp9, shard-3) (push) Has been cancelled
regression / regression-shards (sub-composition-video style-18-prod raf-ball-render-compat font-variant-numeric sub-comp-t0 sub-comp-id-selector, shard-7) (push) Has been cancelled
Windows render verification / Detect changes (push) Has been cancelled
Windows render verification / Preflight (lint + format) (push) Has been cancelled
Windows render verification / Render on windows-latest (push) Has been cancelled
Windows render verification / Tests on windows-latest (push) Has been cancelled
CI / Detect changes (push) Has been cancelled
CI / Build (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Fallow audit (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Typecheck (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Producer: integration tests (push) Has been cancelled
CI / Producer: unit tests (push) Has been cancelled
CI / File size check (push) Has been cancelled
CI / Test: skills (push) Has been cancelled
CI / Skills: manifest in sync (push) Has been cancelled
CI / CLI: npx shim (macos-latest) (push) Has been cancelled
CI / CLI: npx shim (ubuntu-latest) (push) Has been cancelled
CI / CLI: npx shim (windows-latest) (push) Has been cancelled
CI / SDK: unit + contract + smoke (push) Has been cancelled
CI / Test: runtime contract (push) Has been cancelled
CI / Studio: load smoke (push) Has been cancelled
CI / Smoke: global install (push) Has been cancelled
CI / CLI smoke (required) (push) Has been cancelled
CI / Semantic PR title (push) Has been cancelled
Player perf / Detect changes (push) Has been cancelled
Player perf / Preflight (lint + format) (push) Has been cancelled
Player perf / player-perf (push) Has been cancelled
Player perf / Perf: drift (push) Has been cancelled
Player perf / Perf: fps (push) Has been cancelled
Player perf / Perf: parity (push) Has been cancelled
Player perf / Perf: scrub (push) Has been cancelled
Player perf / Perf: load (push) Has been cancelled
preview-regression / Detect changes (push) Has been cancelled
preview-regression / Preflight (lint + format) (push) Has been cancelled
preview-regression / Preview parity (push) Has been cancelled
preview-regression / preview-regression (push) Has been cancelled
regression / regression (push) Has been cancelled
regression / Detect changes (push) Has been cancelled
regression / Preflight (lint + format) (push) Has been cancelled
regression / regression-shards (hdr-regression style-5-prod style-3-prod mov-prores, shard-1) (push) Has been cancelled
regression / regression-shards (overlay-montage-prod style-12-prod chat missing-host-comp-id png-sequence portrait-edge-bleed, shard-6) (push) Has been cancelled
regression / regression-shards (style-13-prod style-6-prod vignelli-stacking gsap-letters-render-compat audio-mux-parity, shard-8) (push) Has been cancelled
regression / regression-shards (style-15-prod hdr-hlg-regression style-1-prod many-cuts vfr-screen-recording render-symlinked-assets, shard-2) (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Docs / Validate docs (push) Has been cancelled
Sync skills to ClawHub / Publish changed skills (push) Has been cancelled
192 lines
8.8 KiB
Markdown
192 lines
8.8 KiB
Markdown
# @hyperframes/aws-lambda
|
||
|
||
AWS Lambda adapter for HyperFrames distributed rendering. Ships three
|
||
things together:
|
||
|
||
1. The **Lambda handler** that wraps the OSS `plan` / `renderChunk` /
|
||
`assemble` primitives behind a single dispatch boundary Step Functions
|
||
can drive (`src/handler.ts`).
|
||
2. A **client-side SDK** — `renderToLambda`, `getRenderProgress`,
|
||
`deploySite`, plus `validateDistributedRenderConfig` and
|
||
`computeRenderCost` (`src/sdk/`).
|
||
3. An **`aws-cdk-lib` L2 construct** (`HyperframesRenderStack`) that
|
||
provisions the same topology as `examples/aws-lambda/template.yaml`
|
||
inside an adopter's own CDK app (`src/cdk/`).
|
||
|
||
The handler ZIP and the SAM template still drive a maintainer-run real-AWS
|
||
smoke flow; the SDK + CDK are the supported public surface for adopters.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────────────────────┐
|
||
│ Step Functions state machine │
|
||
│ Plan → Map(N) RenderChunk → Assemble │
|
||
└──────────────────────────────────────────────────────────────────┘
|
||
│ dispatches by event.Action
|
||
▼
|
||
┌──────────────────────────────────────────────────────────────────┐
|
||
│ One Lambda function (this package's `dist/handler.zip`) │
|
||
│ handler.mjs │
|
||
│ ├─ Action="plan" → @hyperframes/producer/distributed │
|
||
│ ├─ Action="renderChunk" → @hyperframes/producer/distributed │
|
||
│ └─ Action="assemble" → @hyperframes/producer/distributed │
|
||
│ bin/ffmpeg — ffmpeg-static │
|
||
│ node_modules/@sparticuz/chromium/ — Lambda-optimised Chromium │
|
||
└──────────────────────────────────────────────────────────────────┘
|
||
│ pure functions over local paths
|
||
▼
|
||
┌──────────────────────────────────────────────────────────────────┐
|
||
│ S3 bucket — plan tarball + per-chunk outputs + final mp4 │
|
||
└──────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
The handler downloads inputs from S3 into `/tmp`, calls the OSS primitive,
|
||
uploads outputs back to S3, and returns a small JSON result that fits
|
||
inside Step Functions' history budget (under 200 bytes per chunk).
|
||
|
||
## Chrome runtime
|
||
|
||
The package supports two Chromium sources:
|
||
|
||
| Source | Default | Size | When to pick it |
|
||
| ------------------------------- | ------- | ------------------ | --------------------------------------------------------------------------------------------------------------------- |
|
||
| `@sparticuz/chromium` | yes | ~70 MiB compressed | Lambda. Decompresses into `/tmp` at runtime; the rest of the ecosystem already uses it for headless-Chrome-in-Lambda. |
|
||
| Bundled `chrome-headless-shell` | no | ~140 MiB | Fallback. Used if `@sparticuz/chromium` ever drops `HeadlessExperimental.beginFrame` support. |
|
||
|
||
Pick the source at build time:
|
||
|
||
```bash
|
||
bun run --cwd packages/aws-lambda build:zip
|
||
bun run --cwd packages/aws-lambda build:zip -- --source=chrome-headless-shell
|
||
```
|
||
|
||
The handler reads `HYPERFRAMES_LAMBDA_CHROME_SOURCE` at boot. The build
|
||
script sets that env var via Lambda function configuration in
|
||
`examples/aws-lambda/template.yaml`.
|
||
|
||
## BeginFrame regression guard
|
||
|
||
HyperFrames' renderer drives Chrome via the CDP
|
||
`HeadlessExperimental.beginFrame` command — same path the K8s deploy uses.
|
||
The Lambda adapter assumes that `@sparticuz/chromium`'s
|
||
chrome-headless-shell build honours BeginFrame. To prove it (and re-prove
|
||
it on every release), the package ships a Docker probe:
|
||
|
||
```bash
|
||
# Build the Lambda-like container and run the probe.
|
||
bun run --cwd packages/aws-lambda probe:beginframe:docker
|
||
```
|
||
|
||
The probe boots `@sparticuz/chromium` inside
|
||
`public.ecr.aws/lambda/nodejs:22` and asserts CDP `beginFrame` with
|
||
`screenshot: true` returns a PNG buffer. Exit code 0 = green; non-zero =
|
||
fall back to bundling chrome-headless-shell directly via `--source=chrome-headless-shell`.
|
||
|
||
## Building the ZIP
|
||
|
||
```bash
|
||
bun install # at the monorepo root
|
||
bun run --cwd packages/aws-lambda build:zip # → packages/aws-lambda/dist/handler.zip
|
||
bun run --cwd packages/aws-lambda verify:zip-size # CI gate
|
||
```
|
||
|
||
The build script bundles `src/handler.ts` via esbuild, stages
|
||
`@sparticuz/chromium` and `puppeteer-core` under `node_modules/`, copies
|
||
ffmpeg-static into `bin/`, and zips the result. The unzipped layout is
|
||
designed to extract cleanly into Lambda's `/var/task/`.
|
||
|
||
`verify:zip-size` enforces:
|
||
|
||
- Unzipped ≤ 248 MiB (in-house budget; Lambda hard ceiling is 250 MiB unzipped — AWS docs label this "250 MB" but use binary mebibytes)
|
||
- Zipped ≤ 150 MiB (in-house budget; Lambda has no hard zipped cap for S3-deployed functions)
|
||
|
||
CI fails the PR if either is exceeded.
|
||
|
||
## Running tests
|
||
|
||
```bash
|
||
bun run --cwd packages/aws-lambda test # unit tests (no Chrome)
|
||
bun run --cwd packages/aws-lambda probe:beginframe # local probe (Linux only)
|
||
```
|
||
|
||
## Using the SDK
|
||
|
||
After deploying the stack (via the SAM template, CDK construct below, or
|
||
your own CFN of choice), drive renders from Node:
|
||
|
||
```ts
|
||
import { deploySite, getRenderProgress, renderToLambda } from "@hyperframes/aws-lambda";
|
||
|
||
// One-time upload per project version.
|
||
const site = await deploySite({
|
||
projectDir: "./my-composition",
|
||
bucketName: "hyperframes-render-bucket",
|
||
});
|
||
|
||
// Start a render. Returns immediately — does NOT poll.
|
||
const handle = await renderToLambda({
|
||
siteHandle: site,
|
||
bucketName: site.bucketName,
|
||
stateMachineArn: "arn:aws:states:us-east-1:123:stateMachine:hyperframes-render",
|
||
config: {
|
||
fps: 30,
|
||
width: 1920,
|
||
height: 1080,
|
||
format: "mp4",
|
||
chunkSize: 240,
|
||
maxParallelChunks: 16,
|
||
runtimeCap: "lambda",
|
||
},
|
||
});
|
||
|
||
// Poll progress + cost on your own cadence.
|
||
const progress = await getRenderProgress({ executionArn: handle.executionArn });
|
||
console.log(progress.overallProgress, progress.costs.displayCost);
|
||
if (progress.status === "SUCCEEDED" && progress.outputFile) {
|
||
console.log("Render landed at", progress.outputFile.s3Uri);
|
||
}
|
||
```
|
||
|
||
`renderToLambda` validates the config client-side via
|
||
`validateDistributedRenderConfig` and throws a typed `InvalidConfigError`
|
||
before the Step Functions execution starts, so shape errors surface
|
||
synchronously instead of as opaque `ExecutionFailed` results.
|
||
|
||
`getRenderProgress` reports an approximate per-render cost
|
||
(`accruedSoFarUsd` plus a formatted `displayCost`) derived from Lambda
|
||
billed-duration × memory × the us-east-1 on-demand rate plus the Step
|
||
Functions transition price. The math is documented in
|
||
`src/sdk/costAccounting.ts`; numbers are best-effort and exclude S3
|
||
transfer.
|
||
|
||
## Using the CDK construct
|
||
|
||
```ts
|
||
import { App, Stack } from "aws-cdk-lib";
|
||
import { HyperframesRenderStack } from "@hyperframes/aws-lambda/cdk";
|
||
|
||
const app = new App();
|
||
const stack = new Stack(app, "MyApp");
|
||
const render = new HyperframesRenderStack(stack, "Render", {
|
||
// optional: reservedConcurrency: 8,
|
||
// optional: lambdaMemoryMb: 10240,
|
||
// optional: chromeSource: "sparticuz",
|
||
});
|
||
|
||
// Re-export so an adopter app can wire dashboards / SNS topics.
|
||
new CfnOutput(stack, "RenderBucketName", { value: render.bucket.bucketName });
|
||
new CfnOutput(stack, "StateMachineArn", { value: render.stateMachine.stateMachineArn });
|
||
```
|
||
|
||
`aws-cdk-lib` and `constructs` are **optional peer dependencies**: SDK-only
|
||
consumers don't pull them at runtime. The construct itself imports from
|
||
`@hyperframes/aws-lambda/cdk`.
|
||
|
||
## What's still ahead
|
||
|
||
- `hyperframes lambda` CLI (deploy / sites create / render / progress / destroy) — PR 6.5.
|
||
- IAM bootstrap subcommand (`policies role | user | validate`) — PR 6.9.
|
||
- Lambda-local regression harness (`--mode=lambda-local`) — PR 6.6.
|
||
- Adopter-facing migration guide — PR 6.8.
|