85453da49f
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
Docs / Validate docs (push) Waiting to run
CodeQL / Analyze (actions) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
Sync skills to ClawHub / Publish changed skills (push) Waiting to run
regression / regression-shards (style-16-prod style-9-prod style-17-prod iframe-render-compat variables-prod mp4-h265-sdr, shard-4) (push) Has been cancelled
regression / regression-shards (style-4-prod style-11-prod style-2-prod animejs-adapter typegpu-adapter parallel-capture-regression, shard-5) (push) Has been cancelled
regression / regression-shards (style-7-prod style-8-prod style-10-prod css-spinner-render-compat webm-transparency mp4-h264-sdr webm-vp9, shard-3) (push) Has been cancelled
regression / regression-shards (sub-composition-video style-18-prod raf-ball-render-compat font-variant-numeric sub-comp-t0 sub-comp-id-selector, shard-7) (push) Has been cancelled
Windows render verification / Detect changes (push) Has been cancelled
Windows render verification / Preflight (lint + format) (push) Has been cancelled
Windows render verification / Render on windows-latest (push) Has been cancelled
Windows render verification / Tests on windows-latest (push) Has been cancelled
CI / Detect changes (push) Has been cancelled
CI / Build (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Fallow audit (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Typecheck (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Producer: integration tests (push) Has been cancelled
CI / Producer: unit tests (push) Has been cancelled
CI / File size check (push) Has been cancelled
CI / Test: skills (push) Has been cancelled
CI / Skills: manifest in sync (push) Has been cancelled
CI / CLI: npx shim (macos-latest) (push) Has been cancelled
CI / CLI: npx shim (ubuntu-latest) (push) Has been cancelled
CI / CLI: npx shim (windows-latest) (push) Has been cancelled
CI / SDK: unit + contract + smoke (push) Has been cancelled
CI / Test: runtime contract (push) Has been cancelled
CI / Studio: load smoke (push) Has been cancelled
CI / Smoke: global install (push) Has been cancelled
CI / CLI smoke (required) (push) Has been cancelled
CI / Semantic PR title (push) Has been cancelled
Player perf / Detect changes (push) Has been cancelled
Player perf / Preflight (lint + format) (push) Has been cancelled
Player perf / player-perf (push) Has been cancelled
Player perf / Perf: drift (push) Has been cancelled
Player perf / Perf: fps (push) Has been cancelled
Player perf / Perf: parity (push) Has been cancelled
Player perf / Perf: scrub (push) Has been cancelled
Player perf / Perf: load (push) Has been cancelled
preview-regression / Detect changes (push) Has been cancelled
preview-regression / Preflight (lint + format) (push) Has been cancelled
preview-regression / Preview parity (push) Has been cancelled
preview-regression / preview-regression (push) Has been cancelled
regression / regression (push) Has been cancelled
regression / Detect changes (push) Has been cancelled
regression / Preflight (lint + format) (push) Has been cancelled
regression / regression-shards (hdr-regression style-5-prod style-3-prod mov-prores, shard-1) (push) Has been cancelled
regression / regression-shards (overlay-montage-prod style-12-prod chat missing-host-comp-id png-sequence portrait-edge-bleed, shard-6) (push) Has been cancelled
regression / regression-shards (style-13-prod style-6-prod vignelli-stacking gsap-letters-render-compat audio-mux-parity, shard-8) (push) Has been cancelled
regression / regression-shards (style-15-prod hdr-hlg-regression style-1-prod many-cuts vfr-screen-recording render-symlinked-assets, shard-2) (push) Has been cancelled
218 lines
13 KiB
Plaintext
218 lines
13 KiB
Plaintext
---
|
||
title: AWS Lambda
|
||
description: "Deploy distributed HyperFrames rendering to AWS Lambda and drive renders from a laptop or CI."
|
||
---
|
||
|
||
HyperFrames ships a first-class AWS Lambda deployment: one Lambda function fronts a Step Functions standard workflow that fans renders out across many parallel chunk workers, with intermediate artifacts in S3. End-to-end is three commands once your AWS credentials are configured.
|
||
|
||
```bash
|
||
hyperframes lambda deploy
|
||
hyperframes lambda render ./my-project --width 1920 --height 1080 --wait
|
||
hyperframes lambda destroy
|
||
```
|
||
|
||
Templates with [variables](/concepts/variables) work on the same Lambda stack — declare `data-composition-variables` on the composition, then pass values per render with `--variables` or fan a whole batch out with `lambda render-batch`. See the [Templates on Lambda](/deploy/templates-on-lambda) guide for the personalised-render pipeline (single render, batch from JSONL, programmatic SDK) and the 256 KiB Step Functions execution-input cap.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────────────────────┐
|
||
│ Step Functions state machine │
|
||
│ Plan → Map(N) RenderChunk → Assemble │
|
||
└──────────────────────────────────────────────────────────────────┘
|
||
│ dispatches by event.Action
|
||
▼
|
||
┌──────────────────────────────────────────────────────────────────┐
|
||
│ One Lambda function (packages/aws-lambda/dist/handler.zip) │
|
||
│ handler.mjs │
|
||
│ ├─ Action="plan" → @hyperframes/producer/distributed │
|
||
│ ├─ Action="renderChunk" → @hyperframes/producer/distributed │
|
||
│ └─ Action="assemble" → @hyperframes/producer/distributed │
|
||
│ bin/ffmpeg — ffmpeg-static │
|
||
│ node_modules/@sparticuz/chromium/ — Lambda-optimised Chromium │
|
||
└──────────────────────────────────────────────────────────────────┘
|
||
│ pure functions over local paths
|
||
▼
|
||
┌──────────────────────────────────────────────────────────────────┐
|
||
│ S3 bucket — plan tarball + per-chunk outputs + final mp4 │
|
||
└──────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
The Lambda handler is a thin dispatch: parse the Step Functions event, download inputs from S3 into `/tmp`, call the OSS primitive from `@hyperframes/producer/distributed`, upload outputs back, return a small JSON result. Everything heavy — capture, encode, audio mix — happens inside the OSS primitives.
|
||
|
||
## Prerequisites
|
||
|
||
| Tool | Why | Install |
|
||
|------|-----|---------|
|
||
| AWS credentials | The CLI and the deploy step both call AWS APIs. | Env vars, `~/.aws/credentials`, SSO, or IMDS — any chain the AWS SDK for JavaScript v3 would resolve. |
|
||
| AWS SAM CLI | `hyperframes lambda deploy/destroy` shells out to `sam deploy`/`sam delete`. | [Install guide](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) |
|
||
| `bun` | Used to build `packages/aws-lambda/dist/handler.zip` at deploy time. | `npm install -g bun` or [bun.sh](https://bun.sh) |
|
||
| HyperFrames repo checkout | `lambda deploy` builds the Lambda handler ZIP from source. Adopters who deploy outside a checkout can set `HYPERFRAMES_REPO_ROOT` to point at one. | `git clone https://github.com/heygen-com/hyperframes` |
|
||
|
||
## Three deployment paths
|
||
|
||
### Path 1 — `hyperframes lambda` CLI (recommended)
|
||
|
||
The CLI is a thin wrapper around the SAM template + the `@hyperframes/aws-lambda` SDK. For most adopters this is the right starting point.
|
||
|
||
```bash
|
||
hyperframes lambda deploy \
|
||
--stack-name=hyperframes-prod \
|
||
--region=us-east-1 \
|
||
--concurrency=8 \
|
||
--memory=10240
|
||
```
|
||
|
||
The default `--concurrency=8` is deliberately conservative for first-time users. The Lambda Map state's default would let an unbounded number of chunks fan out in parallel; 8 caps your worst-case spend on a runaway render at roughly `8 × (15 min × 10 GB × $0.0000167/GB-s) ≈ $1.20`. Raise it after you've sized your typical render's chunk count.
|
||
|
||
After `deploy`, render anything with:
|
||
|
||
```bash
|
||
hyperframes lambda render ./my-project --width 1920 --height 1080 --wait
|
||
```
|
||
|
||
The `--wait` flag blocks and streams per-chunk progress + accrued cost; drop it to fire-and-forget, then poll with `hyperframes lambda progress <renderId>` on your own cadence.
|
||
|
||
See the [CLI reference](/packages/cli#hyperframes-lambda) for full flag documentation.
|
||
|
||
#### Pre-staging a project with `sites create`
|
||
|
||
Re-rendering the same project tree on every `lambda render` call re-tars and re-uploads it each time. For tight inner loops (CI smoke jobs, prompt iteration in a demo flow), pre-stage the project once and reuse the upload:
|
||
|
||
```bash
|
||
hyperframes lambda sites create ./my-project
|
||
# → Site ID: a1b2c3d4e5f6g7h8 (content-addressed)
|
||
|
||
hyperframes lambda render ./my-project --site-id=a1b2c3d4e5f6g7h8 \
|
||
--width 1920 --height 1080 --wait
|
||
```
|
||
|
||
The `siteId` is content-addressed via a SHA-256 of the project tree; re-running `sites create` on an unchanged tree skips the upload via a `HeadObject` short-circuit. Pass the same `--site-id` to as many `lambda render` calls as you like — they all reuse the one S3 PUT.
|
||
|
||
### Path 2 — Direct SAM deploy
|
||
|
||
If you want to read the CloudFormation before you deploy, or you need to customise the topology (extra alarms, SNS subscribers, KMS keys, …), invoke SAM directly against the template at `examples/aws-lambda/template.yaml`:
|
||
|
||
```bash
|
||
cd packages/aws-lambda
|
||
bun run build:zip # produces dist/handler.zip
|
||
cd ../../examples/aws-lambda
|
||
sam deploy \
|
||
--stack-name=hyperframes-prod \
|
||
--region=us-east-1 \
|
||
--resolve-s3 \
|
||
--capabilities CAPABILITY_IAM \
|
||
--no-confirm-changeset \
|
||
--parameter-overrides ChromeSource=sparticuz ReservedConcurrency=8
|
||
```
|
||
|
||
The template emits three CloudFormation outputs you'll need to invoke renders:
|
||
|
||
- `RenderBucketName` — S3 bucket for plan tarballs + per-chunk outputs + final renders.
|
||
- `RenderStateMachineArn` — the Step Functions standard workflow that orchestrates Plan → Map → Assemble.
|
||
- `RenderFunctionArn` — the single Lambda function the state machine dispatches against.
|
||
|
||
<Warning>
|
||
The SAM template's own default for `ReservedConcurrency` is `-1` (unreserved, account-default). The Path 1 CLI overrides it to `8` to keep first-time spend bounded; if you drop `ReservedConcurrency` from `--parameter-overrides` here, you get the unreserved default. Set it explicitly unless you've already sized your typical render's fan-out.
|
||
</Warning>
|
||
|
||
### Path 3 — CDK construct
|
||
|
||
For users already running CDK, the `@hyperframes/aws-lambda` package exports a `HyperframesRenderStack` L2 construct that emits the same topology as the SAM template:
|
||
|
||
```ts
|
||
import { App, CfnOutput, Stack } from "aws-cdk-lib";
|
||
import { HyperframesRenderStack } from "@hyperframes/aws-lambda/cdk";
|
||
|
||
const app = new App();
|
||
const stack = new Stack(app, "MyApp");
|
||
const render = new HyperframesRenderStack(stack, "Render", {
|
||
projectName: "hyperframes",
|
||
lambdaMemoryMb: 10240,
|
||
reservedConcurrency: 8,
|
||
chromeSource: "sparticuz",
|
||
});
|
||
|
||
new CfnOutput(stack, "RenderBucketName", { value: render.bucket.bucketName });
|
||
new CfnOutput(stack, "StateMachineArn", { value: render.stateMachine.stateMachineArn });
|
||
```
|
||
|
||
`aws-cdk-lib` and `constructs` are declared as **optional peer dependencies** of `@hyperframes/aws-lambda`, so consumers who only need the SDK don't pay the CDK import cost.
|
||
|
||
The construct exposes `.bucket`, `.renderFunction`, and `.stateMachine` so you can wire dashboards, SNS topics, or other AWS resources alongside it without re-deriving ARNs.
|
||
|
||
## IAM permissions
|
||
|
||
The CLI ships a built-in IAM bootstrap to avoid the "User is not authorized to perform iam:CreateRole" first-deploy trap:
|
||
|
||
```bash
|
||
# Print an inline policy doc to attach to the IAM user that runs the CLI.
|
||
hyperframes lambda policies user
|
||
|
||
# Print { TrustRelationship, InlinePolicy } for a CloudFormation service role.
|
||
hyperframes lambda policies role --principal=cloudformation
|
||
|
||
# Validate a checked-in policy still covers the CLI's needs (exit non-zero on missing).
|
||
hyperframes lambda policies validate ./infra/iam/hyperframes-deploy.json
|
||
```
|
||
|
||
The generated documents grant `Resource: "*"` for the CLI's required action set. After your first successful deploy you can narrow `Resource` to the deployed ARNs — predictable per the CloudFormation outputs above. Adopters running the CLI in CI typically check the policy doc into source control and run `policies validate` as a pre-deploy step to catch drift.
|
||
|
||
## Cost shape
|
||
|
||
Lambda renders are billed by GB-seconds (Lambda billed duration × configured memory) plus a tiny per-state-transition fee for Step Functions standard workflows. `hyperframes lambda progress` exposes the running tally:
|
||
|
||
```bash
|
||
hyperframes lambda progress my-render-id
|
||
# Status: SUCCEEDED
|
||
# Progress: 100%
|
||
# Frames: 480 / 480
|
||
# Lambdas: 5
|
||
# Cost: $0.0214 (Lambda $0.0210 + SFN $0.0004)
|
||
# Output: s3://hyperframes-renders/.../output.mp4
|
||
```
|
||
|
||
The cost number is best-effort: Lambda billed duration comes from the handler's own `DurationMs` return value (which SFN history surfaces in the success payload) and S3 transfer is not included. The math is in `packages/aws-lambda/src/sdk/costAccounting.ts` if you want to verify; CLI-shown values match what AWS Billing reports within rounding noise.
|
||
|
||
## Troubleshooting
|
||
|
||
### `sam deploy` fails with "Stack already exists"
|
||
|
||
Pass the same `--stack-name` you used the first time. SAM is idempotent — re-running on an existing stack resolves to a no-op or an in-place update.
|
||
|
||
### `User is not authorized to perform iam:CreateRole`
|
||
|
||
The IAM credential running `lambda deploy` doesn't have permission to create the service role CloudFormation needs. Run `hyperframes lambda policies user` and attach the printed policy to your IAM user (or take the `policies role` output and have your admin create a deploy role).
|
||
|
||
### `Lambda function failed: PLAN_HASH_MISMATCH`
|
||
|
||
Step Functions invoked a `renderChunk` with a plan hash that didn't match the planDir on S3. Almost always means the producer version differs between the local `plan()` build and the deployed Lambda ZIP. Re-run `hyperframes lambda deploy` (which rebuilds the ZIP) and re-render.
|
||
|
||
### `Lambda function failed: BROWSER_GPU_NOT_SOFTWARE`
|
||
|
||
The handler launched Chromium but the runtime probe found a non-SwiftShader GL backend. Hardware GL is non-deterministic across chunk boundaries, so distributed renders refuse it at the runtime-image / launch-flags layer (not at the composition layer). Rebuild the handler ZIP and redeploy:
|
||
|
||
```bash
|
||
bun run --cwd packages/aws-lambda build:zip
|
||
hyperframes lambda deploy --stack-name=<your-stack>
|
||
```
|
||
|
||
The build pipeline pins `@sparticuz/chromium` + the Chrome flags (`--use-gl=swiftshader --use-angle=swiftshader`) so a fresh deploy almost always resolves this. If it persists, your stack's Lambda function is pointing at a stale handler ZIP from a previous deploy — `lambda deploy` always rebuilds, so re-running unsticks it.
|
||
|
||
### Render seems stuck at `RUNNING`
|
||
|
||
Most often a Lambda cold-start chain on a many-chunk render. The Map state's reserved concurrency caps how many chunks can run in parallel — if you set `--concurrency=4` and your render has 16 chunks, the state machine processes them in batches of 4. `hyperframes lambda progress <id>` shows how many invocations are in flight.
|
||
|
||
If progress doesn't advance for >10 minutes, check the Step Functions execution in the AWS console — failed Lambda invocations include the typed error name (`FONT_FETCH_FAILED`, `FFMPEG_VERSION_MISMATCH`, etc.) which short-circuits the state machine.
|
||
|
||
### Tearing down doesn't reclaim S3 storage
|
||
|
||
The render bucket is created with CloudFormation `Retain` on delete — `hyperframes lambda destroy` (or `sam delete`) tears the function + state machine down but the bucket survives. This is intentional: it protects final-rendered MP4s from being lost when you re-deploy. To fully reclaim storage, empty + delete the bucket via the AWS console / `aws s3 rb`.
|
||
|
||
## What's NOT in the v1 surface
|
||
|
||
- **Webhooks on completion.** Not in v1 — poll with `hyperframes lambda progress` or watch the Step Functions execution. A `--webhook` flag with an SNS topic is on the Phase 6c backlog.
|
||
- **`compositions` discovery verb.** Coming separately (PR 6.10 on the plan); for now, point `lambda render` at the project directory containing your `index.html`.
|
||
- **Multi-region.** Each `--region` is an independent stack. There is no built-in cross-region failover.
|
||
- **HDR.** Distributed mode is SDR-only. HDR mp4 with bsf signaling is on the v1.5 backlog.
|