Files
wehub-resource-sync 85453da49f
regression / regression-shards (style-16-prod style-9-prod style-17-prod iframe-render-compat variables-prod mp4-h265-sdr, shard-4) (push) Has been cancelled
regression / regression-shards (style-4-prod style-11-prod style-2-prod animejs-adapter typegpu-adapter parallel-capture-regression, shard-5) (push) Has been cancelled
regression / regression-shards (style-7-prod style-8-prod style-10-prod css-spinner-render-compat webm-transparency mp4-h264-sdr webm-vp9, shard-3) (push) Has been cancelled
regression / regression-shards (sub-composition-video style-18-prod raf-ball-render-compat font-variant-numeric sub-comp-t0 sub-comp-id-selector, shard-7) (push) Has been cancelled
Windows render verification / Detect changes (push) Has been cancelled
Windows render verification / Preflight (lint + format) (push) Has been cancelled
Windows render verification / Render on windows-latest (push) Has been cancelled
Windows render verification / Tests on windows-latest (push) Has been cancelled
CI / Detect changes (push) Has been cancelled
CI / Build (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Fallow audit (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Typecheck (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Producer: integration tests (push) Has been cancelled
CI / Producer: unit tests (push) Has been cancelled
CI / File size check (push) Has been cancelled
CI / Test: skills (push) Has been cancelled
CI / Skills: manifest in sync (push) Has been cancelled
CI / CLI: npx shim (macos-latest) (push) Has been cancelled
CI / CLI: npx shim (ubuntu-latest) (push) Has been cancelled
CI / CLI: npx shim (windows-latest) (push) Has been cancelled
CI / SDK: unit + contract + smoke (push) Has been cancelled
CI / Test: runtime contract (push) Has been cancelled
CI / Studio: load smoke (push) Has been cancelled
CI / Smoke: global install (push) Has been cancelled
CI / CLI smoke (required) (push) Has been cancelled
CI / Semantic PR title (push) Has been cancelled
Player perf / Detect changes (push) Has been cancelled
Player perf / Preflight (lint + format) (push) Has been cancelled
Player perf / player-perf (push) Has been cancelled
Player perf / Perf: drift (push) Has been cancelled
Player perf / Perf: fps (push) Has been cancelled
Player perf / Perf: parity (push) Has been cancelled
Player perf / Perf: scrub (push) Has been cancelled
Player perf / Perf: load (push) Has been cancelled
preview-regression / Detect changes (push) Has been cancelled
preview-regression / Preflight (lint + format) (push) Has been cancelled
preview-regression / Preview parity (push) Has been cancelled
preview-regression / preview-regression (push) Has been cancelled
regression / regression (push) Has been cancelled
regression / Detect changes (push) Has been cancelled
regression / Preflight (lint + format) (push) Has been cancelled
regression / regression-shards (hdr-regression style-5-prod style-3-prod mov-prores, shard-1) (push) Has been cancelled
regression / regression-shards (overlay-montage-prod style-12-prod chat missing-host-comp-id png-sequence portrait-edge-bleed, shard-6) (push) Has been cancelled
regression / regression-shards (style-13-prod style-6-prod vignelli-stacking gsap-letters-render-compat audio-mux-parity, shard-8) (push) Has been cancelled
regression / regression-shards (style-15-prod hdr-hlg-regression style-1-prod many-cuts vfr-screen-recording render-symlinked-assets, shard-2) (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Docs / Validate docs (push) Has been cancelled
Sync skills to ClawHub / Publish changed skills (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 12:58:35 +08:00

129 lines
5.2 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# @hyperframes/gcp-cloud-run
Google Cloud Run + Cloud Workflows adapter for HyperFrames distributed
rendering. The OSS render primitives (`plan``renderChunk` × N →
`assemble`) are pure functions over local file paths; this package is the
deployment, orchestration, and storage glue that runs them on Google Cloud —
the GCP counterpart to [`@hyperframes/aws-lambda`](../aws-lambda).
Two surfaces, one package:
- **Server-side handler** (`./server`) — a Cloud Run HTTP service that
dispatches `plan` / `renderChunk` / `assemble` on the request body's
`Action` field, bridging GCS ↔ the container's filesystem around each OSS
primitive. This is what the bundled `Dockerfile` runs.
- **Client-side SDK** (`./sdk`) — `renderToCloudRun`, `getRenderProgress`,
`deploySite`, `validateDistributedRenderConfig`, and `computeRenderCost`.
Call these from a Node process (CI, CLI, app backend) to drive a deployed
stack without writing GCS / Workflows boilerplate.
The package is **not** a dependency of `@hyperframes/producer`; install it
separately.
## Architecture
```
GCS bucket ←→ Cloud Run service (plan / renderChunk / assemble)
│ OIDC-authenticated http.post, one per step
Cloud Workflows (Plan → parallel RenderChunk → Assemble)
```
- **Plan** downloads the project tarball, runs `plan()`, uploads the planDir
tarball (+ audio) to GCS, and returns the chunk count.
- **RenderChunk** runs in a parallel `for` loop in the workflow, fanned out
up to the plan's chunk count. Each invocation renders one chunk and uploads
it.
- **Assemble** downloads every chunk + audio, stitches the final
deliverable, and uploads it.
Every step is a `POST` to the same Cloud Run URL with a different `Action`.
The workflow accumulates each step's small result body and returns
`{ Plan, Chunks, Assemble }` so `getRenderProgress` can read frame totals and
per-step durations on success.
## Chrome runtime
Unlike the Lambda adapter — which fights a 250 MB ZIP ceiling and
decompresses `@sparticuz/chromium` into `/tmp` at runtime — Cloud Run runs a
container image. The `Dockerfile` installs the same pinned
`chrome-headless-shell` build and font set the production renderer uses, at a
fixed path, and exports `HYPERFRAMES_CHROME_PATH`. CDP-level `BeginFrame`
works because the command lives in the protocol, not the binary. There is no
runtime decompression step and no packaging ceiling.
## Deploying
The `terraform/` module provisions everything: the GCS render bucket, the
Cloud Run service, the Cloud Workflows definition, two least-privilege
service accounts (the service reads/writes the bucket; the workflow invokes
the service), and a runaway-request alert.
```bash
# 1. Build + push the image (Cloud Build or local docker).
gcloud builds submit . \
--tag REGION-docker.pkg.dev/PROJECT/REPO/hyperframes-render:TAG
# 2. Apply the module.
terraform -chdir=node_modules/@hyperframes/gcp-cloud-run/terraform init
terraform -chdir=node_modules/@hyperframes/gcp-cloud-run/terraform apply \
-var project_id=PROJECT \
-var region=us-central1 \
-var image=REGION-docker.pkg.dev/PROJECT/REPO/hyperframes-render:TAG
```
Terraform outputs `render_bucket_name`, `service_url`, `workflow_name`, and
`region` — pass them straight into the SDK.
## Using the SDK
```ts
import { renderToCloudRun, getRenderProgress } from "@hyperframes/gcp-cloud-run/sdk";
const handle = await renderToCloudRun({
projectDir: "./my-composition",
config: { fps: 30, width: 1920, height: 1080, format: "mp4" },
bucketName: "hyperframes-render-my-project", // from terraform output
projectId: "my-project",
location: "us-central1",
workflowId: "hyperframes-render",
serviceUrl: "https://hyperframes-render-abc.us-central1.run.app",
});
// Poll until done.
let progress = await getRenderProgress({ executionName: handle.executionName });
while (progress.status === "running") {
await new Promise((r) => setTimeout(r, 5000));
progress = await getRenderProgress({ executionName: handle.executionName });
}
console.log(progress.status, progress.outputFile, progress.costs.displayCost);
```
`deploySite` is called implicitly when you pass `projectDir`; call it
yourself to pre-upload once and reuse the `siteHandle` across many renders
(e.g. personalised template batches).
## Running tests
```bash
bun test # unit tests over an in-memory GCS double — no network
bun run typecheck
```
The live end-to-end smoke (build image → terraform apply → render a fixture
through the workflow → PSNR-compare → destroy) lives at
`examples/gcp-cloud-run/scripts/smoke.sh` and needs a GCP project with
billing enabled.
## What's still ahead
- **Mid-flight per-chunk progress.** `getRenderProgress` reports coarse
`running` progress and exact numbers on success. Reading the Cloud
Workflows step-entries API would give per-chunk progress while the render
is in flight; tracked as a follow-up.
- **Cloud Run Jobs / Firebase Functions variants.** This first version
targets Cloud Run services + Workflows (the closest analog to Lambda +
Step Functions). The same handler runs unchanged under Cloud Run Jobs;
only the orchestration trigger differs.