chore: import upstream snapshot with attribution
regression / regression-shards (style-16-prod style-9-prod style-17-prod iframe-render-compat variables-prod mp4-h265-sdr, shard-4) (push) Has been cancelled
regression / regression-shards (style-4-prod style-11-prod style-2-prod animejs-adapter typegpu-adapter parallel-capture-regression, shard-5) (push) Has been cancelled
regression / regression-shards (style-7-prod style-8-prod style-10-prod css-spinner-render-compat webm-transparency mp4-h264-sdr webm-vp9, shard-3) (push) Has been cancelled
regression / regression-shards (sub-composition-video style-18-prod raf-ball-render-compat font-variant-numeric sub-comp-t0 sub-comp-id-selector, shard-7) (push) Has been cancelled
Windows render verification / Detect changes (push) Has been cancelled
Windows render verification / Preflight (lint + format) (push) Has been cancelled
Windows render verification / Render on windows-latest (push) Has been cancelled
Windows render verification / Tests on windows-latest (push) Has been cancelled
CI / Detect changes (push) Has been cancelled
CI / Build (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Fallow audit (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Typecheck (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Producer: integration tests (push) Has been cancelled
CI / Producer: unit tests (push) Has been cancelled
CI / File size check (push) Has been cancelled
CI / Test: skills (push) Has been cancelled
CI / Skills: manifest in sync (push) Has been cancelled
CI / CLI: npx shim (macos-latest) (push) Has been cancelled
CI / CLI: npx shim (ubuntu-latest) (push) Has been cancelled
CI / CLI: npx shim (windows-latest) (push) Has been cancelled
CI / SDK: unit + contract + smoke (push) Has been cancelled
CI / Test: runtime contract (push) Has been cancelled
CI / Studio: load smoke (push) Has been cancelled
CI / Smoke: global install (push) Has been cancelled
CI / CLI smoke (required) (push) Has been cancelled
CI / Semantic PR title (push) Has been cancelled
Player perf / Detect changes (push) Has been cancelled
Player perf / Preflight (lint + format) (push) Has been cancelled
Player perf / player-perf (push) Has been cancelled
Player perf / Perf: drift (push) Has been cancelled
Player perf / Perf: fps (push) Has been cancelled
Player perf / Perf: parity (push) Has been cancelled
Player perf / Perf: scrub (push) Has been cancelled
Player perf / Perf: load (push) Has been cancelled
preview-regression / Detect changes (push) Has been cancelled
preview-regression / Preflight (lint + format) (push) Has been cancelled
preview-regression / Preview parity (push) Has been cancelled
preview-regression / preview-regression (push) Has been cancelled
regression / regression (push) Has been cancelled
regression / Detect changes (push) Has been cancelled
regression / Preflight (lint + format) (push) Has been cancelled
regression / regression-shards (hdr-regression style-5-prod style-3-prod mov-prores, shard-1) (push) Has been cancelled
regression / regression-shards (overlay-montage-prod style-12-prod chat missing-host-comp-id png-sequence portrait-edge-bleed, shard-6) (push) Has been cancelled
regression / regression-shards (style-13-prod style-6-prod vignelli-stacking gsap-letters-render-compat audio-mux-parity, shard-8) (push) Has been cancelled
regression / regression-shards (style-15-prod hdr-hlg-regression style-1-prod many-cuts vfr-screen-recording render-symlinked-assets, shard-2) (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Docs / Validate docs (push) Has been cancelled
Sync skills to ClawHub / Publish changed skills (push) Has been cancelled
regression / regression-shards (style-16-prod style-9-prod style-17-prod iframe-render-compat variables-prod mp4-h265-sdr, shard-4) (push) Has been cancelled
regression / regression-shards (style-4-prod style-11-prod style-2-prod animejs-adapter typegpu-adapter parallel-capture-regression, shard-5) (push) Has been cancelled
regression / regression-shards (style-7-prod style-8-prod style-10-prod css-spinner-render-compat webm-transparency mp4-h264-sdr webm-vp9, shard-3) (push) Has been cancelled
regression / regression-shards (sub-composition-video style-18-prod raf-ball-render-compat font-variant-numeric sub-comp-t0 sub-comp-id-selector, shard-7) (push) Has been cancelled
Windows render verification / Detect changes (push) Has been cancelled
Windows render verification / Preflight (lint + format) (push) Has been cancelled
Windows render verification / Render on windows-latest (push) Has been cancelled
Windows render verification / Tests on windows-latest (push) Has been cancelled
CI / Detect changes (push) Has been cancelled
CI / Build (push) Has been cancelled
CI / Lint (push) Has been cancelled
CI / Fallow audit (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Typecheck (push) Has been cancelled
CI / Test (push) Has been cancelled
CI / Producer: integration tests (push) Has been cancelled
CI / Producer: unit tests (push) Has been cancelled
CI / File size check (push) Has been cancelled
CI / Test: skills (push) Has been cancelled
CI / Skills: manifest in sync (push) Has been cancelled
CI / CLI: npx shim (macos-latest) (push) Has been cancelled
CI / CLI: npx shim (ubuntu-latest) (push) Has been cancelled
CI / CLI: npx shim (windows-latest) (push) Has been cancelled
CI / SDK: unit + contract + smoke (push) Has been cancelled
CI / Test: runtime contract (push) Has been cancelled
CI / Studio: load smoke (push) Has been cancelled
CI / Smoke: global install (push) Has been cancelled
CI / CLI smoke (required) (push) Has been cancelled
CI / Semantic PR title (push) Has been cancelled
Player perf / Detect changes (push) Has been cancelled
Player perf / Preflight (lint + format) (push) Has been cancelled
Player perf / player-perf (push) Has been cancelled
Player perf / Perf: drift (push) Has been cancelled
Player perf / Perf: fps (push) Has been cancelled
Player perf / Perf: parity (push) Has been cancelled
Player perf / Perf: scrub (push) Has been cancelled
Player perf / Perf: load (push) Has been cancelled
preview-regression / Detect changes (push) Has been cancelled
preview-regression / Preflight (lint + format) (push) Has been cancelled
preview-regression / Preview parity (push) Has been cancelled
preview-regression / preview-regression (push) Has been cancelled
regression / regression (push) Has been cancelled
regression / Detect changes (push) Has been cancelled
regression / Preflight (lint + format) (push) Has been cancelled
regression / regression-shards (hdr-regression style-5-prod style-3-prod mov-prores, shard-1) (push) Has been cancelled
regression / regression-shards (overlay-montage-prod style-12-prod chat missing-host-comp-id png-sequence portrait-edge-bleed, shard-6) (push) Has been cancelled
regression / regression-shards (style-13-prod style-6-prod vignelli-stacking gsap-letters-render-compat audio-mux-parity, shard-8) (push) Has been cancelled
regression / regression-shards (style-15-prod hdr-hlg-regression style-1-prod many-cuts vfr-screen-recording render-symlinked-assets, shard-2) (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Docs / Validate docs (push) Has been cancelled
Sync skills to ClawHub / Publish changed skills (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,128 @@
|
||||
# @hyperframes/gcp-cloud-run
|
||||
|
||||
Google Cloud Run + Cloud Workflows adapter for HyperFrames distributed
|
||||
rendering. The OSS render primitives (`plan` → `renderChunk` × N →
|
||||
`assemble`) are pure functions over local file paths; this package is the
|
||||
deployment, orchestration, and storage glue that runs them on Google Cloud —
|
||||
the GCP counterpart to [`@hyperframes/aws-lambda`](../aws-lambda).
|
||||
|
||||
Two surfaces, one package:
|
||||
|
||||
- **Server-side handler** (`./server`) — a Cloud Run HTTP service that
|
||||
dispatches `plan` / `renderChunk` / `assemble` on the request body's
|
||||
`Action` field, bridging GCS ↔ the container's filesystem around each OSS
|
||||
primitive. This is what the bundled `Dockerfile` runs.
|
||||
- **Client-side SDK** (`./sdk`) — `renderToCloudRun`, `getRenderProgress`,
|
||||
`deploySite`, `validateDistributedRenderConfig`, and `computeRenderCost`.
|
||||
Call these from a Node process (CI, CLI, app backend) to drive a deployed
|
||||
stack without writing GCS / Workflows boilerplate.
|
||||
|
||||
The package is **not** a dependency of `@hyperframes/producer`; install it
|
||||
separately.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
GCS bucket ←→ Cloud Run service (plan / renderChunk / assemble)
|
||||
▲
|
||||
│ OIDC-authenticated http.post, one per step
|
||||
│
|
||||
Cloud Workflows (Plan → parallel RenderChunk → Assemble)
|
||||
```
|
||||
|
||||
- **Plan** downloads the project tarball, runs `plan()`, uploads the planDir
|
||||
tarball (+ audio) to GCS, and returns the chunk count.
|
||||
- **RenderChunk** runs in a parallel `for` loop in the workflow, fanned out
|
||||
up to the plan's chunk count. Each invocation renders one chunk and uploads
|
||||
it.
|
||||
- **Assemble** downloads every chunk + audio, stitches the final
|
||||
deliverable, and uploads it.
|
||||
|
||||
Every step is a `POST` to the same Cloud Run URL with a different `Action`.
|
||||
The workflow accumulates each step's small result body and returns
|
||||
`{ Plan, Chunks, Assemble }` so `getRenderProgress` can read frame totals and
|
||||
per-step durations on success.
|
||||
|
||||
## Chrome runtime
|
||||
|
||||
Unlike the Lambda adapter — which fights a 250 MB ZIP ceiling and
|
||||
decompresses `@sparticuz/chromium` into `/tmp` at runtime — Cloud Run runs a
|
||||
container image. The `Dockerfile` installs the same pinned
|
||||
`chrome-headless-shell` build and font set the production renderer uses, at a
|
||||
fixed path, and exports `HYPERFRAMES_CHROME_PATH`. CDP-level `BeginFrame`
|
||||
works because the command lives in the protocol, not the binary. There is no
|
||||
runtime decompression step and no packaging ceiling.
|
||||
|
||||
## Deploying
|
||||
|
||||
The `terraform/` module provisions everything: the GCS render bucket, the
|
||||
Cloud Run service, the Cloud Workflows definition, two least-privilege
|
||||
service accounts (the service reads/writes the bucket; the workflow invokes
|
||||
the service), and a runaway-request alert.
|
||||
|
||||
```bash
|
||||
# 1. Build + push the image (Cloud Build or local docker).
|
||||
gcloud builds submit . \
|
||||
--tag REGION-docker.pkg.dev/PROJECT/REPO/hyperframes-render:TAG
|
||||
|
||||
# 2. Apply the module.
|
||||
terraform -chdir=node_modules/@hyperframes/gcp-cloud-run/terraform init
|
||||
terraform -chdir=node_modules/@hyperframes/gcp-cloud-run/terraform apply \
|
||||
-var project_id=PROJECT \
|
||||
-var region=us-central1 \
|
||||
-var image=REGION-docker.pkg.dev/PROJECT/REPO/hyperframes-render:TAG
|
||||
```
|
||||
|
||||
Terraform outputs `render_bucket_name`, `service_url`, `workflow_name`, and
|
||||
`region` — pass them straight into the SDK.
|
||||
|
||||
## Using the SDK
|
||||
|
||||
```ts
|
||||
import { renderToCloudRun, getRenderProgress } from "@hyperframes/gcp-cloud-run/sdk";
|
||||
|
||||
const handle = await renderToCloudRun({
|
||||
projectDir: "./my-composition",
|
||||
config: { fps: 30, width: 1920, height: 1080, format: "mp4" },
|
||||
bucketName: "hyperframes-render-my-project", // from terraform output
|
||||
projectId: "my-project",
|
||||
location: "us-central1",
|
||||
workflowId: "hyperframes-render",
|
||||
serviceUrl: "https://hyperframes-render-abc.us-central1.run.app",
|
||||
});
|
||||
|
||||
// Poll until done.
|
||||
let progress = await getRenderProgress({ executionName: handle.executionName });
|
||||
while (progress.status === "running") {
|
||||
await new Promise((r) => setTimeout(r, 5000));
|
||||
progress = await getRenderProgress({ executionName: handle.executionName });
|
||||
}
|
||||
console.log(progress.status, progress.outputFile, progress.costs.displayCost);
|
||||
```
|
||||
|
||||
`deploySite` is called implicitly when you pass `projectDir`; call it
|
||||
yourself to pre-upload once and reuse the `siteHandle` across many renders
|
||||
(e.g. personalised template batches).
|
||||
|
||||
## Running tests
|
||||
|
||||
```bash
|
||||
bun test # unit tests over an in-memory GCS double — no network
|
||||
bun run typecheck
|
||||
```
|
||||
|
||||
The live end-to-end smoke (build image → terraform apply → render a fixture
|
||||
through the workflow → PSNR-compare → destroy) lives at
|
||||
`examples/gcp-cloud-run/scripts/smoke.sh` and needs a GCP project with
|
||||
billing enabled.
|
||||
|
||||
## What's still ahead
|
||||
|
||||
- **Mid-flight per-chunk progress.** `getRenderProgress` reports coarse
|
||||
`running` progress and exact numbers on success. Reading the Cloud
|
||||
Workflows step-entries API would give per-chunk progress while the render
|
||||
is in flight; tracked as a follow-up.
|
||||
- **Cloud Run Jobs / Firebase Functions variants.** This first version
|
||||
targets Cloud Run services + Workflows (the closest analog to Lambda +
|
||||
Step Functions). The same handler runs unchanged under Cloud Run Jobs;
|
||||
only the orchestration trigger differs.
|
||||
Reference in New Issue
Block a user