wehub-resource-sync
4b6817381b
CI (OpenClaw E2E) / openclaw test (push) Has been cancelled
CI / coverage-report (push) Has been cancelled
CI / test-kubernetes (push) Has been cancelled
CI / should-run-thorough (push) Has been cancelled
CI / test-thorough (cloudwatch-demo) (push) Has been cancelled
CI / test-thorough (flink-ecs) (push) Has been cancelled
CI / test-thorough (upstream-lambda) (push) Has been cancelled
CI / test-thorough (prefect-ecs-fargate) (push) Has been cancelled
Release / build-binaries (zip, opensre.exe, onefile, windows-latest, windows-x64) (push) Has been cancelled
Benchmark image — build + push to ECR (any adapter) / build + push (push) Has been cancelled
CI / quality (ubuntu-latest) (push) Has been cancelled
CI / test (tools-runtime) (push) Has been cancelled
CI / test (e2e-general) (push) Has been cancelled
CI / test (cli-runtime) (push) Has been cancelled
CI / test (e2e-provider-and-openclaw) (push) Has been cancelled
CI / test (integrations-and-misc) (push) Has been cancelled
Release / verify (push) Has been cancelled
Release / build-python-dist (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-15-intel, darwin-x64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-latest, darwin-arm64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04, linux-x64) (push) Has been cancelled
Release / publish-release (push) Has been cancelled
Release / publish-main-release (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-checks (no-LLM) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-live shard ${{ matrix.shard_index }} (push) Has been cancelled
Release / prepare (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04-arm, linux-arm64) (push) Has been cancelled
Synthetic Deterministic Tests / Synthetic offline (deterministic) (push) Has been cancelled
87 lines
2.7 KiB
Plaintext
87 lines
2.7 KiB
Plaintext
---
|
|
title: "Google Batch"
|
|
description: "Install Tracer on Google Cloud Batch under two minutes"
|
|
---
|
|
|
|
## Overview
|
|
|
|
Google Cloud Batch is a fully managed batch processing service that enables you to run large-scale batch jobs on Google Cloud. Tracer integrates with Google Cloud Batch to provide real-time observability for your batch workloads.
|
|
|
|
## Prerequisites
|
|
|
|
- Google Cloud account with Batch API enabled
|
|
- `gcloud` CLI installed and configured
|
|
- Tracer account and API token
|
|
|
|
## Getting Started
|
|
|
|
#### 1. Create a Machine Image With Tracer Installed
|
|
|
|
<Steps>
|
|
<Step title="Create a new VM instance in Google Cloud." />
|
|
<Step title="Install Tracer on the instance." />
|
|
<Step title="Configure Tracer to run as a service so it automatically starts when the system boots." />
|
|
</Steps>
|
|
|
|
Once everything is correctly installed and running, create a Machine Image from this VM.
|
|
This Machine Image will serve as the base template for future instances.
|
|
|
|
#### 2. Set the Machine Image as the Default for New Instances
|
|
|
|
<Steps>
|
|
<Step title="In your Google Batch configuration, set the Machine Image created above to be the default boot image." />
|
|
<Step title="This makes sure that every time a new Batch machine is spawned, it loads the image with Tracer pre-installed and running." />
|
|
</Steps>
|
|
|
|
In this way, all new instances will automatically have Tracer active without any manual setup.
|
|
|
|
#### 3. Update Your nextflow.config to Work With Tracer
|
|
|
|
<Steps>
|
|
<Step title="Improve your nextflow.config so that it integrates correctly with Tracer in Google Batch environments." />
|
|
<Step title="You can base the necessary configuration changes on the example below:" />
|
|
</Steps>
|
|
|
|
```groovy
|
|
params {
|
|
customUUID = java.util.UUID.randomUUID().toString()
|
|
// GCP bucket for work directory - make configurable
|
|
gcpWorkBucket = 'tracer-nextflow-work'
|
|
}
|
|
|
|
workDir = "gs://${params.gcpWorkBucket}/work"
|
|
|
|
process {
|
|
executor = 'google-batch'
|
|
machineType = 'template://my-instance-template'
|
|
|
|
// Set env vars for the containers
|
|
containerOptions = [
|
|
environment: [
|
|
'TRACER_TRACE_ID': "${params.customUUID}"
|
|
]
|
|
]
|
|
|
|
env.TRACER_TRACE_ID = params.customUUID
|
|
|
|
errorStrategy = 'retry'
|
|
maxRetries = 2
|
|
|
|
// Resource labels for Google Batch
|
|
resourceLabels = [
|
|
'launch-time': new java.text.SimpleDateFormat("yyyy-MM-dd_HH-mm-ss").format(new Date()),
|
|
'custom-session-uuid': "${params.customUUID}",
|
|
'project': 'tracer-467514'
|
|
]
|
|
}
|
|
|
|
// GCP Batch/credentials configuration (optional)
|
|
google {
|
|
project = 'tracer-467514'
|
|
location = 'us-central1'
|
|
serviceAccountEmail = 'nextflow-batch@tracer-467514.iam.gserviceaccount.com'
|
|
}
|
|
```
|
|
|
|
Link: https://github.com/Tracer-Cloud/nextflow-test-pipelines/pull/84/files
|