googleapis--mcp-toolbox/docs/en/integrations/serverless-spark/tools/serverless-spark-create-pyspark-batch.md

title, type, weight, description

title	type	weight	description
serverless-spark-create-pyspark-batch	docs	2	A "serverless-spark-create-pyspark-batch" tool submits a Spark batch to run asynchronously.

About

A serverless-spark-create-pyspark-batch tool submits a Spark batch to a Google Cloud Serverless for Apache Spark source. The workload executes asynchronously and takes around a minute to begin executing; status can be polled using the get batch tool.

serverless-spark-create-pyspark-batch accepts the following parameters:

• mainFile: The path to the main Python file, as a gs://... URI.
• args Optional. A list of arguments passed to the main file.
• version Optional. The Serverless runtime version to execute with.

Compatible Sources

{{< compatible-sources >}}

Example

kind: tool
name: serverless-spark-create-pyspark-batch
type: serverless-spark-create-pyspark-batch
source: "my-serverless-spark-source"
runtimeConfig:
  properties:
    spark.driver.memory: "1024m"
environmentConfig:
  executionConfig:
    networkUri: "my-network"

Custom Configuration

This tool supports custom runtimeConfig and environmentConfig settings, which can be specified in a tools.yaml file. These configurations are parsed as YAML and passed to the Dataproc API.

Note: If your project requires custom runtime or environment configuration, you must write a custom tools.yaml, you cannot use the serverless-spark prebuilt config.

Output Format

The response contains the operation metadata JSON object corresponding to batch operation metadata, plus additional fields consoleUrl and logsUrl where a human can go for more detailed information.

{
  "opMetadata": {
    "batch": "projects/myproject/locations/us-central1/batches/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
    "batchUuid": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
    "createTime": "2025-11-19T16:36:47.607119Z",
    "description": "Batch",
    "labels": {
      "goog-dataproc-batch-uuid": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
      "goog-dataproc-location": "us-central1"
    },
    "operationType": "BATCH",
    "warnings": [
      "No runtime version specified. Using the default runtime version."
    ]
  },
  "consoleUrl": "https://console.cloud.google.com/dataproc/batches/...",
  "logsUrl": "https://console.cloud.google.com/logs/viewer?..."
}

Reference

field	type	required	description
type	string	true	Must be "serverless-spark-create-pyspark-batch".
source	string	true	Name of the source the tool should use.
description	string	false	Description of the tool that is passed to the LLM.
runtimeConfig	map	false	Runtime config for all batches created with this tool.
environmentConfig	map	false	Environment config for all batches created with this tool.
authRequired	string[]	false	List of auth services required to invoke this tool.