4b6817381b
CI (OpenClaw E2E) / openclaw test (push) Has been cancelled
CI / coverage-report (push) Has been cancelled
CI / test-kubernetes (push) Has been cancelled
CI / should-run-thorough (push) Has been cancelled
CI / test-thorough (cloudwatch-demo) (push) Has been cancelled
CI / test-thorough (flink-ecs) (push) Has been cancelled
CI / test-thorough (upstream-lambda) (push) Has been cancelled
CI / test-thorough (prefect-ecs-fargate) (push) Has been cancelled
Release / build-binaries (zip, opensre.exe, onefile, windows-latest, windows-x64) (push) Has been cancelled
Benchmark image — build + push to ECR (any adapter) / build + push (push) Has been cancelled
CI / quality (ubuntu-latest) (push) Has been cancelled
CI / test (tools-runtime) (push) Has been cancelled
CI / test (e2e-general) (push) Has been cancelled
CI / test (cli-runtime) (push) Has been cancelled
CI / test (e2e-provider-and-openclaw) (push) Has been cancelled
CI / test (integrations-and-misc) (push) Has been cancelled
Release / verify (push) Has been cancelled
Release / build-python-dist (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-15-intel, darwin-x64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, macos-latest, darwin-arm64) (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04, linux-x64) (push) Has been cancelled
Release / publish-release (push) Has been cancelled
Release / publish-main-release (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-checks (no-LLM) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Interactive Shell Live (PR + post-merge) / turn-live shard ${{ matrix.shard_index }} (push) Has been cancelled
Release / prepare (push) Has been cancelled
Release / build-binaries (tar.gz, opensre, onedir, ubuntu-22.04-arm, linux-arm64) (push) Has been cancelled
Synthetic Deterministic Tests / Synthetic offline (deterministic) (push) Has been cancelled
88 lines
2.5 KiB
Plaintext
88 lines
2.5 KiB
Plaintext
---
|
||
title: "Slurm"
|
||
description: "Monitor Slurm workloads with Tracer"
|
||
---
|
||
|
||
Tracer integrates with Slurm, providing detailed observability for batch jobs and array tasks.
|
||
It works by running the Tracer agent on each compute node — no modification to job scripts required.
|
||
|
||
## Why use Tracer in combination with Slurm
|
||
|
||
Slurm gives job-level scheduling visibility, but not what happens inside the job.
|
||
Tracer adds that missing layer:
|
||
|
||
- Per-process telemetry inside each job allocation
|
||
- Job array correlation and node-level performance
|
||
- Resource and cost insights across users and queues
|
||
- Zero changes to job submission or scripts
|
||
- Real-time updates in the Tracer dashboard
|
||
|
||
## Getting Started
|
||
|
||
### Prerequisites
|
||
|
||
- Slurm cluster access with sudo or admin privileges for installation
|
||
- Tracer installed on [your operating system](/environments/overview)
|
||
|
||
### Just run your pipeline, Tracer will automatically attach
|
||
|
||
If Tracer is already installed on your operating system, you only need to enable the Tracer agent for pipelines that have not been run with Tracer before.<br/>
|
||
In that case, run the following command:
|
||
|
||
```bash
|
||
sudo tracer init --token <your-token>
|
||
```
|
||
<Info> Go to our [onboarding](https://app.tracer.cloud/dashboard) to get your own personal token</Info>
|
||
|
||
When running this command, you will be asked to name your pipeline for clear labeling in the dashboard.
|
||
|
||
## Examples
|
||
|
||
Run a Slurm pipeline under Tracer:
|
||
|
||
```bash
|
||
#!/bin/bash
|
||
#SBATCH --job-name=test
|
||
#SBATCH --cpus-per-task=8
|
||
#SBATCH --time=01:00:00
|
||
|
||
module load python
|
||
python analysis.py
|
||
```
|
||
|
||
Submit this job as usual with:
|
||
```bashsbatch
|
||
my_job.sh
|
||
```
|
||
|
||
or launch the Tracer demo workflow:
|
||
|
||
```bash
|
||
sudo tracer demo
|
||
```
|
||
|
||
Once the pipeline starts, open the Tracer dashboard, and you’ll see each Slurm job as a timeline step updating in real time.
|
||
|
||
<Card href="https://app.tracer.cloud/">
|
||
<img
|
||
className="block dark:hidden"
|
||
src="/images/logo/tracer/Tracer Full Body - Black.png"
|
||
alt="Tracer Logo"
|
||
style={{ width: '10%', height: 'auto', marginBottom: '1rem' }}
|
||
/>
|
||
<img
|
||
className="hidden dark:block"
|
||
src="/images/logo/tracer/Tracer Full Body - Black.png"
|
||
alt="Tracer Logo"
|
||
style={{ width: '10%', height: 'auto', marginBottom: '1rem' }}
|
||
/>
|
||
|
||
<div style={{ fontSize: '1.3rem', fontWeight: '700', marginBottom: '1 rem', color: 'inherit' }}>
|
||
Watch your pipeline run in the Tracer dashboard
|
||
</div>
|
||
|
||
<div style={{ color: 'inherit' }}>
|
||
View real-time metrics, resource usage, and performance insights for your pipeline runs.
|
||
</div>
|
||
</Card>
|