Lab: Safe Euler Job Arrays | IDEAL IT Passport

A Slurm job array starts many similar tasks from one script. A concurrency cap limits how many of those tasks may run at the same time.

Goal: review a repeated-task script and calculate its maximum simultaneous resource use. This lab does not require submitting an array.

Prerequisites: Euler CPU Track and one successful single-task representative job.

New to text commands? Read how to open a terminal and run one command before using a command block. It explains the labels used below and what to copy.

Execution context: review locally or on an Euler login node. Do not submit the full array during the first review.

For the assessed route, follow the current mission. This is an optional review companion; its extra discussion does not add a public submission or require a live allocation.

Unsafe Starting Point#

#SBATCH --array=0-99

This creates 100 tasks with no explicit student-selected concurrency cap.

Safe Starting Point#

Compare the teaching model after reasoning about the unsafe request

#SBATCH --array=0-99%1

After one task succeeds and its measured resource use is understood, a small justified cap may be used, for example:

#SBATCH --array=0-99%2

The array still contains 100 tasks, but at most two may run at once.

That statement applies to one array. Two separate arrays capped at %2 may make four tasks eligible at once. Existing arrays and ordinary jobs must be included in the total before another submission.

Build A Reviewable Script#

Read-only illustration: the configs files and process.py below are not provided by the Passport. Do not paste this block into a terminal or submit it. For the assessed edit, use the local practice file prepared by the Cap And Calculate Euler Job Arrays mission.

#!/usr/bin/env bash
#SBATCH --job-name=passport-array
#SBATCH --account=es_fuge
#SBATCH --time=00:10:00
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=1G
#SBATCH --array=0-9%1
#SBATCH --output=logs/%x_%A_%a.out
#SBATCH --error=logs/%x_%A_%a.err

set -euo pipefail
config="configs/config_${SLURM_ARRAY_TASK_ID}.yaml"
test -f "$config"
python process.py --config "$config"

Tasks#

Explain %A, %a, and $SLURM_ARRAY_TASK_ID.
State why the first cap is %1.
Calculate concurrent CPU, memory, and GPU use for a proposed cap.
Inspect squeue -u "$USER" and calculate total eligible resources across all active arrays and ordinary jobs.
State the measurements and lab limits used to approve a higher cap.
Explain how you would cancel the whole array or one task.

Before using an array in an approved project, validate one representative input and review its accounting. Do not run the illustrative script above.

Verification#

Every array has an explicit %<cap>.
The combined cap across overlapping arrays remains within justified limits.
Log filenames distinguish parent job and task index.
One representative task passes before the full range is enabled.
Concurrent resources fit personal and collective policy.

Common Failures And Safe Recovery#

Many tasks fail identically: cancel the array and debug one task.
Cap consumes the personal GPU limit: reduce it; a limit is not a target.
Duplicate arrays submitted: cancel unwanted parent job IDs rather than waiting for every task.

Understand Before Accepting AI Output#

I calculated concurrent resources myself.
I counted my other active arrays and ordinary jobs.
I validated one representative task.
I know the difference between array size and concurrency.
I know how to cancel the submission.

Evidence#

The Passport submits its one declared fictional script and structured answers. Keep any extra calculations as private practice; no live array is required.

Ask For Help When#

Tasks depend on each other, inputs have very different resource needs, or the project requires concurrency near a lab limit.