Lab: Euler Resource Optimization

Goal: improve CPU, memory, and time requests using accounting evidence.

Prerequisites: First Euler Job and familiarity with sacct and seff.

Execution context: analyze the supplied fictional output locally or on an Euler login node. Do not submit the intentionally oversized example.

Scenario#

A serial preprocessing job requested:

#SBATCH --cpus-per-task=16
#SBATCH --mem-per-cpu=4G
#SBATCH --time=04:00:00

Its fictional accounting summary reports:

State       COMPLETED
Elapsed     00:11:42
AllocCPUS   16
ReqMem      4Gn
MaxRSS      3.8G
CPU Eff     6.4%

Tasks#

1. Calculate the total requested system memory.
2. Explain why low CPU efficiency is important.
3. Propose a safer next request for the same representative input.
4. State what evidence would justify increasing CPUs later.
5. State why a successful job can still be wasteful.

A reasonable next experiment might start with one CPU, total memory modestly above observed MaxRSS, and a shorter time limit with sensible headroom. The exact production request must reflect repeated representative measurements, not this fictional sample alone.

Queued-Job Input Scenario#

You submit train.slurm. It runs train.py with configs/training.yaml, then remains pending. A collaborator edits all three files before the job starts.

Answer:

1. Which version of train.slurm will Slurm run?
2. Which versions of train.py and configs/training.yaml might the job read?
3. What should the team do before submitting the next production job?

Slurm captured the submitted batch script, but it did not capture the external Python or configuration files. The safe response is to use a clean reviewed source revision in an immutable run snapshot, preserve a run-specific configuration/input snapshot, record their identifiers, and leave referenced files untouched while jobs use them. See Freeze inputs before submission.

Apply The Loop To Your Tiny Job#

Use your real first-job accounting:

Requested CPUs:
CPU efficiency:
Requested memory:
Observed MaxRSS:
Requested time:
Elapsed:
State/ExitCode:
What I would keep or change:

Do not optimize the tiny sleep/calculation as if it represented a research workload. This step proves that you can read the fields.

Verification#

The proposal reduces unsupported resources, retains justified headroom, and does not infer parallel scaling from CPU availability alone. The queued-job answer distinguishes the captured script from mutable external files.

Common Failures And Safe Recovery#

• MaxRSS blank for the top-level job: inspect the .batch step.
• Memory request interpreted incorrectly: note whether Slurm reports per-CPU or per-node memory.
• Low efficiency on a very short job: accounting overhead can dominate; measure a representative workload before production decisions.

Understand Before Accepting AI Output#

• I checked the arithmetic myself.
• I distinguished total memory from memory per CPU.
• I know one sample may not represent every input.
• I can state the evidence required for more CPUs.
• I can explain which job inputs Slurm does and does not capture.

Evidence#

Submit both scenario answers and the completed interpretation template for your tiny job.

Ask For Help When#

The workload is multi-process/MPI, memory varies greatly by input, efficiency metrics disagree, or optimization would require a large production run.