Before you start#
The measurements in this lesson are invented. No GPU allocation is needed.
A GPU is an accelerator for suitable calculations. A Slurm GPU request reserves
it with supporting CPU, memory and time resources. es_fuge is the lab computing
share used for this review. A partition is a group of compute nodes; keep its
selection automatic here.
A reserved GPU can sit idle. Find what limits useful work before reserving more.
System RAM is used by the CPU-side program; GPU memory (VRAM) is on the
accelerator. --mem-per-cpu changes system RAM, not VRAM. Throughput is useful
work completed per second.
The GPU waits for input#
A fictional one-GPU run spends 60% of its time waiting for the next input batch. What would you investigate or change first, and why? The table tells you where time is spent; it does not identify the precise cause.
A hint for the input wait
Adding an accelerator helps only if it can receive useful work. Inspect the waiting stage and approved data placement before changing GPU count.
Choose supporting resources#
After the input problem is fixed, three runs use the same representative workload and configuration. Assume one task on one node and exactly the CPUs requested. A supplied CPU comparison found four useful threads. The GPU in this fictional case has 24 GiB of VRAM.
| Measurement | Runs A / B / C |
|---|---|
| Elapsed (minutes) | 12 / 13 / 14 |
| System RAM peak (GiB) | 10 / 11 / 12 |
| GPU VRAM peak (GiB) | 16 / 17 / 18 |
These are supplied application measurements, including all its processes,
not a top-level MaxRSS field. Headroom means room for modest variation above
the observed peak. Choose a next request for the same workload, not every
future dataset.
A hint for memory and time
Multiply allocated CPUs by memory per CPU. Compare that system-memory total and the time limit with the largest values in the table. GPU VRAM cannot make up for a shortage of system RAM.
Is a second GPU useful enough?#
A separate controlled comparison keeps the input, amount of work and output quality equivalent. Rates cover the entire timed run, including setup and waiting. Repeated trials give the supplied representative rates:
| GPUs reserved | Useful throughput |
|---|---|
| 1 | 100 samples per second |
| 2 | 110 samples per second |
For the same amount of work, does the small speed gain justify twice the reserved hardware? GPU reservation time is GPU count multiplied by run time.
A hint for the comparison
Speedup is the new throughput divided by the old throughput. For fixed work, compare GPU count divided by throughput. Faster completion does not necessarily mean less total GPU reservation time.
Try your own request before opening the script model in the practical step.
A starter limit is a ceiling, not a target. For real work, review representative measurements and any multi-GPU request with your supervisor.