HPC kitchen [slurm]: the Slurm job scheduler divides tasks to the cluster скачать в хорошем качестве

HPC kitchen [slurm]: the Slurm job scheduler divides tasks to the cluster

We have talked about running stuff in parallel, but how does it get connected to hardware (processors, memory)? If we have one cluster for everyone, how do people share? That's what this is all about. Extra commentary / corrections: Slurm is the system that allocates resources, but you don't need to know details: for basic use, follow your local instructions. For the most part, you will see the instructions for your own cluster, which will say what options are needed. This is the point where you really need command line and shell scripting, because you have to automate all of the steps you use. 00:06 Buy everyone a big kitchen, or share one really nice one? 45:00 Slurm is the resource allocator 01:11 Batch jobs 01:44 Reserveable resources: (CPUs, memory, GPUs, etc.) × time 02:05 Jargon 02:30 First: priority calculation 02:50 Second: forward allocate in priority order 03:30 Third: Backfilling 04:13 Heterogenous (non-uniform resource use) jobs 05:13 Big jobs are hard to schedule 05:38 Array jobs 07:18 When does my job run? 08:09 Summary: you become a manager and need to handle all of these resource requests. Playlist:    • SciComp Kickstart prep: HPC kitchen