Filter
Top Products
B – Integration with a Batch Queuing System
A Batch Queuing Script
B-2 IB6054601-00 D
Q
require that his node program be the only application running on each node CPU.
In a typical batch environment, the MPI user would still specify the number of node
programs, but would depend on the batch system to allocate specific nodes when
the required number of CPUs becomes available. Thus,
batch_mpirun would take
at least an argument specifying the number of node programs and an argument
specifying the MPI program to be instantiated. For example,
$ batch_mpirun -np n my_mpi_program
After parsing the command line arguments, the next step of batch_mpirun would
be to request an allocation of
n processors from the batch system. In SLURM, this
would use the command
eval ‘srun --allocate --ntasks=$np --no-shell‘
Make sure to use back-quotes rather than normal single-quotes. $np is the shell
variable that your script has set from the parsing of its command line options. The
--no-shell option to srun prevents SLURM from starting a subshell. The srun
command is run with
eval in order to set the SLURM_JOBID shell variable from the
output of the
srun command.
With these specified arguments, the SLURM function
srun blocks until there are
$np processors available to commit to the caller. When the requested resources
are available, this command opens a new shell and allocates the requested number
of processors to it.
B.1.2
Generating the mpihosts File
Once the batch system has allocated the required resources, your script must
generate a
mpihosts file, which contains a list of nodes that will be used. To do this,
it must find out which nodes the batch system has allocated, and how many
processes we can start on each node. This is the part of the script
batch_mpirun
that performs these tasks:
mpihosts_file=‘mktemp -p /tmp mpihosts_file.XXXXXX‘
srun --jobid=${SLURM_JOBID} hostname -s | sort | uniq -c \
| awk ’{printf "%s:%s\n", $2, $1}’ > $mpihosts_file
The first command creates a temporary hosts file with a random name, and assigns
the name to the variable
mpihosts file it has generated.
The next instance of the SLURM
srun command runs hostname -s once per
process slot that SLURM has allocated to us. If SLURM has allocated two slots on
one node, we thus get the output of
hostname -s twice for that node.
The
sort | uniq -c component tells us the number of times each unique line was
printed. The
awk command converts the result into the mpihosts file format used