Last modified on 12 May 2014, at 12:29

PBS Job Array

Revision as of 12:29, 12 May 2014 by Dong (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Here is the source code (jobarray_hello.c) for a sample serial program that takes an integer argument:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main (int argc, char *argv[])
  int id;
  char hostname[80];
  if ( argc == 2 ) {
    id = atoi(argv[1]);
    gethostname(hostname, 80);
    printf("Hello master, I am slave no. %d running on %s!\n", id, hostname);
  else {
    printf("Usage: %s id\n", argv[0]);
    printf("where id is an integer\n");

We compile it as usual using Intel C compiler:

icc jobarray_hello.c -o jobarray_hello.x

The executable takes an integer argument. For example, if we run it on the master node:

./jobarray_hello.x 15

And it says:

Hello master, I am slave no. 15 running on!

If we want to run a lot of instances of the program, with different arguments, for example:

./jobarray_hello.x 101
./jobarray_hello.x 102
./jobarray_hello.x 103
./jobarray_hello.x 104

instead of submitting 4 serial jobs, we can submit one job array (jobarray.pbs):

#PBS -N jobarray
#PBS -l ncpus=1
#PBS -t 101-104
#PBS -j oe
#PBS -l walltime=0:20:00
./jobarray_hello.x $PBS_ARRAYID

I submitted it to the gpu queue:

qsub -q gpu jobarray.pbs

When the jobs completed, I got 4 files:

jobarray.o54654-101  jobarray.o54654-102  jobarray.o54654-103  jobarray.o54654-104

each for a member of the job array. For example:

# cat jobarray.o54654-102
Hello master, I am slave no. 102 running on gpu-8.local!