Getting started with the I2BC cluster
Objective 1
Transcribing your analysis workflow into a PBS submission script.
Using an interactive session (qsub -I) is useful when you’re setting up a new analysis pipeline and getting to know how to use the software within it. In the long run, it’s better to use a submission script instead so that you free the resources as soon as your software has finished running.
My first submission script
The PBS submission script is just a script written in bash (that’s the language of the cluster, i.e. cd and ls are bash commands). You can put in a bash script whatever you would normally write in the terminal with 1 command per line.
Let’s create a script called pbs_script.sh that will print “hello world” on the screen:
#! /bin/bash
echo "hello world"
Submitting the script
Next, we will submit the script to the scheduler, which will look for a free node to run it on.
To submit your script, you can use the same qsub command as seen previously except that we won’t need the -I option this time:
john.doe@cluster-i2bc:/home/john.doe$ qsub pbs_script.sh
918860.pbsserver
john.doe@cluster-i2bc:/home/john.doe$
Note:
- that the prefix didn’t change this time: the script will be run on a node but we’re still on the Frontale (it’s like posting a letter)
- we’re given a unique job id, just like before with the interactive session
Checking that your job worked
The command echo "hello world" should normally print “hello world” on your screen… When running scripts remotely on the nodes, anything that is usually printed on the screen is saved in two files instead: one for the standard output (o) and one for the error messages (e), if any.
Have a look at your working directory, you should see some extra files in there. If you open the standard output log (with the cat command for example), you should see “Hello world” in there.
john.doe@cluster-i2bc:/home/john.doe$ ls
pbs_script.sh pbs_script.sh.e918860 pbs_script.sh.o918860
john.doe@cluster-i2bc:/home/john.doe$ cat pbs_script.sh.o918860
Hello world
Take home message
1) on a routine basis, using a PBS submission script is better than using an interactive session (qsub -I)
2) a submission script is just a bash script & to submit it, it’s the same command as before: qsub
3) with a script, what’s normally printed on the screen will appear at the end of the job in 2 files (*.o and *.e)
4) qstat to list all jobs that are queuing/running.