Recent Posts


Caliban is installed.

How To Run Parallel MPI Jobs Using SGE

A typical submit script would look like this (save it as hello.bash):

#$ -S /bin/bash
#$ -pe mpi 20
#$ -cwd
#$ -o /home/nome_utente/file.out
#$ -l h=compute-x-y.local
/opt/openmpi/bin/mpirun -np $NSLOTS -machinefile $TMPDIR/machines

Remember to do a source of your bash_rc a if you want export your personal configurations

SGE Options

-cwd Run in the current working directory
-j y Stdout and stderr in same output file
-S /bin/bash Use the bash shell
-pe mpi xy Use parallel environment named
“mpi” for xy MPI processes
-l h=name Select host ( where name=compute-x-y.local)
-o filename Output file for stdout
-e filename File for stderr
-q filename Select queue
-l excl true The job is performed exclusively
on a single node

Submitting a parallel job:

qsub -q queue hello.bash

where queue can be :

debug.q nodes from 0-3 to 2-5 (except 1-6) priority 10 wall time 30 min

parallel.q nodes from 0-3 to 2-5 (except 1-6) priority 0 wall time 24 h

long.q nodes from 0-3 to 1-5 priority -10 wall time 2 weeks
(Limited to users in the access list)

smallparallel.q for small parallel jobs (max 8 procs) nodes
from 3-0 to 3-3 priority 0 wall time 24 h

For GPU job use node compute-1-6 and compute 0-2
without sge job scheduler.

Check job status:


Delete a job:

qdel id-job

Check your jobs also under cluster status => Job queue

Filed under: Cluster Links | 3 Comments »

Cluster Status

Click here to monitor your cluster


Department of Information Engineering, Computer Science and Mathematics


Caliban cluster performance from “The National Science Foundation”

Roll Docs

Documentation for the installed rolls


Get help at the Rocks Web Site


View the latest Tripwire reports

Users Guide

Info about Rocks Clusters