Batch System

Submitting Batch Jobs
You can submit jobs from one of the interactive nodes or from within a running batch script. Batch jobs are submitted using standard qsub commands with a valid project account to one of the following sets of production or debug queues.  You can specify options on the command line, or (recommended) put all of them into your batch script file.  See examples below.  There is one PBS server is running on qcdpbs, and PBS commands default to using that server.
In your batch script, please specify at least the following, plus other options useful to your workflow.

account (your project short name)
queue (see below)
resources needed (number of cores, types of GPUs, etc.; see node tags below)
wall time (specifying this more tightly than the default will improve your throughput)

Production Queues:

ib    -- Infiniband cluster queue (parallel x86 jobs); Max walltime: 48 hours
gpu -- GPU cluster queue (uses multiples of 4 GPUs)  Max walltime: 48 hours
phi - Phi cluster queue; Max walltime: 48 hours

Debug queues: (these have higher priority, but shorter time limits and small simultaneous job limits)

ibdbg  -- Infiniband debug queue (Max walltime: 30 minutes)
gpudbg - GPU debug queue (Max walltime: 30 minutes)
phidbg - Phi debug queue (Max walltime: 30 minutes)

Test queues: (only used for new systems, and don't get charged to accounts; currently inactive)

testib - IB cluster test queue
testgpu - GPU cluster test queue for CentOS 6.5 nodes with CUDA 7
testphi - Phi cluster test queue

Note debug queues have a wall time limit of 30 minutes, and a limit of 5
queued jobs at a time. These queues have a higher
priority than regular jobs, and they should ONLY be used for debugging
and not for running regular jobs.