Hint Use tags sparingly, as they tend to restrict the set of nodes eligible to run your job. You can view a node's tags by doing pbsnodes -a <nodename>.
For the phi queue, there are several useful tags for choosing nodes, depending on your use of the fast MCDRAM:
Tag | Description |
cache-quad | if not specifically addressing MCDRAM, cache |
cache-a2a | required for some multi-node jobs |
flat-quad | for code that takes advantage of MCDRAM |
For the ib queue, there are two useful tags for choosing nodes. Core performances are very similar, so it is generally only advised to use these tags to control memory per core or memory per node:
Tag | Description |
cores8 | Nodes with 8 cores, 24 GB, 3 GB/core |
cores16 | Nodes with 16 cores, 32 GB/node, 2 GB/core |
For the gpu queue, the following are the most useful tags for choosing nodes:
Tag | Description |
ecc | All GPUs with ecc memory (suitable for more than inverters) |
GTX | All gaming cards (which do not have ecc, and have smaller memories |
Fermi | All GPUs with the Fermi architecture (ecc & GTX) |
kf | All GPUs with either Fermi or Kepler architecture (excludes only the oldest GPUs) |
K20m | The newest GPU model, Kepler generation |
You can use combinations of tags, such as 4:ecc:kf to run on any Fermi or Kepler generation gaming card, which (if JLab later adds a third ecc GPU type) would allow you to exploit 2 different GPU types without having to run 2 different job types.
The following tags can be used to further restrict where your job will run. These can be useful for benchmarking specific nodes/cores/GPUs, but are discouraged for production running:
Tag | Description |
c12s | a node in the 12s cluster |
---|---|
c10g | a node in the 10g, GPU enabled cluster |
c12k | a node in the 12k, GPU enabled cluster |
GTX580 | A GPU enabled machine with an NVIDIA GTX580 GPU |
GTX690 | A GPU enabled machine with an NVIDIA GTX690 GPU |
Fermi | A GPU enabled machine with an NVIDIA Fermi generation GPU (2010) |
mic | An accelerated machine with an Intel MIC (2012) |
kf | A GPU enabled machine with an NVIDIA Kepler K20 or Fermi generation GPU (this currently covers all GPUs at JLab) |
The following tags are deprecated:
Tag | Description |
---|---|
ib | a compute node with infiniband connectivity (not very useful) |
intel | a node with an Intel cpu (all clusters at this time) |
Tesla | A GPU enabled machine with an NVIDIA Tesla GPU(same as ecc) |
k20 | A GPU enabled machine with an NVIDIA Kepler K20 GPU (2012) |