cancel


add-jsub


add-job


abandon-jobs


Command-Line Overview


Using MCDRAM



Using MCDRAM High Bandwidth Memory on the Knights Landing Nodes

Use Cache Mode

The simplest way to exploit MCDRAM is to use it as a cache. This requires the nodes to be
booted into Cache mode, or jobs to be submitted to jobs with the tag cache-quad.
No code changes are needed for this mode of operation, and it is probably the best place
to start for applications which do not fit entirely into the 16GB of MCDRAM per node

Intel OpenMP support


Intel MPI on the Knights Landing Cluster



Intel MPI on the Knights Landing Cluster

In this chapter we note the main points about running Intel MPI on the Knights Landing cluster.

MPI Setup

It is straightforward to set up MPI by source-ing the psxevars.sh script as detailed in the Compilers and Tools section.

Running with mpirun

The simplest way to run is to use the mpirun command. A list of useful switches is provided below:

Running Knights Landing Jobs



Setting up Shared Libraries

Ensure that shared libraries for OpenMP, MPI, C/C++ etc are set up as described in the Compilers and Tools section

Affinity, Binding and MPI Pinning

Appropriate thread affinity, and MPI process pinning is important for high performance. These factors are often intertwined. We will consider first OpenMP thread affinity (as if we were running a non-MPI job) and then consider MPI.

Submitting Jobs to the Knights Landing Cluster



Submitting Jobs to the Knights Landing Cluster

Submitting jobs to the Knights Landing cluster is similar to submitting to any other cluster at JLab. Currently one must submit to the phi queue:

$ qsub -q phi job_script.sh

Job Size

Request the number of nodes needed (note that originally number of cores was specified in the early science running period, but was changed for production running):

Pages