Cache manager policy

Tue, 07/06/2021 - 14:05 — amitoj

When a project is over their High Quota limit the following data backup and deletion policies will be enforced.

NOTE: You can pin files on /cache if you need to keep them available for an extended period. 

Please do not abuse the pin mechanism as /cache is a shared resource.

LQCD User's Guide

Thu, 06/24/2021 - 17:04 — amitoj

This guide is meant to cover everything related to using the LQCD Computing Facilities at Jefferson Lab, from batch systems to file storage and management.  More detailed information on the hardware available at Jefferson Lab is contained in the companion book, Scientific Computing Resources.

Create custom kernel for Jupyter Hub

Thu, 11/12/2020 - 00:31 — kishan

This is how you can setup your JLab jupyterhub notebook to run inside a python virtual environment. This is useful for installing python packages via pip from your own account without needing admin privliges (which you don't have). The following will need to be run from a terminal on a CUE computer. I recommend doing this via a terminal launched via jupyterhub since you can use a python version available there that should have all necessary system libraries installed.

Start using JLab Jupyter Hub

Thu, 11/12/2020 - 00:17 — kishan
  • To start using Jupyter hub you need a jlab CUE account with Farm access and to be added to slurm (scicomp batch system).
  • Go to https://jupyterhub.jlab.org/ and log in with your CUE username and password.
  • When you log in for the first time it will show a QR code. You need to download any one of the following applications: Google Authenticator, Microsoft Authenticator, or FreeOTP and scan that QR code to set up the future authentication.

Login to SciComp GPUs

Tue, 11/03/2020 - 23:41 — kishan

The following is how to use one of the ML scicomp machines that has 4 Titan RTX GPU cards installed. Steps:

conda activate tf-gpu

Other Intel KNL Resources

Fri, 07/31/2020 - 10:17 — chen


Slurm User Commands

Tue, 10/15/2019 - 11:02 — chen


Interactive Jobs

Tue, 09/24/2019 - 11:51 — chen
It is possible to request and use a farm node for interactive work. This allows users to do interactive work with more resources than they would have on the ifarm machines. Interactive slurm sessions can be used from the command line and will soon be used for other services such as Jupyter notebooks.
 
NOTE: While useful for testing in some circumstances, this is generally NOT what you want to do.
 
From an ifarm machine, it is possible to create a shell on a farm node like so:

GPU Jobs

Fri, 09/13/2019 - 12:56 — chen



GPUs are available on the batch farm for both interactive
and batch processing using slurm commands. GPU use from Auger is not supported.
The slurm commands (sbatch, salloc, etc) are available on the ifarm machines.

 

The following GPU resources are available:

 

Sample Scripts

Fri, 09/13/2019 - 12:43 — chen
To submit a job to slurm, one can call "sbatch" with all necessary options, or put all options into a submission-script, then call "sbatch submission-script". Here are few samples of submission-script.
 
1) A simplest one core job.
 

Pages

Powered by Drupal