Job Input, Output and Working Directory

When a user submit a job, he/she can specify what directory the job will run on the allocated execution host. The Slurm option is:
  • #SBATCH --chdir=particularDirectory
If the specified directory is not available on the host, the job will be executed in the /tmp directory.

If the above option is not specified or the specified directory is residing in the NFS home directory,  the submitted job will be executed in the directory /scratch/slurm/username where username is the name of the user who submits the job. There is an environment variable JLAB_SLURM_O_WORKDIR that points to the working directory of the running job. The created files of a job in the working directory will be removed after several days upon the completion of the job.

In comparison to using Auger which manages input/output files for submitted jobs, users have to manage the Input/Output files of submitted jobs within the jobs or before and after the job. In the future we may provide similar capabilities as what Auger is doing at this moment.

For the input files of a job, the user needs to jcache the files first (if they are not on the cache disk). When the job starts, the user can copy the files to the computing nodes under the directory /scratch/slurm/username if the files are small files and will be access randomly during the execution period of the job. On the other hand, if the files are very large files and can be accessed efficiently (e.g. read/write sequentially in large I/O blocks) from the Lustre file system, the user can leave the input files on the cache disk and access the files within the job using fullpath names inside the cache disk system.

For the output files generated from a job, the user needs to copy the files to the proper locations such as cache, volatile or work. If the files need to be copied to the tape library, just copy the files to the correct location in /cache system and the files will eventually be copied to the tape library. However, if one wishes to copy an output file to the tape library immediately, one can issue "jcache put" after the file is copied to the proper locations.