File system layout

The Farm interactive and batch systems include parts of the Common User Environment (CUE), which means that you have access to the same /home  /group  /apps  /site  file systems as you do from your desktop. Documentation on these desktop accessible files systems is at CUE Directories.  Note: the /home area's backup system supports recovering files you accidentally delete.

Scientific Computing Filesystems

/scratch in scientific computing is an area on the node-local disk(s), i.e. specific to each node and not intended for sharing between systems. For sharing files and for files that persist after the job ends use the large shared /volatile scratch space (see below). Note that this area is different than the central CUE /u/scratch filesystem.

/work is a project-oriented, ZFS file system for use in the data analysis process.  It is NOT backed up! Users must explicitly jput files to tape for backup if necessary. Work areas are allocated to a group and under its control. Work areas are allocated by the computing coordinators who may shrink or grow the allocations to coincide with batch farm processing.

The Farm systems also mount two file systems that are implemented above the shared Lustre distributed file system in the data center.  These provide both high capacity and high performance to farm jobs.

/cache is a Lustre write-through cache to the tape library with auto-removal of the disk copy as needed.

/volatile is a Lustre global scratch space, and also is not backed up.  It is useful for holding batch job outputs that will be consumed by subsequent jobs, and is large enough that files can be expected to persist for weeks if the system is not abused.

The details of the management of these areas are in the following subsections.