Computational Cluster Programs

Data Storage on the Hoffman2 Cluster

Your Home Directory

All user home directories are physically located on the BlueArc Storage Server.

Users of the Campus General Purpose Cluster:

Each user of the Campus General Purpose Cluster has a home directory which is accessable from the login nodes and from each of the compute nodes. The home directory is named:

/u/home/campus/loginid

You can get to your home directory by entering the command:

cd

without any arguments.

Users of the Campus General Purpose Cluster have a 20GB disk space quota on their home directories. Use the:

myquota -pm

command to determine how much of your quota is remaining.

Or, to see your current usage, from your home directory, enter:

du -sh

Users who have purchased storage from our Shared Storage System:

Research groups participating in the Shared Storage program, where storage beyond the standard 20GB can be purchased, are subject to group quotas on the BlueArc Storage Server. Use the:

myquota -pm

command to determine how much of your groups quota is remaining. A group's quota will grow whenever the group purchases additional disk storage. Individual quotas are not used unless directed so by the research group's PI.

If you belong to a research group that participates in the Shared Storage System, your home directory is named:

/u/home/research_group/loginid

For more information about the Shared Storage System please see IDRE Shared HPC Storage Program.

All home directories are periodically backed up. In case of catastrophic failure on the BlueArc, all data in home directories will be restored to the way they were when the last backup was taken.

The purpose of the backup is not to be able to restore individual files that you accidentally loose. There are measures that you can take to protect your files from accidental loss.

File Systems for Temporary Use

$TMPDIR and /work

Each login node and compute node has a local scratch file system which at minimum is 100 GB. There is a different local scratch file system on each node.

• Local storage on compute nodes is intended to be used for life-of-job storage. The job scheduler will automatically delete $TMPDIR at the end of your job. Therefore you need to copy important files to permanent space before the end of your job or session. There is a different $TMPDIR on each node. Example: $TMPDIR/myfile
• Local storage on login nodes can be used for high activity files to avoid the overhead and network traffic associated with the network file systems. To place files in /work, make a directory there named with your login id and place the files in it. Files in /work more than 24 hours old become eligible for automatic deletion.

$SCRATCH

This file system is available on all nodes of the Hoffman2 cluster. There is a 3TB per user limit on the scratch filesystem. Additional space is available upon request to atshpc@ucla.edu

• Because scratch resides on high-performance storage, it is recommended that for performance reasons, parallel jobs, especially those with high I/O requirements, write to scratch instead of to home directories.
• To place files in $SCRATCH have your jobs scripts or command files write to $SCRATCH/filename
• Permissions on your $SCRATCH directory by default make it world-readable. Use the chmod command to change its permissions: chmod 700 $SCRATCH
• Under normal circumstances, files you store in scratch are allowed to remain there for 7 days. Any files older than 7 days can automatically be deleted by the system to guarantee that enough space exists for the creation of new files. However, there may be occasions when even though all files older than 7 days have been deleted, there is still insufficient free space remaining. Under that circumstance and that circumstance only, some of the files belonging to those users who are using the preponderance of the disk space in scratch will be deleted even though they have not been there for 7 days.

 

March 2012