massive-website-banner

File System and Disk Quotas

File System

The File System is arranged in 3 parts, your home directory, your project directory and project scratch space. In your home directory you should find a number of symbolic links (ls -al). These point to the project space and scratch you have been allocated, you can request more space later via help@massive.org.au

For example if your project name is "YourProject001" you will see the following two links:

ls -al ~/

YourProject001 -> /home/projects/YourProject001

YourProject001_scratch -> /scratch/YourProject001

 

The first one is for your project data that will be backed up daily (as with your home directory).

The second one is the faster system scratch space and will not be backed up and is used for temporary data.

What to put on each file system?

That is up to you, but as a general guide

Home directory (~2GB)

This should contain all of your hidden files and configuration files. Things like personal settings for editors and other programs can go here.

Project directory (~1TB+ shared with everyone on your project)

This area is backed up but limited in space. It should contain all of your input data, a copy of all your job scripts and final output data. It might be worth while to keep copies of any spreadsheets you might use to analyse results or any matlab or mathematica scripts/programs here as well. Basically anything that would be hard to regenerate.

Generally each user in the project should create a subdirectory in the project folder for themselves. 

Scratch directory (10's of TBs)

This area is not backed up. Generally all your intermediate data will go here. Anything that you can recreate by submitting a job script and waiting (even if its for quite a long time) can go here. Anything that you can't regnerate automatically, things that you have to think about and create rather than asking the computer to calculate should go in the project directory because that is backed up.

You should delete all data that is no longer needed to free up space for other projects.

Disk Quotas

MASSIVE uses "soft" and "hard" quota limits.

A "soft" limit allows you to have more than your allocated space for a short period of "grace" time (in days). After the grace time has been exceeded, the filesystem will prevent further files being added until the excess is removed.

A "hard" limit prevents further files being added. 

The quotas on the Project directories are much larger than the space users get in their own home directories, so it is much better to use the Project space. Also the project space is available for all members of your project, so you can use it to share data with your colleagues.

Currently the default quotas are:

home directory = 2GB (soft limit with 7 days grace, 10GB hard limit) 

project directory = 1TB (soft limit with 7 days grace, 1100GB hard limit) 

scratch = 2TB (soft limit with 30 days grace, 20T hard limit)

Scratch Usage Policies

Demand for scratch space is high so the following policies are now enforce to ensure fair access to this high performance resource.

  1. Scratch space is only to be used for data that is activily being processed
  2. The system quota (above) will allow data up to 20T to be stored for up to 30 days before new data will be prevented from being created
  3. Projects that have not submitted jobs in the last 3 months will be requested to delete all scratch data

Due the specialist facility nature of MASSIVE, exceptions to the above policies can be catered for and can be requested via help@massive.org.au. We can also help make archival storage (such as RDS) available on MASSIVE for integrating into your workflows directly.

Over Quota

If you are over quota you will see the following message:

*********************************************************************************************************
You are over your quota!

Filesystem     type     blocks     quota     limit     in_doubt     graceM2Home         FILESET  3G         2G        10G       0            6daysFor more information on the File System and Disk Quotas see:http://www.massive.org.au/userguide/getting-started/file-system-and-disk-quotas

*********************************************************************************************************

The above states that you are using 3G of space (blocks) of your 2G allocated quota (i.e. you are over quota by 1G). You are still able to work up to 10G (hard limit) however this is just to allow you to login to shift data to the project directory. After the grace period you will not be able create more files and this will impact your ability to do anything that requires a file in your home dir.

You will be required to remove files or shift them to your project directory.

Tools for Helping Manage Files

To quickly see your current usage use the following command:

user_info

To find and delete old data you can use the ncdu disk usage utility. The -x flag will restrict the search to the current fileset. This will list all directories by size and give you the option to delete.

ncdu -x <project or project_scratch folder>

Desktop users can access the above through the following menu path:

Applications > MASSIVE > MASSIVE Desktop Utilities > Check Disk Space Useage of Home Dir

To transfer files off MASSIVE see Transferring Files

System Backups and File Recovery

The data storage on MASSIVE is RAID 6, which distributes data across a number of disks and provides mechanisim to manage disk failures. This means that the MASSIVE file system is fault tolerant and provides a high level of data protection.

In addition to this, the project and home directories are backed up daily, which means that if you create a file on Tuesday, the following day there will be a copy of the file stored in the backup system. If a file is deleted, the backup system will store the file for approximately one month, before the system permanently removes the file to make space for new data.

If you delete a file/directory by mistake, you will be able to recover the file through a request to help@massive.org.au. To recover files we require the full path to the data, give info on when it was last seen and when it was deleted. If the file has been backed up and a restore requested before it has been permanently removed, we will be able to recover the file for you which may take a few days.

The project scratch space is not backed up. Files are purged from this space as new space is required.

 

Information for Desktop Users

Desktop users should be aware that many application and Desktop defaults dump data to your home directory. Care must be taken when dealing with large files as these can create large amounts of hidden data and that can make your home directory go over quota. You can select Applications > MASSIVE > MASSIVE Destop Utilities > Check Diskspace Usage of Home Dir  to find where space is being consumed in your home folder

The following is some information for solving common issues:

Thumbnails Generating Too Much Data

The act of viewing large amounts of images in a file browser cause the generation of many Gigabytes of thumbnail images. To fix:

  • Go to Applications - System Tools - File Browser
  • At your File Browser, Go to Edit - Preference
  • At preference, Go to Preview
  • At Other Previewable files - Show thumbnails - Change to 'Never'
  • Ok .
Remember to empty your trash folder

Some users still encountered dsk quota full messages when they have already many files or housekeep their Home directorries.

  • The trash folder's quota falls under an user's home directory
  • Do clear your trash folder when you wanted to exit your MASSIVE Desktop Session
Already over quota?

If you are over quota, and cannot login via the desktop, you can login using a login shell and use the commands described above in "Tools for Helping Manage Files". Please do not hesitate to email help@massive.org.au if this is not possible (e.g. for Beamline accounts) or if you need assistance.

Copyright © 2016 MASSIVE. All Rights Reserved.