Checking your quota, requesting quota increases and dealing with large Conda environments

Introduction

Use the checkquota command to check your current storage and your storage limits. Here's an example:

$ checkquota
          Storage/size quota filesystem report for user: ceisgrub
Filesystem            Mount              Used   Limit  MaxLim Comment
Tiger2 home           /home            13.9GB  18.6GB    20GB 
Tiger2 scratch GPFS   /scratch/gpfs     9.2GB   500GB   512GB 
Tigress GPFS          /tigress        397.9MB   500GB   512GB  
          Storage number of files used report for user: ceisgrub
Filesystem            Mount              Used   Limit  MaxLim Comment
Tiger2 home           /home            104.2K       0       0 
Tiger2 scratch GPFS   /scratch/gpfs    172.2K       0       0 
Tigress GPFS          /tigress            229       0       0 
For quota increase requests please use this website:
         https://forms.rc.princeton.edu/quota

The difference between Limit and Used is the available space for a given filesystem. For instance, in the example above the user has 18.6 - 13.9 = 4.7 GB available on /home. Follow the link at the bottom of the output of checkquota to request a quota increase for any of the filesystems.

The first section of the two in the output above shows the amount of storage used. The rightmost three numbers are the block limits, listed in either kilobytes (K), gigabytes (G), or megabytes (M). The first is the number of blocks in use. The second is the soft quota; warnings are issued when this threshold is reached. The third is the "hard" quota (or limit); file creation or extension will fail when this threshold is reached.

The second section shows the number of files. The rightmost three numbers are the current usage and limits. The first is the number of files in existence. The second is the soft quota; warnings are issued when this threshold is reached. The third is the "hard" quota (or limit); file creation will fail when this threshold is reached.

Note: There are 1024 bytes in a KB, 1024 KBs in an MB, 1024 MBs in a GB, and 1024 GBs in a TB.

How do I request a quota increase?

Please complete the Quota Increase Form. If you need an increase in the number of files then indicate this in the "Notes" field of the form.

How do I find and remove unnecessary files?

First identify the directories that are using the most space. To inspect your /home directory use this command after replacing <YourNetID> :

$ du -h --max-depth=1 /home/<YourNetID> | sort -hr

For another filesystem such as /scratch/gpfs/ use this command (it may take a minute):

$ du -h --max-depth=1 /scratch/gpfs/<YourNetID> | sort -hr

You can remove individual files with:

$ rm <file1> <file2> <file3>

Or remove entire directories with:

$ rm -rf <directory1> <directory2> <directory3>

 

Are you using /scratch/gpfs or /scratch/network?

Be aware that in addition to your /home directory, you have a large amount of space on /scratch/gpfs/<YourNetID> or /scratch/network/<YourNetID>. See the Data Storage page to learn about these filesystems. In many cases, users do not need a quota increase but instead need to learn about the different filesystems.

How should I deal with large Conda environments?

(Note: If your account was created on or after November 6, 2023, your .conda directory was automatically created as a soft link to a folder in scratch and you will not have the listed limitations.)

Conda environments are typically 1-10 GB in size. By default they are stored in /home/<YourNetID>/.conda. Because a typical quota size for a /home directory is 10-50 GB, it is common to run out of space. This section suggests ways of dealing with the storage requirements of Conda environments.

Move Your ~/.conda Directory to /scratch/gpfs or /scratch/network

If you already have one or more Conda environments then consider moving them to a location where you have more space using these commands:

Adroit

$ rsync -avu $HOME/.conda /scratch/network/$USER/
$ rm -Rf $HOME/.conda
$ cd $HOME && ln -s /scratch/network/$USER/.conda .conda

Della, Stellar, Tiger

$ rsync -avu $HOME/.conda /scratch/gpfs/$USER/
$ rm -Rf $HOME/.conda
$ cd $HOME && ln -s /scratch/gpfs/$USER/.conda .conda

The commands above create a symbolic link in your /home directory that points to the .conda directory in /scratch/network or /scratch/gpfs. While these locations are not backed up, it is generally easy to recreate an environment if there is a failure. You can also periodically export an environment file that would allow you to recreate the environment.

Request More Space

If for some reason you are not able to move your .conda directory (as explained above) then you can submit a request for a larger quota for /home by completing the Quota Increase Form. There is a limit to how large the quota can be made so please try to move your .conda directory when possible.

Remove Unnecessary Packages and Caches

Packages and other files for each environment are downloaded and saved in a cache directory called "pkgs". Over time there may be packages that are no longer needed by any of your environments. Run the following commands to remove these packages and related files:

$ module load anaconda3/2024.6
$ conda clean --all
$ checkquota

For more on this command see the Conda docs.

.cache Folder

Other software may be installed in your .cache folder and be taking up additional space. Aside from pip (which is usually remedied by using pip inside an anaconda environment after having moved it to scratch), below are some other packages to consider moving and/or clearing that can take up space in your .cache directory.

Adroit

$ rsync -avu $HOME/.cache /scratch/network/$USER/
$ rm -Rf $HOME/.cache
$ cd $HOME
$ ln -s /scratch/network/$USER/.cache .cache

Della, Stellar, Tiger, Traverse

$ rsync -avu $HOME/.cache /scratch/gpfs/$USER/
$ rm -Rf $HOME/.cache
$ cd $HOME
$ ln -s /scratch/gpfs/$USER/.cache .cache

Here are some other (less common) folders of this type that we've encountered and would recommend moving to scratch in a similar way:

  • .mambaforge

Hugging Face

Similar to anaconda above, please see our page on Hugging Face to learn the best ways to direct its files to be stored on /scratch instead of /home