Use the checkquota command to check your current storage and your storage limits. Here's an example:
$ checkquota Storage/size quota filesystem report for user: ceisgrub Filesystem Mount Used Limit MaxLim Comment Tiger2 home /home 13.9GB 18.6GB 20GB Tiger2 scratch GPFS /scratch/gpfs 9.2GB 500GB 512GB Tigress GPFS /tigress 397.9MB 500GB 512GB Storage number of files used report for user: ceisgrub Filesystem Mount Used Limit MaxLim Comment Tiger2 home /home 104.2K 0 0 Tiger2 scratch GPFS /scratch/gpfs 172.2K 0 0 Tigress GPFS /tigress 229 0 0 For quota increase requests please use this website: https://forms.rc.princeton.edu/quota
The difference between Limit and Used is the available space for a given filesystem. For instance, in the example above the user has 18.6 - 13.9 = 4.7 GB available on /home. Follow the link at the bottom of the output of checkquota to request a quota increase for any of the filesystems: https://forms.rc.princeton.edu/quota.
The first section of the two in the output above shows the amount of storage used. The rightmost three numbers are the block limits, listed in either kilobytes (K), gigabytes (G), or megabytes (M). The first is the number of blocks in use. The second is the soft quota; warnings are issued when this threshold is reached. The third is the "hard" quota (or limit); file creation or extension will fail when this threshold is reached.
The second section shows the number of files. The rightmost three numbers are the current usage and limits. The first is the number of files in existence. The second is the soft quota; warnings are issued when this threshold is reached. The third is the "hard" quota (or limit); file creation will fail when this threshold is reached.
Note: There are 1024 bytes in a KB, 1024 KBs in an MB, 1024 MBs in a GB, and 1024 GBs in a TB.
How do I request a quota increase?
Please complete this form: https://forms.rc.princeton.edu/quota.
How do I remove unnecessary files?
First identify the directories that are using the most space. To inspect your /home directory use this command after replacing <YourNetID> :
$ du -h --max-depth=1 /home/<YourNetID> | sort -hr
For another filesystem such as /scratch/gpfs/ use this command (it may take a minute):
$ du -h --max-depth=1 /scratch/gpfs/<YourNetID> | sort -hr
You can remove individual files with:
$ rm <file1> <file2> <file3>
Or remove entire directories with:
$ rm -rf <directory1> <directory2> <directory3>
How should I deal with large Conda environments?
One solution is to request a larger quota for /home (see link above).
Note that the .conda/pkgs directory is a cache. It can be removed at any time to save space. The only disadvantage to removing this directory is that packages for new environments will have to be downloaded instead of being pulled from the local cache (if previously available).
Another approach is to install the packages directly on /scratch/gpfs/<YourNetID> from the beginning. While the files on this filesystem are not backed up, most environments are easy to recreate in the event of a failure. This approach will reduce the storage requirements for your ~/.conda directory which is stored in /home by default. Here's an example:
$ module load anaconda3 $ conda create --prefix /scratch/gpfs/$USER/tf2-gpu tensorflow-gpu matplotlib ipython
A modification is needed to activate the environment in the Slurm script:
#!/bin/bash #SBATCH --job-name=myjob # create a short name for your job #SBATCH --nodes=1 # node count #SBATCH --ntasks=1 # total number of tasks across all nodes #SBATCH --cpus-per-task=1 # cpu-cores per task (>1 if multi-threaded tasks) #SBATCH --gres=gpu:1 # number of gpus per node #SBATCH --mem=1G # total memory (RAM) per node #SBATCH --time=00:00:30 # total run time limit (HH:MM:SS) module purge module load anaconda3 conda activate /scratch/gpfs/$USER/tf2-gpu python myscript.py
A third solution is to move your current .conda directory to /scratch/gpfs/<YourNetID> and then set your new configuration in ~/.condarc. Here are the steps:
$ mv ~/.conda /scratch/gpfs/<YourNetID>
Then create ~/.condarc with the following contents (be sure to insert your NetID):
pkgs_dirs: - /scratch/gpfs/<YourNetID>/.conda/pkgs envs_dirs: - /scratch/gpfs/<YourNetID>/.conda/envs
You could also use miniconda to do the same.