How Do I Get My Files Onto (or Off) the Cluster?
One of the most frequently asked questions is how to get files to Adroit or any other cluster. Again, Linux/MacOS has an answer out of the box: the 'scp' command. For Windows, clients like PuTTY and Mobaxterm or FTP clients (like WS_FTP–paid, sadly–or Filezilla) are needed. In both cases, make sure you're using interactive mode. Filezilla especially can be a pain with Duo Authentication for Nobel, but we have some tips here.
If you're transferring a lot of files, consider zipping them.
The default client to transfer files is 'scp.' This is a copy command that uses SSH to copy files.
The general structure of the scp command is:
scp [options] [netid]@source-host:file/location [netid@]destination-host:file/location
It's generally more straightforward to transfer files from your personal computer to the clusters, since you don't need to specify the user and host for the system you're already in.
Example - Using scp Command
Note that for the following commands to work from off-campus you must be connected via the campus VPN.
From your local machine (e.g., laptop) to a Princeton cluster
A Princeton student with the netid jessedoe wants to transfer a 'data.csv' file from their personal laptop to the '/scratch/network/jessedoe' directory on the Adroit cluster. To do this, they would use the following command:
scp ~/mydatafiles/data.csv firstname.lastname@example.org:/scratch/network/jessedoe
Transferring a folder requires the -r option, and would therefore look like this:
scp -r ~/mydatafiles email@example.com:/scratch/network/jessedoe
From a Princeton cluster to your local machine (e.g., laptop)
To transfer the file myfile.txt from Adroit to your Deskop:
$ scp firstname.lastname@example.org:/scratch/network/jessedoe/myfile.txt /Users/jessedoe/Desktop
To transfer the project1 directory and all of its contents to the "research" directory on your laptop:
$ scp -r email@example.com:/scratch/network/jessedoe/project1 /Users/jessedoe/research
If you need to transfer particularly large files, you may need to use Globus. We have additional information on Globus on our website, and run a regular workshop on data transfers with Globus.
Learn More About File Transfer Options
Additional information on learning how to transfer data–with tools such as scp, ftp, rsync, and Globus–can be found on Research Computing's Learning Resources: Data Transfer page.
SSH Keys: scp without typing passwords
Typing passwords every time you want to connect to a machine or, more annoying, every time you want to copy a file to/from a remote machine gets annoying quickly. One solution is to enable passwordless login/remote operations by generating a public/private pair of ssh keys and using them to negotiate the connection. The procedure is explained in this guide.