Globus

Globus at Princeton

Globus is, among other things, an infrastructure for transferring large amounts of research data. Some researchers at Princeton have been using Globus for years, either on departmental servers or on their desktop workstations or laptops. Research Computing supports Globus data transfer to and from the GPFS-based /tigress and scratch disk space (/della/scratch/gpfs, /perseus/scratch/gpfs, /tiger/scratch/gpfs2) connected to the Research Computing clusters. Some departments at Princeton University have and manage their own Globus data transfer node. Several known departments are: Princeton Neuroscience Institute (PNI), Lewis-Sigler Institute (LSI), Astrophysics, and Physics. Please contact your department staff if you want to utilize your department's Globus data transfer node. Research Computing will support the GPFS-based /tigress and scratch disk space side of the transfer, whether it is the source or destination of your data transfer. 

Today, the Globus service offered by Research Computing consists of a single Dell R630 server, named tigress-dtn.princeton.edu. This server is connected to the Campus Data Network with 10 Gb/s Ethernet, and is connected to the TIGRESS facility's GPFS storage cluster with FDR (54Gb/s) Infiniband.

 

When Should You Use Globus for Data Transfer? 

For transferring small amounts of data, the scp and sftp utilities generally work well and are easier to use than Globus. However, if your transfer takes a long time (more than 15 minutes) or is using an unreliable connection, Globus is recommended. A rule of thumb is that Globus is a good choice when transferring between campus hosts takes more than 15 minutes per 100 GB over a 1G connection or more than 4 minutes over a 10G connection. For transfer to or from an off-campus host, Globus is recommended if your transfer takes more than 1 hour per 100 GB.

 

Who Can Use Research Computing's Globus Service and How?

Users of Tigress systems can use Globus to transfer data to and from the GPFS-based /tigress and scratch disk space (/della/scratch/gpfs, /perseus/scratch/gpfs, /tiger/scratch/gpfs2). Authentication to Research Computing's Globus service utilizes the same NetID and password used to access other Princeton services, such as email and the Nobel computational cluster. Other, older Globus servers on campus may utilize globusid.org credentials, which may be linked to Princeton credentials, if that is so desired.  

Primary access to the Globus data transfer service is through the web interface at Globus.org. The Globus organization provides a series of "how to" documents including a getting started guide that covers logging in and transferring files. Research Computing has a locally written document describing How to Log in to Globus with a Princeton NetID.

When logging in with your Princeton NetID, you will using CILogon.com which, in turn, utilizes Princeton's membership in the InCommon federation. InCommon allows users at member institutions to use their local credentials to access services offered elsewhere, such as Globus. While the tigress-dtn server is located on campus, the service is run by off-campus servers which manage and monitor the transfers.

The first time you login with your Princeton credentials, you will be offered an opportunity to link those credentials with an existing Globus account. Long-time Globus users may wish to do this, as it will allow you to use a single set of credentials (e.g. your Princeton NetID and password), rather than needing to login with multiple accounts. Research computing has a locally written document about How to link a New Identity to an Existing Globus Account, which assumes that you are currently logged in to Globus with credentials other than your Princeton NetID.

 

Example Data Transfer Walkthrough

Connect to Globus.org with your web browser, and log in using your Princeton credentials. You should land on the "Transfer Files" tab of the "Manage Data" page. On the left side of the transfer panel, you should see "Start here..." in the Endpoint box. Click in this box to display a pop-up window.

In the endpoint box in the pop-up window, start typing "esnet test" which should cause a list of test Data Transfer Nodes (DTNs) to be displayed. Click on one of the names. You will be returned to the transfer page, with a list of folders and files now appearing on the left side.

Now click in the Endpoint box on the right side of the page. This time, enter "princeton tigress" in the pop-up window. Click on the name "Princeton TIGRESS."

You may be prompted to authenticate to the endpoint. If so, click "continue" to proceed to a CILogon page. Select "Princeton University" as your identity provider, optionally check the "Remember this selection" box and click on "Log On." Authenticate through CAS as you normally do.

You should now be back on the "Transfer Files" page with the ESnet DTN files listed on the left side, and all of /tigress listed on the right. Either type a destination directory after "/tigress/" in the Path box (and hit the return key), or scroll through the list and double-click on the desired destination directory.

Back on the left side of the page, use a single click to select the "10G.dat" file. This should highlight the right-pointing arrow. Click on the arrow to start your transfer, which will pop up an informational box with your Globus transfer ID. Notice that there is now a number "1" displayed next to the circle icon for RECENT ACTIVITY.

Click on "Activity" in the tab list near the top of the page, and then click on the 3 vertically stacked dots in the box listing your transfer. This will display statistics about your test transfer. This tab defaults to showing only recent transfer activity, but also allows access to your entire transfer history.

Once the transfer completes, you will receive an email message with a link that will take you to a statistics display for this transfer.