Globus Data Transfer Guide

Globus supplies high speed, reliable, and asynchronous transfers to DesignSafe. Once setup, Globus will allow you to not only transfer files to and from DesignSafe, but also other cyberinfrastructure resources at TACC and other research centers. While the setup of Globus can take slightly longer than the other transfer methods (see Data Transfer Guide), it only needs to be performed once, making later transfers as fast (if not faster due to Globus' superior speed) than the other methods. For these reasons, Globus is the recommend approach for moving large quantities of data to and from DesignSafe.

The following provides detailed instructions for setting up Globus access to DesignSafe.

Step 1: Submit a ticket to activate Globus access for your DesignSafe/TACC Account

Create a new ticket on DesignSafe (https://designsafe-ci.org/help) to request Globus access for your account. 

In the body of the message indicate you wish to use Globus and list the Data Depot location(s), such as My Data or a project in My Projects, where you intend to transfer files to/from.

You will get a response via email when the activation is complete.

Step 2: Create a Distinguished Name (DN)

Globus requires a unique identifier, called a Distinguished Name (DN), for each user.

To create a DN, you need to log in through an authoritative source that can verify your identity, typically your university or employer.

If you already have a DN, you may reuse it here and skip to Step 3.

If you do not have a DN, you can create one for free using the “CI Logon” service.

Important: You must use your institution’s credentials, not your personal Google account, when setting up Globus to prevent issues when accessing the DesignSafe endpoint (Corral).

To create a DN, go to https://cilogon.org.

Select an "Identity Provider" from the drop-down list, and click "Log On" to be redirected to your Identity Provider's log in screen.

If your university or employer is not in the list, we recommend using Cyberduck instead. See the Cyberduck Data Transfer Guide for instructions.

After authenticating via the Identity Provider, you will be redirected back to CILogon, where you can find your Certificate Subject that you will need to copy and paste for Step3.

/DC=org/DC=cilogon/C=US/O=University of Texas at Austin/CN=Sample Person A00000

Step 3: Associate your DN with your DesignSafe/TACC Account

Login to the TACC User Portal (https://portal.tacc.utexas.edu) with your DesignSafe/TACC account.

Select "Home" > "Account Profile".

TACC User Portal

Select "Manage DNs" in the lower left.

TACC Account Profile

Enter the Certificate Subject obtained from CILogon.org in the text field, then click "Associate DN". This will associate the new DN with your account.

TACC Mange DNs

 

Important: It may take up to 2 hours for your DN to propagate to all TACC systems.

Step 4: Activate Your Desktop/Laptop as a Globus Endpoint and Connect

After giving your DN time to propagate through the systems (up to 2 hours), go to https://globus.org and log in.

Globus Login

Upon successful login you, you will be directed to the "File Manager" landing page.

"File Manager"

Click on "Endpoints" in the left-hand menu, then click “Create a personal endpoint”.

"Endpoints" > "In Use"

If you are using Windows click "Download Globus Connect Personal for Windows" in the upper right, otherwise click "Show me other supported operating systems" and select the one appropriate for your desktop/laptop.

Create a personal endpoint

Download and Install the Globus Connect Personal client. When setting the "Collection Name" be sure to select a descriptive name. We will use "My Laptop" as the name of our endpoint.

After installation and setup is complete return to the Globus online interface. Select the “File Manager” tab (upper left), then click on the search bar immediately to the right of the label "Collection".

To initiate a transfer between your local desktop/laptop and DesignSafe, select "Your Collections" followed by the name of your endpoint, this is "My Laptop" for our example.

"File Manager" > "Your Collections"

You can now access the files on your desktop/laptop via Globus.

Estabilish connection to local endpoint

Step 5: Connect to the DesignSafe (TACC Corral3) Endpoint

To view both endpoint simultaneously, change the Globus' interface to the "two pane" view by toggling the buttons next to "Panels" in the upper right.

To establish the second endpoint, select the right-hand search bar and search for "TACC". Select "TACC Corral3" as your second end point.

After successfully authenticating, you will be redirected back to Globus and you will now be able to access your data on Corral, where the files for DesignSafe are stored.

Corral3 is a large (40 PB), shared data resource, as such, the data stored on DesignSafe is only a small fraction of that stored on the entire system. Therefore, Corral3 requires a complex file structure for organization.

To access your data on DesignSafe

  • For My Data set Path to /corral-repl/projects/NHERI/shared/<username>/
  • For My Projects set Path to /corral-repl/projects/NHERI/projects/<project-uid>/
  • For Published DesignSafe projects set Path to /corral-repl/projects/NHERI/published/<PRJ-XXXX>/
  • For Published (NEES) projects set Path to /corral-repl/projects/NHERI/public/projects/<NEES-XXXX-XXXX.groups>/

For more information on path selection please see the detailed guide on Setting the Path to DesignSafe on Corral.

After entering the appropriate path to DesignSafe on Corral, you are ready to perform your file transfer.

Note: For directories you connect to frequency, for example My Data, you can create a bookmark for easy access using the bookmark icon immediately to the right of the "Path" bar.

Establish connection to Corral endpoint

Step 6: Perform Transfer between Your Local Enpoint and the DesignSafe (TACC Corral3) Endpoint

To begin your transfer, select the file/folder you wish to move to/from DesignSafe.

Select the receiving location for the data in the other panel. You can do this by either typing the desired path or clicking through the directory structure.

Press the "Start" button to begin the transfer.

Once the transfer is initiated, you can view the state of the transfer by clicking "Activity".

You do not need to keep the Globus interface open through the duration of the transfer.

Globus will email you when the transfer is complete.

Important: To avoid problems when transferring files to DesignSafe, name your files using standard ASCII characters, for example [a-z][A-Z][0-9][-,_], and avoid the use of special characters, for example [%,*,#,@,°].

If you have any issues transferring files to DesignSafe using Globus, please create a ticket (https://designsafe-ci.org/help).

Last Update: October 2021