DesignSafe supports multiple ways of moving data in and out of the Data Depot – which one is best depends on how you will use DesignSafe. While the web interface in the DesignSafe portal is easy for moving small numbers of modest size files, if you need to move large volumes of data (> 100 MB), large numbers of files (> 25), or move directories, Globus is the recommended way of moving data in and out of DesignSafe.
Globus supplies high speed, reliable, asynchronous transfers to DesignSafe. Globus is fast, for large volumes of data, as it uses multiple network sockets simultaneously to transfer data. It is reliable for large numbers of directories and files, as it can catch errors during the transfer process and automatically restart itself.
Of course, to get these benefits, there are a few setup steps you have to do beyond the normal Data Depot transfer process. Most of the steps you are only required to do once when you set up Globus to use for the first time. Several steps will need to be repeated each time you set up a new computer to use Globus for DesignSafe. Once you are set up, you can use Globus not only for transfers to and from DesignSafe, but also to access other cyberinfrastructure resources at TACC and at other centers around the world.
To start using Globus, you need to do three things: enable Globus for your DesignSafe account, generate a unique identifier for all Globus services, and enroll the machine you are transferring data to/from with Globus (this can be your personal laptop or desktop, or a server to which you have access). Follow this one-time process to set up the Globus file transfer capability.
PLEASE NOTE: You must use your institution’s credentials and not your personal Google account when setting up Globus. If you use a personal account, you will encounter an issue when accessing the DesignSafe endpoint (Corral).
Step 1: Submit a ticket to activate Globus access for your DesignSafe/TACC Account
Use the DesignSafe Help feature to request Globus access for your account. Simply submit a ticket here: https://designsafe-ci.org/help and in the body of the message say you wish to use Globus. Please list the Data Depot locations(s), such as My Data or a project in My Projects, where you intend to transfer files to/from. You will get a response via email when this is complete.
Step 2: Retrieve and Associate a DN to your DesignSafe/TACC Account
In order for Globus to know who you are when you move data in and out of DesignSafe from your computer, or between any other pair of systems, Globus needs a unique identifier for you, which is called a “Distinguished Name”, or DN. You can generate a DN instantly for free. To create a DN, you need to log in from some authoritative source that can verify your identity, typically your university or employer. If you already have a DN from another source, you can use that. If you do not, you can associate one with your account from many of the major universities in the world via the “CI Logon” service.
To retrieve your DN, go to https://cilogon.org in your browser. Select an "Identity Provider" from the drop-down list, and click "Log On" which will take you to the login screen for the Identity Provider you selected. If your university or employer is not in the list, we recommend registering for an XSEDE account
After successfully authenticating using your chosen Identity Provider, you will be redirected back to CILogon, where you can find your Certificate Subject that you will need to copy and paste for the next step:
/DC=org/DC=cilogon/C=US/O=University of Texas at Austin/CN=Sample Person A00000
Login to the TACC User Portal https://portal.tacc.utexas.edu with your DesignSafe/TACC account and select "Account Profile" from the main menu under the "Home" dropdown.
On the left of the page is a list of account actions, select "Manage DNs". You will be presented with a list of the DNs currently associated with your TACC account and a text field to associate a new DN to your account. Enter the Certificate Subject obtained from CILogon.org in the text field. Click the button to "Associate DN". This will associate the new DN with your account. Please note, it may take up to 30 minutes for this change to propagate to all TACC systems.
Step 3: Activate Your Desktop/Laptop as a Globus Endpoint and Transfer Files
Now that you have associated the DN with your TACC account and given the DN time to propagate to the systems (up to 30 minutes), you can activate the Globus transfer endpoints and begin transferring files.
Go to https://globus.org and log in.
Upon successful login you, you will be directed to the "File Manager" landing page.
Click on "Endpoints" in the left-hand menu, then click “Create a personal endpoint”.
If you are using Windows click "Download Globus Connect Personal for Windows" in the upper right, otherwise click "Show me other supported operating systems" and select the one appropriate for your desktop/laptop.
Download and Install the Globus Connect Personal client. When setting the "Display Name" be sure to select a descriptive name. We will use "My Laptop" as the name of our endpoint.
After installation and setup is complete return to the Globus online interface. Select the “File Manager” tab (upper left), then click on the search bar immediatley to the right of the label "Collection".
To initiate a transfer between your local desktop/laptop and DesignSafe, select "Your Collections" followed by the name of your end point, this is "My Laptop" for our example.
You can now access the files on your desktop/laptop via Globus.
You can change the display of Globus' interface by toggling the options next to "Panels" in the upper right of the screen. The "two pane" view is recommended.
To establish the second transfer endpoint, select the right-hand search bar and search for "TACC". Select "TACC Corral3" as your second end point.
After successfully authenticating, you will be redirected back to Globus and you will now be able to access your data on Corral, where the files for DesignSafe are stored:
The project-uid can be found in the URL link in the DesignSafe portal while viewing the project (see figure below).
You can now transfer files between the selected endpoints.
Once the transfer is initiated, you can view the state of the transfer by clicking "Activity".
You will also receive an email to the registered email address once the transfer is completed.
If you have any issues transferring files to DesignSafe using Globus please fill out a ticket by going to the DesignSafe website, selecting "Help" then "Submit a Ticket".