This guide assumes some familiarity with Command Line Interfaces (CLI). This method is of most use to NHERI centers that need to bulk upload their data on a schedule.
Step 1: Submit a ticket to activate Corral access for your DesignSafe/TACC Account
If you have already requested access to this for using Globus or other data transfer methods, you may skip this step.
Use the DesignSafe Help feature to request Corral access for your account. Simply submit a ticket here: https://designsafe-ci.org/help and in the body of the message say you wish to use Globus. Please list the Data Depot locations(s), such as My Data or a project in My Projects, where you intend to transfer files to/from. You will get a response via email when this is complete.
Step 2: Retrieve and Associate a DN to your DesignSafe/TACC Account
If you have not already completed this step for Globus transfers, follow the instructions found on our Globus guide:
Step 3: Activate your desktop/laptop/machine as a Globus Endpoint
If your files are stored on your institution's internal storage machine, check to see if your organization has an available Globus Endpoint already available for use for transfers. If not, talk to your administrator about making this access available.
If you do not already have an available Globus Endpoint, follow the steps on our Globus Guide for installing Globus Personal Connect found here: https://www.designsafe-ci.org/rw/user-guides/globus-data-transfer-guide/#!#step3
Step 4: Install Globus CLI
Go here for instructions for installing the Globus CLI: https://docs.globus.org/cli
The recommended method requires a system with Python3 and the ability to run pip commands.
Step 5: Get settings for CLI Transfer
Now that all the programs are installed, we will get all the information we need to set up the automation script.
Step 6: Test Globus CLI Transfer
With the endpoint IDs and the Paths, we can now do a basic transfer with the Globus CLI.
The basic structure for a Transfer command is:
globus transfer [OPTIONS] SOURCE_ENDPOINT_ID[:SOURCE_PATH] DEST_ENDPOINT_ID[:DEST_PATH]
An example Transfer command:
globus transfer --recursive [endpoint uuid for your machine]:[path on your machine] [endpoint uuid for tacc corral3]:[path on tacc corral3 to your My Data or Project]
Fill out the variables in the example command with the information collected in the previous steps and submit the transfer. You will get a message that tells you if your transfer command worked or not. If your transfer command worked, you will get this message:
Message: The transfer has been accepted and a task has been created and queued for executionTask ID: [taskid]
The full reference for the Globus CLI is found here: https://docs.globus.org/cli/reference
The full reference for the Transfer command is found here: https://docs.globus.org/cli/reference/transfer You should reference this for any other Options you may want to add to your Transfer command.
Step 7: Automate Globus Transfers
After doing a basic transfer to confirm your command was correct, you can generate a script and a cron job to automate transfers. The script will help automate transfers and you can initate a transfer whenever you want. The cron job is only needed if you know you will need transfers out of the same directory on a regular basis. If you only need transfers on a infrequent basis, you can skip the cron job.
Here's an example script for automating transfers. Note that this does hard code your DesignSafe project uuid into the script, so this assumes you are always transferring to the same DesignSafe project.
An example crontab you can use to for using cron to use this script to automatically run your transfer:
0 */6 * * * /location/of/your/globustransfer.sh > /dev/null