Changes between Version 6 and Version 7 of cypress/GlobusInBarchJob
- Timestamp:
- 09/21/2026 09:10:03 AM (4 hours ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
cypress/GlobusInBarchJob
v6 v7 1 1 = Example Pipeline for Data Transfer Using Globus and Computation in Batch Jobs = 2 Here we consider a pipeline to dothe following processes.2 Here we consider a pipeline to perform the following processes. 3 3 1. Transfer data files from Box to the Cypress Lustre directory. 4 4 2. Perform computation using the data. … … 44 44 For each transfer, the user must set whether to keep or delete the source directory. (see below) 45 45 46 Note that the directory on Cypress must be set with write -access by '''~/.globusonline/lta/config-paths'''.46 Note that the directory on Cypress must be set with write access by '''~/.globusonline/lta/config-paths'''. 47 47 48 48 === Data Transfer Script === … … 99 99 else 100 100 SOURCE_EP=$MY_UUID:$CYPRESS_WORK_DIR 101 DEST_EP=$TULANE_BOX:$BOX_ DATA_DIR101 DEST_EP=$TULANE_BOX:$BOX_RESULT_DIR 102 102 fi 103 103 104 # Check logged in to Globus104 # Check if logged in to Globus 105 105 output=$(globus whoami >/dev/null 2>&1) 106 106 output_code=$? … … 114 114 output_code=$? 115 115 if [ $output_code -ne 0 ]; then 116 echo "ERROR! The transfer of data incould not be started."116 echo "ERROR! The transfer of data could not be started." 117 117 globusconnect -stop 118 118 exit 1 119 119 fi 120 120 121 # wait u til the taskdone.121 # wait until the task is done. 122 122 output=$(globus task wait $task_id) 123 123 output_code=$? 124 124 if [ $output_code -ne 0 ]; then 125 echo "ERROR! The transfer of data wasfailed."125 echo "ERROR! The transfer of data failed." 126 126 globus task cancel $task_id 127 127 globusconnect -stop … … 171 171 exit 0 172 172 }}} 173 This is the main computing script , which the user has toedit. It doesn't have to use CentOS7 nodes.173 This is the main computing script the user must edit. It doesn't have to use CentOS7 nodes. 174 174 175 175 176 176 == How to submit a job == 177 On login node, '''submitJob.sh''', '''transferData.sh''', and '''computing.sh''' must be in the same directory.177 On the login node, '''submitJob.sh''', '''transferData.sh''', and '''computing.sh''' must be in the same directory. 178 178 {{{ 179 179 sh ./submitJob.sh
