Changes between Version 1 and Version 2 of cypress/GlobusInBarchJob
- Timestamp:
- 04/03/25 17:09:19 (2 days ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
cypress/GlobusInBarchJob
v1 v2 6 6 4. Delete files in Cypress Lustre. 7 7 8 To do this, first of all, the user must '''log in to Globus with the command line tool once''', see [https://wiki.hpc.tulane.edu/trac/wiki/cypress/Globus#FileTransferwithGlobusCommandlineTools here]. 8 9 == Scripts == 10 There are three scripts. 11 9 12 === Job Submission Script === 10 13 '''submitJob.sh''' … … 33 36 echo $JOB3 "Submitted" 34 37 }}} 35 '''JOB_NAME''' is the job name. 38 The user must set the following entries. 39 * '''JOB_NAME''' is the job name. 40 * '''BOX_DATA_DIR''' is the directory in Box where the source data is stored. 41 * '''CYPRESS_WORK_DIR''' is the directory where the downloaded data is stored. 42 * '''BOX_RESULT_DIR''' is the directory where results are uploaded in Box. 36 43 37 '''BOX_DATA_DIR''' is the directory in Box where the source data is stored. 44 For each transfer, the user must set to keep or delete the source directory. (see below) 38 45 39 '''CYPRESS_WORK_DIR''' is the directory where the downloaded data is stored.40 41 '''BOX_RESULT_DIR''' is the directory where results are uploaded in Box.42 46 43 47 === Data Transfer Script === … … 135 139 exit 0 136 140 }}} 137 141 The user doesn't have to edit this file. But in '''submitJob.sh''', the user should set the keep or delete option. 142 '''Usage: transferData.sh [DOWNLOAD | UPLOAD] [KEEP | DELETE]''' 143 The first parameter '''[DOWNLOAD | UPLOAD]''', which 144 * When '''DOWNLOAD''' is set, it downloads data from Box to Cypress. 145 * When '''UPLOAD''' is set, it uploads data from Cypress to Box. 146 The second parameter '''[KEEP | DELETE]''', which 147 * When '''KEEP''' is set, data is kept. 148 * When DELETE is set, it deletes the source directory. If '''DOWNLOAD''' is set, it deletes files in Box. If '''UPLOAD''' is set, it deletes files in Cypress. 138 149 139 150 === Computing Script === … … 159 170 exit 0 160 171 }}} 172 This is the main computing script, which the user has to edit. It doesn't has to use CentOS7 nodes. 161 173 162 174 163 175 == How to submit a job == 176 On login node, '''submitJob.sh''', '''transferData.sh''', and '''computing.sh''' must be in the same directory. 164 177 {{{ 165 sh ./ SubmitJob.sh178 sh ./submitJob.sh 166 179 }}} 180 This submits three jobs. The first job downloads data from Box. The second job depends on the first job and performs the computing task. The third job depends on the second job and uploads the results to Box.
