Changes between Version 10 and Version 11 of cypress/Globus


Ignore:
Timestamp:
04/03/25 13:29:50 (5 days ago)
Author:
fuji
Comment:

—

Legend:

Unmodified
Added
Removed
Modified

  • TabularUnified cypress/Globus

    v10 v11  
    11[[PageOutline]]
    22= Globus =
    3 ​[https://www.globus.org Globus] is research cyberinfrastructure, developed and operated as a not-for-profit service by the University of Chicago. With Globus, you can easily, reliably and securely move, share, & discover data no matter where it lives – from a supercomputer, lab cluster, tape archive, public cloud or laptop. Access and manage all your data, even protected data, from anywhere, using your existing identities, with just a web browser.​
     3​[https://www.globus.org Globus] is a research cyberinfrastructure, developed and operated as a not-for-profit service by the University of Chicago. With Globus, you can easily, reliably, and securely move, share, & discover data no matter where it lives – from a supercomputer, lab cluster, tape archive, public cloud or laptop. Access and manage all your data, even protected data, from anywhere, using your existing identities, with just a web browser.​
    44
    55== Access Files on Cypress From Globus Web App ==
    6 This is an instruction to access your files on Cypress with Globus Web App.
     6This is an instruction to access your files on Cypress with the Globus Web App.
    77
    88=== Initial Setup ===
    9 First of all, you need a globus account. You can get access to globus with GitHub, Google, or ORCID iD, and you can also get access with Tulane credentials (see [https://libguides.tulane.edu/globus here]).
     9First of all, you need a Globus account. You can get access to Globus with GitHub, Google, or ORCID iD, and you can also get access with Tulane credentials (see [https://libguides.tulane.edu/globus here]).
    1010
    11 To setup on Cypress, see [GlobusConnctPersonalSetup this page].
     11To set up on Cypress, see [GlobusConnctPersonalSetup this page].
    1212
    1313
    … …  
    4040
    4141== Running Globus Connect Personal on Cypress ==
    42 You have to use Centos7 computing node to run Glonus Connect Personal. Below is an example Slum script.
     42You have to use a CentOS 7 computing node to run Glonus Connect Personal. Below is an example Slum script.
    4343
    4444
    … …  
    6060
    6161== File Transfer with Globus Web App ==
    62 Login to Globus with a web browser and open FILE MANAGER.
     62Log in to Globus with a web browser and open FILE MANAGER.
    6363
    6464[[Image(FileManager.png, 40%)]]
    … …  
    8383}}}
    8484
    85 On a centos7 computing node, run Glonus Connect Personal in the background.
     85On a CentOS 7 computing node, run Glonus Connect Personal in the background.
    8686{{{
    8787module load globusconnectpersonal/3.2.5
    … …  
    9393source activate globus-cli
    9494}}}
    95 Log in to globus. This requires an authentication process with a web browser.
     95Log in to authenticate Globalus services. This needs to be done only once, and it requires an authentication process with a web browser.
    9696{{{
    97 globus login
     97globus login --no-local-server
    9898}}}
     99
     100You can see your UUID by
     101{{{
     102globus endpoint local-id
     103}}}
     104
     105Your UUID is a unique ID and is used to identify your Cypress endpoint. It is convenient to define it as an environmental variable, for example,
     106{{{
     107MY_UUID=$(globus endpoint local-id)
     108}}}
     109
     110UUID for the Tulane Box endpoint is defined by the environmental variable $TULANE_BOX.
    99111
    100112To access Tulane Box, this may require another authentication process.
    … …  
    103115}}}
    104116
     117To look into a specific directory in Box, for example, /projects/data,
     118{{{
     119globus ls "$TULANE_BOX:/projects/data"
     120}}}
     121
     122To transfer a file in Box to Cypress, for example/projects/data/dataset1.dat in Box to /lustre/project/mygoup/mydir/data/dataset1.dat,
     123{{{
     124globus transfer "$TULANE_BOX:/projects/data/dataset1.dat" "$MY_UUID:/lustre/project/mygoup/mydir/data/dataset1.dat" --label "TRANSFER1"
     125}}}
     126This submits a transfer task with the label 'TRANSFER1', you can monitor the progress on the Web App.
     127
     128To transfer files in a directory in Box to Cypress recursively, for example/projects/data/ in Box to /lustre/project/mygoup/mydir/data/,
     129{{{
     130globus transfer "$TULANE_BOX:/projects/data/" "$MY_UUID:/lustre/project/mygoup/mydir/data/" --recursive --label "TRANSFER_DIR"
     131}}}
     132
     133Note that the directory on Cypress must be set write-access by '''~/.globusonline/lta/config-paths''' (see above).
     134See [https://docs.globus.org/cli/reference/transfer/] for more options.
     135
    105136See [https://docs.globus.org/cli/quickstart/] for further process.
    106137
     138=== File Transfer and Computation in Batch Jobs ===
     139[GlobusInBarchJob Example Pipeline 1]