Changes between Version 1 and Version 2 of cypress/Box


Ignore:
Timestamp:
05/11/22 15:35:07 (9 months ago)
Author:
cbaribault
Comment:

Added at bottom the section Box Transfer Rate Limits

Legend:

Unmodified
Added
Removed
Modified
  • cypress/Box

    v1 v2  
     1
     2[[PageOutline]]
    13
    24== Tulane Box Accounts ==
     
    6567[tulaneID@cypress1 ~]$ rclone ls TU-Box:
    6668}}}
    67 3. To list the contents of a top-level folder in your Tulane Box account...
     693. To list the contents of a top-level directory in your Tulane Box account...
    6870{{{
    69 [tulaneID@cypress1 ~]$ rclone ls TU-Box:<top-level-folder>
     71[tulaneID@cypress1 ~]$ rclone ls TU-Box:<top-level-directory>
    7072}}}
    71 4. To copy the contents of a top-level folder in your Tulane Box account to your current Cypress working directory...
     734. To copy the contents of a top-level directory in your Tulane Box account to your current Cypress working directory...
    7274{{{
    73 [tulaneID@cypress1 ~]$ rclone copy TU-Box:<top-level-folder> .
     75[tulaneID@cypress1 ~]$ rclone copy TU-Box:<top-level-directory> .
    7476}}}
    75 5. To show progress using the '''-P''' option while copying the (~5GB) file bigFile.txt in your current Cypress working directory to a top-level folder in your Tulane Box account...
     775. To show progress using the '''-P''' option while copying the (~5GB) file bigFile.txt in your current Cypress working directory to a top-level directory in your Tulane Box account...
    7678{{{
    7779[tulaneID@cypress1 ~]$ ls -l bigFile.txt
    7880-rw-r--r-- 1 tulaneID group 5000000000 Oct 25  2019 bigFile.txt
    79 [tulaneID@cypress1 ~]$ rclone -P copy bigFile.txt TU-Box:<top-level-folder>
     81[tulaneID@cypress1 ~]$ rclone -P copy bigFile.txt TU-Box:<top-level-directory>
    8082Transferred:      475.586M / 4.657 GBytes, 10%, 37.208 MBytes/s, ETA 1m55s
    8183Errors:                 0
     
    8688 *                                   bigFile.txt:  9% /4.657G, 35.761M/s, 2m0s
    8789}}}
    88 6. To simply list the available subcommands under rclone.
     906. To simply list the available subcommands under rclone...
    8991{{{
    9092[tulaneID@cypress1 ~]$ rclone --help
    9193}}}
     947. To show the help message - including a detailed explanation and helpful examples - for the rclone copy subcommand...
     95{{{
     96[tulaneID@cypress1 ~]$ rclone copy --help
     97}}}
     988. To list the available flags used with the rclone command... (See also [[Box#BoxTransferRateLimits|Box Transfer Rate Limits]] below.)
     99{{{
     100[tulaneID@cypress1 ~]$ rclone help flags
     101}}}
     102
     103==== Box Transfer Rate Limits ====
     104
     105Depending on the number of files contained in the <source> directory of your command {{{rclone copy <source> <destination>}}}, many of those files may fail to transfer with error '''429 Too Many Requests'''. To protect quality of service, Box accounts enforce the following transfer rate limits, though with 2 additional retries for each failed transfer request by default. (See [https://developer.box.com/guides/api-calls/permissions-and-errors/rate-limits Box Rate Limits].)
     106
     1071. 1000 API requests per minute, per user (for Box API requests other than file uploads)
     1082. 240 file upload requests per minute, per user (for file uploads to Box specifically)
     109
     110To avoid exceeding the above mentioned rate limits, you can limit your HTTP transactions per second by including the rclone flag '''--tpslimit''' in your rclone command  as in the following. (See '''--tpslimit''' and other flags for rclone using the command {{{rclone help flags}}} for details.)
     1111. '''Downloading files:''' To limit your rate (to 1000 requests per minute) of downloading files from a top level directory of your Tulane Box account to your current Cypress working directory...
     112{{{
     113[tulaneID@cypress1 ~]$ rclone copy TU-Box:<top-level-directory> . --tpslimit 16.6 # 1000 per minute or 16.6 per second
     114}}}
     1151. '''Uploading files:''' To limit your rate (to 240 requests per minute) of uploading files to a top level directory of your Tulane Box account from your desired Cypress directory...
     116{{{
     117[tulaneID@cypress1 ~]$ rclone copy TU-Box:<top-level-directory> <Cypress-directory> --tpslimit 16.6 # 1000 per minute or 16.6 per second
     118}}}