Changes between Version 1 and Version 2 of cypress/Box

Timestamp:

05/11/22 15:35:07 (3 years ago)

Author:

cbaribault

Comment:

Added at bottom the section Box Transfer Rate Limits

cypress/Box

@@ +1 @@
+
+[[PageOutline]]
 
 == Tulane Box Accounts ==
@@ -65 +67 @@
 [tulaneID@cypress1 ~]$ rclone ls TU-Box:
 }}}
-3. To list the contents of a top-level folder in your Tulane Box account...
+3. To list the contents of a top-level directory in your Tulane Box account...
 {{{
-[tulaneID@cypress1 ~]$ rclone ls TU-Box:<top-level-folder>
+[tulaneID@cypress1 ~]$ rclone ls TU-Box:<top-level-directory>
 }}}
-4. To copy the contents of a top-level folder in your Tulane Box account to your current Cypress working directory...
+4. To copy the contents of a top-level directory in your Tulane Box account to your current Cypress working directory...
 {{{
-[tulaneID@cypress1 ~]$ rclone copy TU-Box:<top-level-folder> .
+[tulaneID@cypress1 ~]$ rclone copy TU-Box:<top-level-directory> .
 }}}
-5. To show progress using the '''-P''' option while copying the (~5GB) file bigFile.txt in your current Cypress working directory to a top-level folder in your Tulane Box account...
+5. To show progress using the '''-P''' option while copying the (~5GB) file bigFile.txt in your current Cypress working directory to a top-level directory in your Tulane Box account...
 {{{
 [tulaneID@cypress1 ~]$ ls -l bigFile.txt
 -rw-r--r-- 1 tulaneID group 5000000000 Oct 25  2019 bigFile.txt
-[tulaneID@cypress1 ~]$ rclone -P copy bigFile.txt TU-Box:<top-level-folder>
+[tulaneID@cypress1 ~]$ rclone -P copy bigFile.txt TU-Box:<top-level-directory>
 Transferred:      475.586M / 4.657 GBytes, 10%, 37.208 MBytes/s, ETA 1m55s
 Errors:                 0
@@ -86 +88 @@
  *                                   bigFile.txt:  9% /4.657G, 35.761M/s, 2m0s
 }}}
-6. To simply list the available subcommands under rclone.
+6. To simply list the available subcommands under rclone...
 {{{
 [tulaneID@cypress1 ~]$ rclone --help
 }}}
+7. To show the help message - including a detailed explanation and helpful examples - for the rclone copy subcommand...
+{{{
+[tulaneID@cypress1 ~]$ rclone copy --help
+}}}
+8. To list the available flags used with the rclone command... (See also [[Box#BoxTransferRateLimits|Box Transfer Rate Limits]] below.)
+{{{
+[tulaneID@cypress1 ~]$ rclone help flags
+}}}
+
+==== Box Transfer Rate Limits ====
+
+Depending on the number of files contained in the <source> directory of your command {{{rclone copy <source> <destination>}}}, many of those files may fail to transfer with error '''429 Too Many Requests'''. To protect quality of service, Box accounts enforce the following transfer rate limits, though with 2 additional retries for each failed transfer request by default. (See [https://developer.box.com/guides/api-calls/permissions-and-errors/rate-limits Box Rate Limits].)
+
+1. 1000 API requests per minute, per user (for Box API requests other than file uploads)
+2. 240 file upload requests per minute, per user (for file uploads to Box specifically)
+
+To avoid exceeding the above mentioned rate limits, you can limit your HTTP transactions per second by including the rclone flag '''--tpslimit''' in your rclone command  as in the following. (See '''--tpslimit''' and other flags for rclone using the command {{{rclone help flags}}} for details.)
+1. '''Downloading files:''' To limit your rate (to 1000 requests per minute) of downloading files from a top level directory of your Tulane Box account to your current Cypress working directory...
+{{{
+[tulaneID@cypress1 ~]$ rclone copy TU-Box:<top-level-directory> . --tpslimit 16.6 # 1000 per minute or 16.6 per second
+}}}
+1. '''Uploading files:''' To limit your rate (to 240 requests per minute) of uploading files to a top level directory of your Tulane Box account from your desired Cypress directory...
+{{{
+[tulaneID@cypress1 ~]$ rclone copy TU-Box:<top-level-directory> <Cypress-directory> --tpslimit 16.6 # 1000 per minute or 16.6 per second
+}}}