[[PageOutline]] == Data Retention on Lustre == === No backup service === In general, there is no backup service running on Cypress. For information on using your Tulane Box account as a backup for your data on Cypress, see [wiki://cypress/Globus Globus] or [wiki://cypress/Box Tulane Box account (via rclone).] === Loss of data access === Your can lose access to your data on Lustre in either of the following ways. 1. You fail to read the data for at least one year - with the exceptions of any (relative path) directories **opt** and **conda-envs**. 2. The user ID of Principal Investigator (PI) associated with the data is no longer active. === Data inactivity after 1 year === If any given data on Lustre has not been read in 1 year or more - with the exceptions of directories **opt** and **conda-envs**, then the owner and their PI will receive an email notification regarding data inactivity with a 30-day response window before data access removal. All access to the given data will be removed for 30-days prior to purging. (For example, see last access times in the results of the commands **ls -l --time-style=iso** and **man ls**.) Afterward, within the 30-day period, you can submit a request for [https://tulane.service-now.com/tulaneit?id=sc_cat_item&sys_id=277b5905db004c10e8ce5bd2ca961945&sysparm_category=8d648d93dbc44c10f460562bdc9619ad&catalog_id=-1| Cypress HPC Consultation] to have access to your data on Lustre restored. Your request will be completed subject to the approval of your Principal Investigator (PI). === Inactive Tulane Status of Principal Investigator === If the Tulane email/LDAP status of the PI associated with any given data on Lustre has become “inactive”, then action similar to the above will be taken for the given data on Lustre. For further assistance, please submit a request for [https://tulane.service-now.com/tulaneit?id=sc_cat_item&sys_id=277b5905db004c10e8ce5bd2ca961945&sysparm_category=8d648d93dbc44c10f460562bdc9619ad&catalog_id=-1| Cypress HPC Consultation]. === How to check for older files === To obtain the ages of all files in your group's directory under /lustre/project, you'll need read permissions for all of the (sub)directories containing those files. ==== lfs find ==== You can use the following command on Cypress to determine those files and subdirectories in your group's directory under /lustre/project for which the access time is (1) year (equivalently 365 days) or more. {{{ # this can fail due to lack of read permissions lfs find --atime +365 /lustre/project/ }}} where denotes your group-id - same as the Principal Investigator’s Tulane ID. In case you encounter a read permission error, the above command will abort before finishing. The resulting output will include the file path of the offending directory as well as the following error message. {{{ error: find failed for /lustre/project/ }}} For assistance with this error and read permissions, you can submit a request for [https://tulane.service-now.com/tulaneit?id=sc_cat_item&sys_id=277b5905db004c10e8ce5bd2ca961945&sysparm_category=8d648d93dbc44c10f460562bdc9619ad&catalog_id=-1| Cypress HPC Consultation] ==== (Linux) find ==== Alternatively, to find older files you can use the Linux [[cypress/BasicLinuxComands#find|find]] command as in the following - using the -ls option to include dates of last access, etc. in the result. {{{ # this can be incomplete due to lack of read permissions find /lustre/project/ -atime +365 -ls }}} In case you encounter a read permission error, the above command will abort before finishing. The resulting output will include the file path of the offending directory as well as the following error message. {{{ Permission denied }}} For assistance with this error and read permissions, you can submit a request for [https://tulane.service-now.com/tulaneit?id=sc_cat_item&sys_id=277b5905db004c10e8ce5bd2ca961945&sysparm_category=8d648d93dbc44c10f460562bdc9619ad&catalog_id=-1| Cypress HPC Consultation] === Lustre Quota Review === Lustre quotas are adjusted at regular intervals to reflect actual usage, allowing for up to 10% over current storage consumption. For example, if your group is currently using 30 TB of a 70 TB quota, your revised allocation will be adjusted to approximately 33 TB (30 TB + 10%). For any changes to a group's existing quota, the group's Principal Investigator (PI) will receive an email notice for that change. ==== Review Cadence ==== Quota adjustments are performed at regular intervals depending on the quota level as follows. * '''1 TB''': No PI group will receive less than 1 TB of storage on Lustre. * '''8 TB or less''': For groups with 8 TB or less, reviews will be conducted annually. * '''More than 8 TB''': For groups with allocations exceeding 8 TB, reviews will occur quarterly. We appreciate your cooperation as we continue improving the efficiency and availability of HPC resources. Please feel free to reach out with any questions or if your group's storage needs are expected to change significantly in the near term.