Data Retention on Lustre
Your can lose access to your data on Lustre in either of the following ways.
- You fail to read the data for at least one year - with the exceptions of any (relative path) directories opt and conda-envs.
- The user ID of Principal Investigator (PI) associated with the data is no longer active.
Data inactivity after 1 year
If any given data on Lustre has not been read in 1 year or more - with the exceptions of directories opt and conda-envs, then the owner and their PI will receive an email notification regarding data inactivity with a 30-day response window before data access removal. All access to the given data will be removed for 30-days prior to purging. (For example, see last access times in the results of the commands ls -l —time-style=iso and man ls.)
Afterward, within the 30-day period, you can submit a request for Cypress HPC Consultation to have access to your data on Lustre restored. Your request will be completed subject to the approval of your Principal Investigator (PI).
Inactive Tulane Status of Principal Investigator
If the Tulane email/LDAP status of the PI associated with any given data on Lustre has become “inactive”, then action similar to the above will be taken for the given data on Lustre.
For further assistance, please submit a request for Cypress HPC Consultation.
How to check for older files
To obtain the ages of all files in your group's directory under /lustre/project, you'll need read permissions for all of the (sub)directories containing those files.
lfs find
You can use the following command on Cypress to determine those files and subdirectories in your group's directory under /lustre/project for which the access time is (1) year (equivalently 365 days) or more.
# this can fail due to lack of read permissions lfs find --atime +365 /lustre/project/<group-id>
where <group-id> denotes your group-id - same as the Principal Investigator’s Tulane ID.
In case you encounter a read permission error, the above command will abort before finishing.
The resulting output will include the file path of the offending directory as well as the following error message.
error: find failed for /lustre/project/<group-id>
For assistance with this error and read permissions, you can submit a request for Cypress HPC Consultation
(Linux) find
Alternatively, to find older files you can use the Linux find command as in the following - using the -ls option to include dates of last access, etc. in the result.
# this can be incomplete due to lack of read permissions find /lustre/project/<group-id> -atime +365 -ls
In case you encounter a read permission error, the above command will abort before finishing.
The resulting output will include the file path of the offending directory as well as the following error message.
Permission denied
For assistance with this error and read permissions, you can submit a request for Cypress HPC Consultation
