Changes between Version 1 and Version 2 of cypress/data_retention

Timestamp:

06/05/25 16:13:37 (5 months ago)

Author:

cbaribault

Comment:

Added options for checking for older files

cypress/data_retention

@@ -9 +9 @@
 2. The user ID of Principal Investigator (PI) associated with the data is no longer active. 
 
-=== Data inactivity ===
+=== Data inactivity after 1 year ===
 
 If any given data on Lustre has not been read in 1 year or more - with the exceptions of directories **opt** and **conda-envs**, then the owner and their PI will receive an email notification regarding data inactivity with a 30-day response window before data access removal. All access to the given data will be removed for 30-days prior to purging. (For example, see last access times in the results of the commands **ls -l --time-style=iso** and **man ls**.)
@@ -21 +21 @@
 
 For further assistance, please submit a request for [https://tulane.service-now.com/tulaneit?id=sc_cat_item&sys_id=277b5905db004c10e8ce5bd2ca961945&sysparm_category=8d648d93dbc44c10f460562bdc9619ad&catalog_id=-1| Cypress HPC Consultation].
+
+=== How to check for older files ===
+
+To obtain the ages of all files in your group's directory under /lustre/project, you'll need read permissions for all of the (sub)directories containing those files.
+
+=== lfs find ===
+
+You can use the following command on Cypress to determine those files and subdirectories in your group's directory under /lustre/project for which the access time is (1) year (equivalently 365 days) or more.
+
+{{{
+# this can fail due to lack of read permissions
+lfs find --atime +365 /lustre/project/<group-id>
+}}}
+
+where <group-id> denotes your group-id - same as the Principal Investigator’s Tulane ID.
+
+In case you encounter a read permission error, the above command will abort before finishing.
+
+The resulting output will include the file path of the offending directory as well as the 
+following error message.
+
+{{{
+error: find failed for /lustre/project/<group-id>
+}}}
+
+For assistance with this error and read permissions, you can submit a request for [https://tulane.service-now.com/tulaneit?id=sc_cat_item&sys_id=277b5905db004c10e8ce5bd2ca961945&sysparm_category=8d648d93dbc44c10f460562bdc9619ad&catalog_id=-1| Cypress HPC Consultation]
+
+=== (Linux) find ===
+
+Alternatively, to find older files you can use the Linux [[cypress/BasicLinuxComands#find|find]] command as in the following - using the -ls option to include dates of last access, etc. in the result.
+
+{{{
+# this can be incomplete due to lack of read permissions
+find /lustre/project/<group-id> -atime +365 -ls
+}}}
+
+In case you encounter a read permission error, the above command will abort before finishing.
+
+The resulting output will include the file path of the offending directory as well as the 
+following error message.
+
+{{{
+Permission denied
+}}}
+
+For assistance with this error and read permissions, you can submit a request for [https://tulane.service-now.com/tulaneit?id=sc_cat_item&sys_id=277b5905db004c10e8ce5bd2ca961945&sysparm_category=8d648d93dbc44c10f460562bdc9619ad&catalog_id=-1| Cypress HPC Consultation]
+