Changes between Version 8 and Version 9 of Workshops/JobCheckpointing/Examples

Timestamp:

04/07/2026 03:55:56 PM (4 months ago)

Author:

Carl Baribault

Comment:

Added Plotting section

Workshops/JobCheckpointing/Examples

@@ -474 +474 @@
 === Post-workflow analysis in Python ===
 
+==== Plotting execution time and memory usage ====
+
+Once the last in the series of [wiki://Workshops/JobCheckpointing/Examples#checkpoint_submitter.sh checkpoint_submitter.sh] jobs has detected the end of the computational workflow, it will call on the [wiki:///Workshops/JobCheckpointing/Examples#Pythonscripts Python scripts] listed below to perform a post-worflow analysis in order to plot execution time and memory usage (see output fields '''Elapsed''' and '''MaxRSS''' in '''man sacct''') and identify possible anomalies from among all of the [wiki://Workshops/JobCheckpointing/Examples#checkpoint_runner.sh checkpoint_runner] jobs that have participated in the workflow.
+
+The types of anomalies determined, if any are the following.
+
+ * | Z | > 3.5 for execution time, where
+
+{{{
+Z = (value - mean) / (standard deviation)
+}}}
+
+ * modZ > 3.5 for '''MaxRSS''' (SLURM max job memory usage), where
+
+{{{
+modZ = 0.6745 * | x - median | / (Median Absolute Deviation)
+}}}
+
+See also [https://www.itl.nist.gov/div898/handbook/eda/section3/eda35h.htm NIST - Detection of Outliers].
+
+==== Python scripts ====
+
 Here are two Python scripts used by the checkpoint submitter to analyze and plot usage and annotate anomalies in execution time and memory usage by the checkpoint runner jobs.