Changes between Version 12 and Version 13 of Workshops/JobCheckpointing/Examples


Ignore:
Timestamp:
04/09/2026 04:42:17 PM (3 months ago)
Author:
Carl Baribault
Comment:

Added overview and execution diagram

Legend:

Unmodified
Added
Removed
Modified

  • Workshops/JobCheckpointing/Examples

    v12 v13  
    11[[PageOutline]]
    22= Job Checkpointing Examples =
     3
     4== Overview ==
     5
     6This example implements a '''submitter/runner''' (also called cyclic checkpoint/restart) pattern.
     7
     8A lightweight '''submitter''' job launches a '''runner''' job that executes the '''application''' for a bounded time.
     9
     10After that time, the application writes a checkpoint, and exits cleanly.
     11
     12The submitter monitors progress and resubmits a new submitter job as needed until the workflow completes. See [wiki://Workshops/JobCheckpointing/Examples#Checkpointworkflowimplementation Checkpoint workflow implementation] below.
    313
    414== Before you adopt checkpointing ==
    … …  
    4757
    4858== Checkpoint workflow implementation ==
     59
     60=== Execution Model ===
     61
     62{{{
     63+-----------+               +-----------+             +---------------+
     64| Submitter |               |  Runner   |             |  Application  |
     65|  (job)    |               |   (job)   |             |   (process)   |
     66+-----------+               +-----------+             +---------------+
     67      |                           |                           |
     68      |  if Application not done, |                           |
     69      |  submit(sbatch)           |                           |
     70      | ------------------------> |                           |
     71      |  otherwise, exit          |        run                |
     72      |                           | ------------------------> |
     73      |                           |                           |
     74      | ----+                     |    SIGTERM @ timeout      |
     75      |     | wait for self       | ------------------------> |
     76      |     | timeout             |                           | ----+
     77      |     |                     |                           |     |
     78      |     |                     |                           |     | handle SIGTERM
     79      |     |                     |                           |     | & write checkpoint
     80      |     |                     |                           | <---+
     81      |     |                     |           exit code       |
     82      |     |                     + <-------------------------+
     83      |     |        Runner exit (progress / done / failure)
     84      | <---+
     85      |
     86      | update workflow state
     87      | submit new Submitter
     88      | interpret Runner exit code
     89      + exit
     90
     91(Submitter state persisted e.g. to submitter_state.txt)
     92}}}
    4993
    5094=== Components explained ===