[[PageOutline]]

= Submitting Jobs from MATLAB Command Window =
This document provides the steps to configure MATLAB to submit jobs to Cypress, retrieve results, and debug errors.
There are two ways to do this, one is [https://wiki.hpc.tulane.edu/trac/wiki/Matlab-SlurmOnCypress 'submitting MATLAB jobs from MATLAB running on Cypress'], another is [https://wiki.hpc.tulane.edu/trac/wiki/Matlab-Slurm#MATLABclientonyourlocalcomputer 'submitting MATLAB jobs from your laptop/desktop'].


== MATLAB client on your local computer ==
With this method, you can submit MATLAB jobs to Cypress from your local computer.

[[Image(https://docs.google.com/drawings/d/e/2PACX-1vTo_5IOHBbnAZbhvUUoEFcX3AMtZ8wC1GZXmdlfl-pFk-AQGW_V0SjXX8QojF6OmAeizyMMNslFQSyL/pub?w=755&h=200)]]

=== Prerequisites ===
* You need to have MATLAB installed on your laptop/desktop which must be the same version of MATLAB on Cypress.
* Currently R2020a and R2022b are available on Cypress. 
* You must have the Matlab '''Parallel Computing Toolbox''' installed.  To see what toolboxes are installed, type the '''ver''' command in the Matlab command window.  You should see '''Parallel Computing Toolbox''' listed.
* Make sure you have an account on Cypress and can ''ssh'' to Cypress from your local laptop/desktop.

=== Installation and Configuration ===
In order to submit MATLAB jobs to Cypress from your laptop/desktop, you need to install custom Matlab plugin scripts that are configured to interact with the Slurm job scheduler on Cypress.  Use the link below to download the script package to your laptop/desktop. 

[https://wiki.hpc.tulane.edu/trac/attachment/wiki/Matlab-Slurm/TU.nonshared.R2022b.zip TU.nonshared.R2022b.zip]

Download the archive file and start MATLAB.  The archive file should be unzipped in the location returned by calling

{{{
>> userpath
}}}

See [https://www.mathworks.com/help/matlab/ref/userpath.html here] to view or change the default user work folder on MATLAB.



==== Run the configCluster.m Script to Create a Cluster Profile ====
Configure MATLAB to run parallel jobs on your cluster by calling '''configCluster'''.  This only needs to be called '''once''' per version of MATLAB.

{{{
>> configCluster
}}}

Submission to Cypress requires SSH credentials.  You will be prompted for your ssh username and password or identity file (private key).  The username and location of the private key will be stored in MATLAB for future sessions.


== Configuring Jobs ==
Prior to submitting the job, we can specify various parameters to pass to our jobs, such as queue, e-mail, walltime, etc. 

{{{
>> % Get a handle to the cluster
>> c = parcluster;
}}}

If you are using R2022b or later, you must set '''!QueueName''' to '''centos7'''.
{{{
>> % Set partition to centos7
>> c.AdditionalProperties.QueueName='centos7';
}}}

=== Optional ===
{{{
>> % Specify e-mail address to receive notifications about your job
>> c.AdditionalProperties.EmailAddress = 'user-id@tulane.edu';
}}}

{{{
>> % Specify Debug mode 
>> c.AdditionalProperties.EnableDebug = 'false';
}}}

{{{
>> % Specify memory to use for MATLAB jobs, per core (MB)
>> c.AdditionalProperties.MemUsage = '4000';
}}}

{{{
>> % Specify processors per node (maximum 20 on Cypress)
>> c.AdditionalProperties.ProcsPerNode = 20;
}}}

{{{
>> % Specify Quality of Service
>> c.AdditionalProperties.QoS = 'long';
}}}
The default is ''normal''. See [https://wiki.hpc.tulane.edu/trac/wiki/cypress/about#SLURMresourcemanager here] for other options.

{{{
>> % Specify the walltime (e.g. 5 hours)
>> c.AdditionalProperties.WallTime = '05:00:00';
}}}
See [https://wiki.hpc.tulane.edu/trac/wiki/cypress/about#SLURMresourcemanager here] for the maximum walltime.

{{{
>> c.AdditionalProperties.RemoteJobStorageLocation='/lustre/project/group/userid/MATLAB'
}}}
Location to store job files on the cluster. The default is your home directory but it may be better to change it to ''lustre'' file system.

----


Save changes after modifying !AdditionalProperties for the above changes to persist between MATLAB sessions.
{{{
>> c.saveProfile
}}}

To see the values of the current configuration options, display !AdditionalProperties.
{{{
>> % To view current properties
>> c.AdditionalProperties
}}}


Unset a value when no longer needed.
{{{
>> % Turn off email notifications 
>> c.AdditionalProperties.EmailAddress = '';
>> c.saveProfile
}}}


== INDEPENDENT BATCH JOB ==
Use the {{{batch}}} command to submit asynchronous jobs to the cluster.  The batch command will return a job object which is used to access the output of the submitted job.  See the MATLAB documentation for more help on [https://www.mathworks.com/help/parallel-computing/batch.html batch].

=== Running Serial Job ===
Let’s use the following example for a serial job, which is saved as {{{serial_example.m}}}.   
{{{
% Serial Example
disp('Start sim')

t0 = tic;
for idx = 1:8
     A(idx) = idx;
     pause(2)
end
t = toc(t0);

disp('Sim Completed')
}}}


{{{
>> % Get a handle to the cluster
>> c = parcluster;
}}}

{{{
>> % Below, submit a batch job that calls the 'mywave.m' script.
>> % Also set the parameter AutoAddClientPath to false so that Matlab won't complain when paths on 
>> % your desktop don't exist on the cluster compute nodes (this is expected and can be ignored).

>> myjob = batch(c,'serial_example','AutoAddClientPath',false)
}}}

*** If you get a pop-up window 'Use identity file to login to cypress.tulane.edu', press 'NO' ***

{{{
>> % Wait for the job to finish.  
>> wait(myjob)
}}}

{{{
>> % display the job diary (This is the Matlab standard output text, if any)
>> diary(myjob)
--- Start Diary ---
Start sim
Sim Completed

--- End Diary ---
}}}

{{{
>> % load the 'A' array (computed in serial_example) from the results of job 'myjob':
>> load(myjob,'A');
>> A

A =

     1     2     3     4     5     6     7     8
}}}


{{{
>> % Query job for state
>> myjob.State
}}}

{{{
>> % If state is finished, fetch the results
>> myjob.fetchOutputs{:}
ans = 

  struct with fields:

      A: [1 2 3 4 5 6 7 8]
    ans: [1×1 struct]
    idx: 8
      t: 16.0127
     t0: 1626209674755060
}}}
{{{
>> % Delete the job after results are no longer needed
>> myjob.delete
}}}
To retrieve a list of currently running or completed jobs, call parcluster to retrieve the cluster object.  The cluster object stores an array of jobs that were run, are running, or are queued to run.  This allows us to fetch the results of completed jobs.  Retrieve and view the list of jobs as shown below.

{{{
>> c = parcluster;
>> jobs = c.Jobs;
}}}

Once we’ve identified the job we want, we can retrieve the results as we’ve done previously. 
fetchOutputs is used to retrieve function output arguments; if calling batch with a script, use load instead.   Data that has been written to files on the cluster needs be retrieved directly from the file system (e.g. via ftp).
To view results of a previously completed job:

{{{
>> % Get a handle to the job with ID 2
>> j2 = c.Jobs(2);
}}}

NOTE: You can view a list of your jobs, as well as their IDs, using the above c.Jobs command.  

{{{
>> % Fetch results for job with ID 2
>> j2.fetchOutputs{:}
}}}

=== Running Parallel Job ===
Users can also submit parallel workflows with the batch command.  Let’s use the following example for a parallel job, which is saved as {{{parallel_example.m}}} that uses the '''parfor''' statement to parallelize the '''for''' loop

{{{
disp('Start sim')

t0 = tic;
parfor idx = 1:8
     A(idx) = idx;
     pause(2)
end
t = toc(t0);

disp('Sim Completed')
}}}

In the next example, we will run a parallel job using 8 processors on a single node.
This time when we use the batch command, to run a parallel job, we’ll also specify a MATLAB Pool.  
  
{{{ 
>> % Get a handle to the cluster
>> c = parcluster;
}}}

{{{
>> % Submit a batch pool job using 8 workers for 8 iterations
>> myjob = batch(c,'parallel_example','pool', 8, 'AutoAddClientPath',false)
}}}

*** If you get a pop-up window 'Use identity file to login to cypress.tulane.edu', press 'NO' ***

{{{
>> % View current job status
>> myjob.State
}}}

{{{
>> % Fetch the results after a finished state is retrieved
>> myjob.fetchOutputs{:}
ans = 

  struct with fields:

     A: [1 2 3 4 5 6 7 8]
     t: 2.3438
    t0: 1626210921701584
}}}

The job ran in 2.3438 seconds using eight workers.  '''Note that these jobs will always request N+1 CPU cores''', since one worker is required to manage the batch job and pool of workers.   For example, a job that needs eight workers will consume nine CPU cores.  	


==== retrieve the results later ====
{{{
>> % Get the job ID
>> id = j.ID
id =
	4
}}}

{{{
>> % Clear myjob from workspace (as though we quit MATLAB)
>> clear myjob
}}}

Once we have a handle to the cluster, we’ll call the findJob method to search for the job with the specified job ID.   

{{{
>> % Get a handle to the cluster
>> c = parcluster;
}}}

{{{
>> % Find the old job
>> myjob = c.findJob('ID', 4);
}}}

{{{
>> % Retrieve the state of the job
>> myjob.State
ans = 
finished
}}}

{{{
>> % Fetch the results
>> myjob.fetchOutputs{:};

ans = 

  struct with fields:

     A: [1 2 3 4 5 6 7 8]
     t: 2.3438
    t0: 1626210921701584
}}}

Alternatively, to retrieve job results via a graphical user interface, use the Job Monitor (Parallel > Monitor Jobs).

== DEBUGGING ==
If a serial job produces an error, call the getDebugLog method to view the error log file.  When submitting independent jobs, with multiple tasks, specify the task number.  
{{{
>> c.getDebugLog(j.Tasks(3))
}}}
For Pool jobs, only specify the job object.
{{{
>> c.getDebugLog(j)
}}}
When troubleshooting a job, the cluster admin may request the scheduler ID of the job.  This can be derived by calling schedID
{{{
>> schedID(j)
ans = 
25539
}}}


== TO LEARN MORE ==
To learn more about the MATLAB Parallel Computing Toolbox, check out these resources:
* [http://www.mathworks.com/products/parallel-computing/code-examples.html Parallel Computing Coding Examples]
* [http://www.mathworks.com/help/distcomp/index.html Parallel Computing Documentation]
* [http://www.mathworks.com/products/parallel-computing/index.html Parallel Computing Overview]
* [http://www.mathworks.com/products/parallel-computing/tutorials.html Parallel Computing Tutorials]
* [http://www.mathworks.com/products/parallel-computing/videos.html Parallel Computing Videos]
* [http://www.mathworks.com/products/parallel-computing/webinars.html Parallel Computing Webinars]