SERGHEI-tutorial.md

@@ -132,7 +132,7 @@ would use 8 threads to run the compilation. Give this a try. If everything is co
 
 Let's run a simple test to check if everything is working fine. We will use an analytical dam break case, included with SERGHEI as a unit test.
 
-1. Navigate to `serghei/unitTests/ShallowWater/Analytical/dambreakX_4_1`
+1. Navigate to `serghei/unitTests/ShallowWater/Analytical/dambreakX_4_1`.
 2. You will see that this here contains an `input` directory, which contains a number of input files to define the simulation. At this stage, we will not be concerned with how this works. We will focus on that later.
 3. To run this case with SERGHEI, run
 
@@ -148,6 +148,57 @@ The command above should run a dam break simulation for 10 seconds. If all works
 
 If something goes wrong, check the paths, make sure you are in the right directory, and make sure that `serghei` was properly built (from the previous step).
 
-**IMPORTANT DISCLAIMER**
+**IMPORTANT**: Bare in mind that this is **not the correct** way to use the HPC system. By running the command above we ran a command in the ***login node*** of the HPC system, which is not meant to run simulations. We have only done this to quickly test if our build was ok, with an extremely small simulation, on a single CPU. To properly run a simulation we will use `slurm` and `sbatch` scripts in the next tutorial.
 
-Bare in mind that this is **not the correct** way to use the HPC system. By running the command above we ran a command in the ***login node*** of the HPC system, which is not meant to run simulations. We have only done this to quickly test if our build was ok, with an extremely small simulation, on a single CPU. To properly run a simulation we will use `slurm` and `sbatch` scripts in the next tutorial.
\ No newline at end of file
+#6. Run a case using sbatch
+
+We now want to launch this job using the compute nodes. Although this can be done through the command line (using the `srun` command, it is better to use an `sbatch` script to configure the job in an HPC system.
+
+A minimal `sbatch` script for SERGHEI looks like this below.
+
+```bash
+#!/bin/bash -x
+
+#SBATCH --job-name="SERGHEI"
+#SBATCH --account=##account_name##
+#SBATCH --time=00:05:00
+#SBATCH --ntasks-per-node=1      
+#SBATCH --cpus-per-task=##how_many?##
+#SBATCH --nodes=1
+#SBATCH --partition=##partition_to_use##
+
+##### until here, we have configured the HPC resources #####
+##### now we configure some additional goodies #####
+
+export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK}
+
+## load the modules
+source $SERGHEIPATH/jurecaDCModules      ## should match the ones you used to build
+
+## define the path to the simulation
+casePath="/the/path/to/the/input/files"
+
+## define some output folder name
+OUTDIR=output
+
+## remove the output folder (otherwise SERGHEI will prompt an error if it exists)
+rm -rf $casePath/$OUTDIR
+
+## launch the job in the HPC system
+srun $SERGHEIPATH/bin/serghei $casePath/input/ $casePath/$OUTDIR/ $OMP_NUM_THREADS
+
+```
+The first block (with the `SBATCH` keyword) informs the system of the HPC resources we want to use. It requires an account name to which the compute time will be billed against, a maximum job time, how many tasks we want per node, how many nodes we want and how many cpus we wish to use per task. Finally, since the HPC system is divided into **partitions** of nodes, we must specify which one we will use.
+
+For the problem we will run here, we will use the `dc-cpu-devel` partition, we will use all of the CPUs in a single node, and will only use one task, as all of our computational domain will be in the same node. 
+Configure the sbatch script with this information and save it in some reasonable place where you can find it. Remember to also update the `casePath` in the script to where the `input` folder lies.
+
+Finally, we can run this script and launch the job. 
+```
+sbatch my_sbatch_script
+```
+
+If things go well, you should get an output folder and inside it a `log.out` file. 
+You will also get a `slurm` job report showing everything that happened behind the scenes and not shown in the terminal while the job was launched in the compute nodes. Inspecting the contents of this file which also show if there was an error or a successful run. If you had errors, try troubleshooting through them.
+
+We will use this sbatch script as a base for further tutorials.
\ No newline at end of file