README.md

Caution: Caffe is no longer being actively developed, which is why we prefer not to support it as a system-wide module on the supercomputers for long. This is why Caffe is available with Python 2 support only on JURECA, while it is not at all supported on JUWELS. The users are advised to switch to other frameworks such as Tensorflow/Keras and PyTorch.

Notes

There are three ways in which Caffe can be used,

1. As a command line tool with only built-in layers
2. As a library from within a Python program. Either only built-in layers can be used, or one or more custom layers can be written in Python.
3. As a command line tool with one or more custom C++ layers.

Caffe as a command line tool

The mnist_cmd sub-directory contains configuration and job scripts for running Caffe as a command line tool with only built-in layers. This example represents use case 1 as described above. The lenet_solver.prototxt and lenet_train_test.prototxt were taken from the MNIST examples directory available in the Caffe repository here. Minor changes have been made just so the path to the input dataset is correct. The caffe command in the job submission scripts can be modified as follows to run training on all available GPUs on the node (value for the -gpu option has been changed from 0 to all):

caffe train --solver=lenet_solver.prototxt -gpu all

Using Caffe within a Python program

The lenet_python sub-directory contains the required files for an example of using Caffe as a library from within a Python program. This corresponds to use case 2 as described above. The train_lenet.py file contains source code adapted from the IPython notebook 01-learning-lenet.ipynb available in the Caffe examples here. Running this example results in the generation of a learning curve plot in the current directory.

Caffe with custom C++ layers

Working with custom C++ layers requires recompiling Caffe with the custom code. As this is not possible with a system-wide installation, we have decided not to include an example of this use case. Nevertheless, if you must work with custom C++ layers and require assistance, please send an email to the JULAIN mailing list (more information here).