Merge branch 'add_automatic_checkpoint_and_restart' of...

Merge branch 'add_automatic_checkpoint_and_restart' of https://gitlab.jsc.fz-juelich.de/opengptx/bigscience-code into add_automatic_checkpoint_and_restart

run_scripts/test

-2022-05-30 20:19:50.614599: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
-2022-05-30 20:19:50.614819: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (jwlogin24.juwels): /proc/driver/nvidia/version does not exist
-Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
-TensorBoard 2.6.0 at http://localhost:63743/ (Press CTRL+C to quit)