* @jitsev1: Have a look on our workshop's tutorials, especially [Day 2 Tutorial 2](https://gitlab.version.fz-juelich.de/MLDL_FZJ/juhaicu/jsc_public/sharedspace/teaching/intro_scalable_dl_2021/course-material/-/tree/master/tutorials/day2/tutorial2) that gives intro into "Horovodifying" single node code for multi node execution
* In general, our [Intro into Scalable Deep Learning Course](https://gitlab.version.fz-juelich.de/MLDL_FZJ/juhaicu/jsc_public/sharedspace/teaching/intro_scalable_dl_2021/course-material/) is a good way to start (Horovod things come from Day 2 on) <br>
There are also code examples there how to run training on our HPC machines with Horovod on multiple nodes
* A short, concise tutorial on converting single GPU training code for distributed execution on multi-node supercomputers: [Horovod data parallel training tutorial](https://gitlab.version.fz-juelich.de/MLDL_FZJ/juhaicu/jsc_public/sharedspace/code/distributed_dl/-/tree/master/horovod_tutorial)
* A short, concise tutorial on converting single GPU training code for distributed execution on multi-node supercomputers by @cherti1: [Horovod data parallel training tutorial](https://gitlab.version.fz-juelich.de/MLDL_FZJ/juhaicu/jsc_public/sharedspace/code/distributed_dl/-/tree/master/horovod_tutorial)
