Data-distributed training of ML models
In scope of this branch, data-distributed training will be enabled.
While the training itself will be distributed with the help of the Horovod framework, the dataloader also have to be become distributable.
For the latter, sdifferent options will be explored using built-in methods such as tf.data.Dataset.shard
or other (manual) approaches.