Add warmup-phase for training with Horovod
Following Goyal et.al, 2018, a warm-up phase is required to maintain accuracy of the model that is trained over multiple GPU.
Following Goyal et.al, 2018, a warm-up phase is required to maintain accuracy of the model that is trained over multiple GPU.