Source: https://blog.rwth-aachen.de/itc-events/files/2022/02/7_Multi-GPU_Horovod.pdf