Merge branch 'issue_2' into 'master'

Issue 2 (Proper data distribution with Horvod) +  Licensing

Closes #2

See merge request !2

.gitattributes

@@ -9,3 +9,7 @@ datasets/mnist/raw/t10k-images-idx3-ubyte.gz filter=lfs diff=lfs merge=lfs -text
 datasets/mnist/raw/t10k-labels-idx1-ubyte.gz filter=lfs diff=lfs merge=lfs -text
 datasets/mnist/raw/train-images-idx3-ubyte.gz filter=lfs diff=lfs merge=lfs -text
 datasets/mnist/raw/train-labels-idx1-ubyte.gz filter=lfs diff=lfs merge=lfs -text
+datasets/mnist/partitioned/train/x/*.npy filter=lfs diff=lfs merge=lfs -text
+datasets/mnist/partitioned/train/y/*.npy filter=lfs diff=lfs merge=lfs -text
+datasets/mnist/partitioned/test/x/*.npy filter=lfs diff=lfs merge=lfs -text
+datasets/mnist/partitioned/test/y/*.npy filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -117,4 +117,4 @@ mnist_convnet_model/
 
 # Error and output files from the supercomputers
 *.er
-*.out
\ No newline at end of file
+*.out

LICENSE

+All contents of this work, except for the contents of the "datasets/mnist"
+sub-directory are licensed under The MIT License (see license details below).
+Contents of the "datasets/mnist" sub-directory are licensed under the Creative
+Commons Attribution-ShareAlike 3.0 Unported License (see "datasets/mnist/LICENSE").
+
+MIT License
+
+Copyright (c) 2019 Forschungszentrum Juelich GmbH
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

NOTICE

+This project includes derived work from the following:
+
+
+Horovod
+Copyright 2018 Uber Technologies, Inc.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+
+Tensorflow
+Copyright 2016 The TensorFlow Authors.  All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+
+Keras
+All contributions by François Chollet:
+Copyright (c) 2015 - 2019, François Chollet.
+All rights reserved.
+
+All contributions by Google:
+Copyright (c) 2015 - 2019, Google, Inc.
+All rights reserved.
+
+All contributions by Microsoft:
+Copyright (c) 2017 - 2019, Microsoft, Inc.
+All rights reserved.
+
+All other contributions:
+Copyright (c) 2015 - 2019, the respective contributors.
+All rights reserved.
+
+Licensed under The MIT License (MIT)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -15,12 +15,17 @@ visit [this](https://gitlab.version.fz-juelich.de/MLDL_FZJ/MLDL_FZJ_Wiki/wikis/E
 
 ### Announcements
 
-*  Tensorflow and Keras examples (with and without Horovod) are now fully functional on JUWELS as well.
-*  Python 2 support has been removed from the tutorial for all frameworks except Caffe.
-*  Even though PyTorch is available as as system-wide module on the JSC supercomputers, all PyTorch 
+*  **November 18, 2019:** The `horovod_data_distributed` directory has been added that contains code 
+samples to illustrate proper data-distributed training with Horovod, i.e., a distribution mechanism 
+where the training data is distributed instead of epochs. Further information is available in the 
+directory-local `README.md`.
+*  **September 02, 2019:** Even though PyTorch is available as as system-wide module on the JSC supercomputers, all PyTorch 
 examples have been removed from this tutorial. This is due to the fact that the tutorial
 developers are not currently working with PyTorch, and are therefore not in a position to provide
 support for PyTorch related issues.
+*  **August 23, 2019:**
+   *  Tensorflow and Keras examples (with and without Horovod) are now fully functional on JUWELS as well.
+   *  Python 2 support has been removed from the tutorial for all frameworks except Caffe.
 
 # Table of contents
 <!-- TOC -->

datasets/mnist/LICENSE

+The mnist directory is licensed under the Creative Commons Attribution-ShareAlike 3.0 Unported License.
+To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/3.0/ or send
+a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

datasets/mnist/NOTICE

+The contents of the mnist directory are derived from the MNIST dataset:
+
+Yann LeCun (Courant Institute, NYU) and Corinna Cortes (Google Labs, New York)
+hold the copyright of MNIST dataset (http://yann.lecun.com/exdb/mnist), which is
+a derivative work from original NIST datasets. MNIST dataset is made available
+under the terms of the Creative Commons Attribution-Share Alike 3.0 license. The
+license details are available via the following URL:
+
+http://creativecommons.org/licenses/by-sa/3.0/
+
+Individual images and labels have not been changed in this work. The only changes
+made are to the dataset format.