Skip to content
Snippets Groups Projects
Commit 0ededaec authored by Fahad Khalid's avatar Fahad Khalid
Browse files

Updates corresponding to the renaming of the hpc4ns package to hpc4neuro.

parent 27f29076
No related branches found
No related tags found
No related merge requests found
No preview for this file type
......@@ -4,8 +4,8 @@ Please see the main docstring in each program for details.
# Notes
On JURECA and JUWELS, the `mnist_data_distributed.py` program requires the [`hpc4ns.distribution`](
https://gitlab.version.fz-juelich.de/hpc4ns/hpc4ns_utils#1-hpc4nsdistribution)
On JURECA and JUWELS, the `mnist_data_distributed.py` program requires the [`hpc4neuro.distribution`](
https://gitlab.version.fz-juelich.de/hpc4ns/hpc4neuro#1-hpc4neurodistribution)
module for distribution of training data filenames across multiple ranks. On JURON, multiple additional
package are required. Please follow the steps below to setup the environment before submitting the
training job.
......@@ -17,9 +17,9 @@ are eight training files.
1. Change to the source directory for this sample, i.e., to `dl_on_supercomputers/horovod_data_distributed`
2. Load the system-wide Python module: `module load Python/3.6.8`
3. Install the `hpc4ns` package:
3. Install the `hpc4neuro` package:
`pip install --user git+https://gitlab.version.fz-juelich.de/hpc4ns/hpc4ns_utils.git`
`pip install --user git+https://gitlab.version.fz-juelich.de/hpc4ns/hpc4neuro.git`
4. Submit the job
......
......@@ -22,8 +22,8 @@ import tensorflow as tf
import horovod.tensorflow.keras as hvd
from tensorflow.python.keras import backend as K
from hpc4ns.errors import MpiInitError
from hpc4ns.distribution import DataDistributor
from hpc4neuro.errors import MpiInitError
from hpc4neuro.distribution import DataDistributor
sys.path.insert(0, '../utils')
from data_utils import DataValidator
......@@ -93,7 +93,7 @@ def initialize_hvd_and_mpi():
are no conflicts between Horovod and mpi4py communicator
initialization.
:exception: hpc4ns.errors.MpiInitError is raised in the case
:exception: hpc4neuro.errors.MpiInitError is raised in the case
of initialization failure.
"""
......
......@@ -4,10 +4,10 @@
module load python/3.6.1
# Create a virtual environment
python -m venv venv_dl_hpc4ns
python -m venv venv_dl_hpc4neuro
# Activate the virtual environment
source venv_dl_hpc4ns/bin/activate
source venv_dl_hpc4neuro/bin/activate
# Upgrade pip and setuptools
pip install -U pip setuptools
......@@ -18,7 +18,7 @@ env MPICC=/gpfs/software/opt/openmpi/3.1.2-gcc_5.4.0-cuda_10.0.130/bin/mpicc pip
# Install six
pip install six
# Install hpc4ns
pip install git+https://gitlab.version.fz-juelich.de/hpc4ns/hpc4ns_utils.git
# Install hpc4neuro
pip install git+https://gitlab.version.fz-juelich.de/hpc4ns/hpc4neuro.git
printf "%s\n\n" "Setup complete."
......@@ -15,7 +15,7 @@ module load tensorflow/1.12.0-gcc_5.4.0-cuda_10.0.130
module load horovod/0.15.2
# Activate the virtual environment
source venv_dl_hpc4ns/bin/activate
source venv_dl_hpc4neuro/bin/activate
# Run the program
mpirun -bind-to none \
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment