diff --git a/readme.md b/readme.md index 389256f5b8c2a9bf6b142608805dd969e223b8e3..9707c3693c62c2622e5c0799e2ad0cb59a80a44f 100644 --- a/readme.md +++ b/readme.md @@ -7,7 +7,7 @@ in the "normal" supercomputer workflow as well as in Jupyter-JSC. There are two flavors of this type of repo: 1. The "normal" version based on environment modules and a python virtual environment -2. The singularity version where a Singularity container replaces the module environment +2. The singularity version where a Singularity container replaces the module environment. Check out the branch `singularity`. The user creates a working environment as a fork of this repo, developes it along with the requirement of their project and finally archives or publishes it jointly with the results of the research project. @@ -22,14 +22,15 @@ their project and finally archives or publishes it jointly with the results of t 5. `source activate.sh` to enter the environment ## HowTo Singularity -1. Clone the repo into a folder with the name of the environment module `git clone /path/to/repo ./my_favourite_environment`. -2. Edit `config.sh` to select the singularity container. -3. If required, edit `setup.sh` to make changes to the container. This is possible as we support a persistent overlay file system -3. Edit `requirements.txt` to select the packages to be installed via pip -4. Create the environment +1. Get you favourite base image to use as starting point. Example: `singularity pull docker://nvcr.io/nvidia/pytorch:22.10-py3` +2. Clone the repo into a folder with the name of the environment module `git clone /path/to/repo ./my_favourite_environment`. +3. Edit `config.sh` to select the singularity container. +4. If required, edit `setup.sh` to make changes to the container. This is possible as we support a persistent overlay file system +5. Edit `requirements.txt` to select the packages to be installed via pip +6. Create the environment * execute `setup.sh` to create the virtual environment * execute `create_kernel.sh` to create a kernel for Jupyer-JSC -5. Make sure you execute `source activate.sh` each time you execute something within the container. +7. Make sure you execute `source activate.sh` each time you execute something within the container. # Description This project contains a lightweight set of scripts to easily create Python working environments on @@ -51,7 +52,7 @@ them into the workflow. This can e.g. mean that a compilation step is added in t setting appropriate environment variables is included in the activation step. ## Singularity -A useful alternative to environment modules are container-based environments. ToDo: More Details +A useful alternative to environment modules are container-based environments. On JUWELS Booster and JURECA DC, we highly recommend using NVIDIA's pre-built docker containers in the NGC https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch . Singularity can import docker images from public container registries with a command link `singularity pull docker://nvcr.io/nvidia/pytorch:22.10-py3`. This will automatically create a `.sif` file that you can use as starting point. # Details The setup is configured in the bash script `config.sh`. The user can define a name for the venv and directory