README.md

Singularity

In this guide, you will learn how to create a containerized environment that can be used on a supercomputer. We will explore three different methods for building Singularity images, starting with the simplest approach and progressing to more advanced techniques.

Method 1: Building a Singularity Image Using Specified Python Version and requirements.txt

Note: This method does not support environments requiring Conda packages.

1. Create a Base Singularity Image (base python version)
2. Install Dependencies (Create the Final Singularity Image)
3. Run the Containerized Environment

Create a Base Singularity Image (base python version)

Building your base singularity image is a crucial step in preparing your environment. If you are cloning a repository to create a Singularity image, it is essential to identify the required base Python version.

Once you have reviewed the repository details and determined the appropriate Python version, you can proceed to build the base Python image using the following command:

bash scripts/build_base_image.sh -A {account} -G {group_name} -p {partition} -V {python_version} -N {base_image_name}

• account: The account name for using supercomputers (you can find the name of the budget on JuDoor).
• group_name: The group name for using supercomputers (you can find the name of the Group on JuDoor). Sometimes account and group name are the same.
• partition: The partition name on the supercomputer. Please use a partition that has internet access (e.g., devel, develgpus, develbooster, all (JEDI), etc.).
• python_version: The Python version (e.g., 3.10).
• base_image_name: The name of the base image with a .sif extension (e.g., base.sif).

Here is the example how to build the base image:

bash scripts/build_base_image.sh -A atmlaml -G atmlaml -P devel -V 3.10 -N test.sif

After running scripts/build_base_image.sh, a SLURM job will be launched, and a report will be generated in the slurm_report path. Please make sure to review this report to confirm that your image was created without any errors.

The above command will create an singularity image in the following path:

/p/project1/{account}/$USER/apptainer/test.sif

Install Dependencies (Create the Final Singularity Image)

After creating the base image, you can use it to install the pip packages listed in requirements.txt in order to create your final image.

bash scripts/build_customized_image.sh -A {account} -G {group_name} -P {partition} -S {absolute_path_base_image} -R {absolute_path_requirements.txt}  -F {final_image_name}

• account: The account name for using supercomputers.
• group_name: The group name for using supercomputers (you can find the name of the Group on JuDoor). Sometimes account and group name are the same.
• partition: The partition name on the supercomputer. Please use a partition that has internet access (e.g., devel, develgpus, develbooster, etc.).
• absolute_path_base_image: The absolute path of the base image that was created before.
• absolute_path_requirements.txt: The absolute path of the requirements.txt that all containes all pip dependencies.
• final_image_name: The name of the final image with a .sif extension (e.g., final.sif).

Here is the example how to build the final image:

bash scripts/build_customized_image.sh -A atmlaml -G atmlaml -P devel -S /p/project1/atmlaml/{$USER}/apptainer/test.sif -R {path_to_requirements}/requirements.txt -F final.sif

Check slurm report in slurm_report to make sure the singularity image has been created sucessfully. Athe end of this report, if you see Hello world. This is my Python version cangradulation, you built the singularity image sucessfully.

You can find the final singularity image in the following path:

/p/project1/{account}/$USER/apptainer/final.sif

In General, If the containers from the NVIDIA container registry link are used as a base image you can be quite sure that you are as fast as it gets.

If you would like to use NVIDIA container as a base image, please go to the second method.

Method 2: Building a Singularity Image Using Specified NVIDIA container and requirements.txt

Note: This method does not support environments requiring Conda packages.

1. Create a Base Singularity Image (NVIDIA container)
2. Install Dependencies (Create the Final Singularity Image)
3. Run the Containerized Environment

Create a Base Singularity Image (NVIDIA container)

If you would like to install PyTorch, you can find the PyTorch NVIDIA container link. There are many PyTorch tags avaiable and you can choose your own and use it for building the base Nvidia image. If you are cloning a repository to create a Singularity image, it is essential to identify the PyTorch version. Once you find the proper PyTorch version, please select the proper NVIDIA PyTorch Container Versions using link. Once you have found the Container Version (for example 23.05), You can go this like link and find this version. After that please copy the path of this image (for example nvcr.io/nvidia/pytorch:23.05-py3). We will need this path to build our PyTorch NVIDIA container base singularity image.

you can proceed to build the base NVIDIA image using the following command:

bash scripts/build_base_nvidia_image.sh -A {account}  -G {group_name} -P {partition} -V {Nvidia_container_version} -N {base_image_name}

• account: The account name for using supercomputers.
• group_name: The group name for using supercomputers (you can find the name of the Group on JuDoor). Sometimes account and group name are the same.
• partition: The partition name on the supercomputer. Please use a partition that has internet access (e.g., devel, develgpus, develbooster, etc.).
• Nvidia_container_version: The Nvidia container (e.g., nvcr.io/nvidia/pytorch:23.05-py3).
• base_image_name: The name of the base image with a .sif extension (e.g., pytorch-23-05-py3.sif).

Here is the example how to build the base image:

bash scripts/build_base_nvidia_image.sh -A atmlaml -G atmlaml -P devel -V nvcr.io/nvidia/pytorch:23.05-py3 -N pytorch-23-05-py3.sif

After running scripts/build_base_nvidia_image.sh, a SLURM job will be launched, and a report will be generated in the slurm_report path. Please make sure to review this report to confirm that your image was created without any errors.

The above command will create an singularity image in the following path:

/p/project1/{account}/$USER/apptainer/pytorch-23-05-py3.sif

Install Dependencies (Create the Final Singularity Image)

After creating the base image, you can use it to install the pip packages listed in requirements.txt in order to create your final image. Note: Since you are using Nvidia container, PyTorch and other dependencies is already compiled in the base container. If there is a torch or torchvision inside of requirements.txt, please remove it in order to avoid conflict between the Nvidia PyTorch inside of the conatiner.

bash scripts/build_customized_nvidia_image.sh -A {account} -G {group_name} -P {partition} -S {absolute_path_base_image} -R {absolute_path_requirements.txt}  -F {final_image_name}

• account: The account name for using supercomputers.
• group_name: The group name for using supercomputers (you can find the name of the Group on JuDoor). Sometimes account and group name are the same.
• partition: The partition name on the supercomputer. Please use a partition that has internet access (e.g., devel, develgpus, develbooster, etc.).
• absolute_path_base_image: The absolute path of the base image that was created before.
• absolute_path_requirements.txt: The absolute path of the requirements.txt that all containes all pip dependencies.
• final_image_name: The name of the final image with a .sif extension (e.g., pytorch-23-05-py3-final.sif).

Here is the example how to build the final image:

bash scripts/build_customized_nvidia_image.sh -A atmlaml -G atmlaml -P devel -S /p/project1/atmlaml/{$USER}/apptainer/pytorch-23-05-py3.sif -R {path_to_requirements}/requirements.txt -F pytorch-23-05-py3-final.sif

Check slurm report in slurm_report to make sure the singularity image has been created sucessfully. Athe end of this report, if you see Hello world. This is my Python version cangradulation, you built the singularity image sucessfully.

You can find the final singularity image in the following path:

/p/project1/{account}/$USER/apptainer/pytorch-23-05-py3-final.sif

Congratulations!!! You have successfully created a singularity image. To use this image for your application, you can find an example submission script in example_submission_script.sh.

Method 3: Building a Singularity Image Using docke to singularity recipe conversion

It will be added soon!!!