Update Build authored by Yannik Müller's avatar Yannik Müller
\# Dependencies # Dependencies
\* A standard compliant c++ compiller \* A standard compliant c++ compiller
\* The system, compiler and implementations must support silent NaN floating-point values. ```plaintext
\* The system, compiler and implementations must support silent NaN floating-point values.
```
\* An MPI implementation (ideally CUDA aware) \* An MPI implementation (ideally CUDA aware)
...@@ -10,19 +12,25 @@ ...@@ -10,19 +12,25 @@
\* If using NVLink (cuda), Omni-Path (psm2), UCX (ucp), or IBVerbs (ibverbs) \* If using NVLink (cuda), Omni-Path (psm2), UCX (ucp), or IBVerbs (ibverbs)
\* \`minipmi\`: <https://github.com/kraused/minipmi> ```plaintext
\* \`minipmi\`: <https://github.com/kraused/minipmi>
```
\* If using NVLink (cuda): \* If using NVLink (cuda):
\* A standard compliant CUDA compiller ```plaintext
\* A standard compliant CUDA compiller
\* Python version 3.0.0 or a derivative \* Python version 3.0.0 or a derivative
```
\* If using linktest-report tool \* If using linktest-report tool
\* Python 3.8.5 or above ```plaintext
\* Python 3.8.5 or above
\* numpy and matplotlib \* numpy and matplotlib
```
\# Build procedure \# Build procedure
...@@ -32,7 +40,7 @@ In general you have to do the following steps ...@@ -32,7 +40,7 @@ In general you have to do the following steps
2\. Set-Up Environment, Pahs etc. 2\. Set-Up Environment, Pahs etc.
3\. Install \`linktest\` with \`make\`, disable unavailable transportlayers 3\. Install \`linktest\` with \`make\`, disable unavailable transportlayers
4\. Install \`linktest-report\` via pip (Update PYTHONPATH or use virtual environments) 4\. Install \`linktest-report\` via pip (Update PYTHONPATH or use virtual environments)
...@@ -44,25 +52,25 @@ Quick-Start Example installing only MPI and TCP support. (Might not work on your ...@@ -44,25 +52,25 @@ Quick-Start Example installing only MPI and TCP support. (Might not work on your
mkdir -p install mkdir -p install
ml GCC ParaStationMPI SIONlib SciPy-Stack # 1 + 2 ml GCC ParaStationMPI SIONlib SciPy-Stack # 1 + 2
cd benchmark; # cd benchmark; #
make HAVE_MINIPMI=0 PREFIX=../install clean install # 3 make HAVE_MINIPMI=0 PREFIX=../install clean install # 3
python -m pip install ../python/setup.py --prefix=../install # 4 python -m pip install ../python/setup.py --prefix=../install # 4
\`\`\` \`\`\`
exampleBuild.sh showcases a more complicated build procedure. You can adapt it to your needs. exampleBuild.sh showcases a more complicated build procedure. You can adapt it to your needs.
A file named \`pingpong_results_bin.sion\` should be produced. A file named \`pingpong_results_bin.sion\` should be produced.
Some examples for JSC systems can also be found in chapter \[JSC Build Examples\](#JSC-Build-Examples) Some examples for JSC systems can also be found in chapter [JSC Build Examples](#JSC-Build-Examples)
\## Transport Layers \## Transport Layers
Transport Layers can be (de-)activated with \`make HAVE_<LAYER>=(0)1\` Transport Layers can be (de-)activated with \`make HAVE_=(0)1\`
The following transport layers and other variables may be used: The following transport layers and other variables may be used:
...@@ -72,9 +80,9 @@ The following transport layers and other variables may be used: ...@@ -72,9 +80,9 @@ The following transport layers and other variables may be used:
| \`mpi\` | \`HAVE_MPI\` | Enabled | MPI | | \`mpi\` | \`HAVE_MPI\` | Enabled | MPI |
| \`tcp\` | \`HAVE_TCP\` | Enabled | TCP sockets, if deactivated, layers below will crash at runtime (see chapter \[Execution\](Execution)) | | \`tcp\` | \`HAVE_TCP\` | Enabled | TCP sockets, if deactivated, layers below will crash at runtime (see chapter [Execution](/cstao-public/linktest/-/wikis/Execution)) |
| | \`HAVE_MINIPMI\` | Enabled | If deactivated, deactivates all layers below | | | \`HAVE_MINIPMI\` | Enabled | If deactivated, deactivates all layers below |
| \`ibverbs\` | \`HAVE_IBVERBS\` | Enabled | Verbs-based implementation | | \`ibverbs\` | \`HAVE_IBVERBS\` | Enabled | Verbs-based implementation |
...@@ -84,7 +92,7 @@ The following transport layers and other variables may be used: ...@@ -84,7 +92,7 @@ The following transport layers and other variables may be used:
| \`cuda\` | \`HAVE_CUDA\` | Enabled | NVLink (node-internal) | | \`cuda\` | \`HAVE_CUDA\` | Enabled | NVLink (node-internal) |
| | \`CUARCH\` | | Needs to be set if \`cuda\` is enable | | | \`CUARCH\` | | Needs to be set if \`cuda\` is enable |
For \`CUARCH\` values see <https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#virtual-architecture-feature-list> For \`CUARCH\` values see <https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#virtual-architecture-feature-list>
...@@ -96,7 +104,7 @@ For \`CUARCH\` values see <https://docs.nvidia.com/cuda/cuda-compiler-driver-nvc ...@@ -96,7 +104,7 @@ For \`CUARCH\` values see <https://docs.nvidia.com/cuda/cuda-compiler-driver-nvc
| ----------- | ------- | | ----------- | ------- |
| Cluster | \`xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>>\` (uses the \`generic\` system settings) | | Cluster | \`xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>>\` (uses the \`generic\` system settings) |
| Booster | \`xenv -L CUDA -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>> SYSTEM="JUWELS Booster"\` | | Booster | \`xenv -L CUDA -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>> SYSTEM="JUWELS Booster"\` |
...@@ -106,7 +114,7 @@ For \`CUARCH\` values see <https://docs.nvidia.com/cuda/cuda-compiler-driver-nvc ...@@ -106,7 +114,7 @@ For \`CUARCH\` values see <https://docs.nvidia.com/cuda/cuda-compiler-driver-nvc
| ----------- | ------- | | ----------- | ------- |
| JURECA-DC CPU | \`xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 HAVE_CUDA=0 PREFIX_DIR=<<<PREFIX_DIR>>>\` (uses the \`generic\` system settings) | | JURECA-DC CPU | \`xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 HAVE_CUDA=0 PREFIX_DIR=<<<PREFIX_DIR>>>\` (uses the \`generic\` system settings) |
| JURECA-DC CPU | \`xenv -L CUDA -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>> SYSTEM="JURECA-DC GPU"\` | | JURECA-DC CPU | \`xenv -L CUDA -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>> SYSTEM="JURECA-DC GPU"\` |
...@@ -126,51 +134,59 @@ The \`GenericSystem\` instance can be customized via the environment, via a wrap ...@@ -126,51 +134,59 @@ The \`GenericSystem\` instance can be customized via the environment, via a wrap
export LINKTEST_SYSTEM_NODENAME_SUFFIX="i" export LINKTEST_SYSTEM_NODENAME_SUFFIX="i"
if \[\[ "0" == "$MPI_LOCALRANKID" \]\] ["0" == "$MPI_LOCALRANKID"](/cstao-public/linktest/-/wikis/%220%22%20==%20%22%24MPI_LOCALRANKID%22)
then then
export LINKTEST_SYSTEM_HCA_NAME="mlx5_0" ```plaintext
export LINKTEST_SYSTEM_HCA_NAME="mlx5_0"
export LINKTEST_SYSTEM_HCA_PORT="1" export LINKTEST_SYSTEM_HCA_PORT="1"
export LINKTEST_SYSTEM_GPU="0" export LINKTEST_SYSTEM_GPU="0"
```
fi fi
if \[\[ "1" == "$MPI_LOCALRANKID" \]\] ["1" == "$MPI_LOCALRANKID"](/cstao-public/linktest/-/wikis/%221%22%20==%20%22%24MPI_LOCALRANKID%22)
then then
export LINKTEST_SYSTEM_HCA_NAME="mlx5_1" ```plaintext
export LINKTEST_SYSTEM_HCA_NAME="mlx5_1"
export LINKTEST_SYSTEM_HCA_PORT="1" export LINKTEST_SYSTEM_HCA_PORT="1"
export LINKTEST_SYSTEM_GPU="1" export LINKTEST_SYSTEM_GPU="1"
```
fi fi
if \[\[ "2" == "$MPI_LOCALRANKID" \]\] ["2" == "$MPI_LOCALRANKID"](/cstao-public/linktest/-/wikis/%222%22%20==%20%22%24MPI_LOCALRANKID%22)
then then
export LINKTEST_SYSTEM_HCA_NAME="mlx5_2" ```plaintext
export LINKTEST_SYSTEM_HCA_NAME="mlx5_2"
export LINKTEST_SYSTEM_HCA_PORT="1" export LINKTEST_SYSTEM_HCA_PORT="1"
export LINKTEST_SYSTEM_GPU="2" export LINKTEST_SYSTEM_GPU="2"
```
fi fi
if \[\[ "3" == "$MPI_LOCALRANKID" \]\] ["3" == "$MPI_LOCALRANKID"](/cstao-public/linktest/-/wikis/%223%22%20==%20%22%24MPI_LOCALRANKID%22)
then then
export LINKTEST_SYSTEM_HCA_NAME="mlx5_3" ```plaintext
export LINKTEST_SYSTEM_HCA_NAME="mlx5_3"
export LINKTEST_SYSTEM_HCA_PORT="1" export LINKTEST_SYSTEM_HCA_PORT="1"
export LINKTEST_SYSTEM_GPU="3" export LINKTEST_SYSTEM_GPU="3"
```
fi fi
... ...
......