Changes
Page history
Update Build
authored
Jul 29, 2021
by
Yannik Müller
Hide whitespace changes
Inline
Side-by-side
Build.md
View page @
f44078d1
\
#
Dependencies
# Dependencies
\*
A standard compliant c++ compiller
\* The system, compiler and implementations must support silent NaN floating-point values.
```
plaintext
\* The system, compiler and implementations must support silent NaN floating-point values.
```
\*
An MPI implementation (ideally CUDA aware)
...
...
@@ -10,19 +12,25 @@
\*
If using NVLink (cuda), Omni-Path (psm2), UCX (ucp), or IBVerbs (ibverbs)
\* \`minipmi\`: <https://github.com/kraused/minipmi>
```
plaintext
\* \`minipmi\`: <https://github.com/kraused/minipmi>
```
\*
If using NVLink (cuda):
\* A standard compliant CUDA compiller
```
plaintext
\* A standard compliant CUDA compiller
\* Python version 3.0.0 or a derivative
\* Python version 3.0.0 or a derivative
```
\*
If using linktest-report tool
\* Python 3.8.5 or above
```
plaintext
\* Python 3.8.5 or above
\* numpy and matplotlib
\* numpy and matplotlib
```
\#
Build procedure
...
...
@@ -32,7 +40,7 @@ In general you have to do the following steps
2
\.
Set-Up Environment, Pahs etc.
3
\.
Install
\`
linktest
\`
with
\`
make
\`
, disable unavailable transportlayers
3
\.
Install
\`
linktest
\`
with
\`
make
\`
, disable unavailable transportlayers
4
\.
Install
\`
linktest-report
\`
via pip (Update PYTHONPATH or use virtual environments)
...
...
@@ -44,25 +52,25 @@ Quick-Start Example installing only MPI and TCP support. (Might not work on your
mkdir -p install
ml GCC ParaStationMPI SIONlib SciPy-Stack
# 1 + 2
ml GCC ParaStationMPI SIONlib SciPy-Stack # 1 + 2
cd benchmark;
#
cd benchmark;
#
make HAVE_MINIPMI=0 PREFIX=../install clean install
# 3
make HAVE_MINIPMI=0 PREFIX=../install clean install # 3
python -m pip install ../python/setup.py --prefix=../install
# 4
python -m pip install ../python/setup.py --prefix=../install # 4
\`\`\`
exampleBuild.sh showcases a more complicated build procedure. You can adapt it to your needs.
exampleBuild.sh showcases a more complicated build procedure. You can adapt it to your needs.
A file named
\`
pingpong_results_bin.sion
\`
should be produced.
A file named
\`
pingpong_results_bin.sion
\`
should be produced.
Some examples for JSC systems can also be found in chapter
\
[
JSC Build Examples
\
]
(#JSC-Build-Examples)
Some examples for JSC systems can also be found in chapter
[
JSC Build Examples
](
#JSC-Build-Examples
)
\#
# Transport Layers
Transport Layers can be (de-)activated with
\`
make HAVE_
<LAYER>
=(0)1
\`
Transport Layers can be (de-)activated with
\`
make HAVE_=(0)1
\`
The following transport layers and other variables may be used:
...
...
@@ -72,9 +80,9 @@ The following transport layers and other variables may be used:
|
\`
mpi
\`
|
\`
HAVE_MPI
\`
| Enabled | MPI |
|
\`
tcp
\`
|
\`
HAVE_TCP
\`
| Enabled | TCP sockets, if deactivated, layers below will crash at runtime (see chapter
\
[
Execution
\
]
(Execution)) |
|
\`
tcp
\`
|
\`
HAVE_TCP
\`
| Enabled | TCP sockets, if deactivated, layers below will crash at runtime (see chapter
[
Execution
](
/cstao-public/linktest/-/wikis/
Execution
)
) |
|
|
\`
HAVE_MINIPMI
\`
| Enabled | If deactivated, deactivates all layers below |
| |
\`
HAVE_MINIPMI
\`
| Enabled | If deactivated, deactivates all layers below |
|
\`
ibverbs
\`
|
\`
HAVE_IBVERBS
\`
| Enabled | Verbs-based implementation |
...
...
@@ -84,7 +92,7 @@ The following transport layers and other variables may be used:
|
\`
cuda
\`
|
\`
HAVE_CUDA
\`
| Enabled | NVLink (node-internal) |
|
|
\`
CUARCH
\`
|
| Needs to be set if
\`
cuda
\`
is enable |
| |
\`
CUARCH
\`
| | Needs to be set if
\`
cuda
\`
is enable |
For
\`
CUARCH
\`
values see
<https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#virtual-architecture-feature-list>
...
...
@@ -96,7 +104,7 @@ For \`CUARCH\` values see <https://docs.nvidia.com/cuda/cuda-compiler-driver-nvc
| ----------- | ------- |
| Cluster |
\`
xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=
<
<<
PREFIX_DIR
>
>>
\`
(uses the
\`
generic
\`
system settings) |
| Cluster |
\`
xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=
<
<<
PREFIX_DIR
>
>>
\`
(uses the
\`
generic
\`
system settings) |
| Booster |
\`
xenv -L CUDA -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=
<
<<
PREFIX_DIR
>
>> SYSTEM="JUWELS Booster"
\`
|
...
...
@@ -106,7 +114,7 @@ For \`CUARCH\` values see <https://docs.nvidia.com/cuda/cuda-compiler-driver-nvc
| ----------- | ------- |
| JURECA-DC CPU |
\`
xenv
-L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 HAVE_CUDA=0 PREFIX_DIR=
<
<<
PREFIX_DIR
>
>>
\`
(uses the
\`
generic
\`
system settings) |
| JURECA-DC CPU |
\`
xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 HAVE_CUDA=0 PREFIX_DIR=
<
<<
PREFIX_DIR
>
>>
\`
(uses the
\`
generic
\`
system settings) |
| JURECA-DC CPU |
\`
xenv -L CUDA -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=
<
<<
PREFIX_DIR
>
>> SYSTEM="JURECA-DC GPU"
\`
|
...
...
@@ -126,51 +134,59 @@ The \`GenericSystem\` instance can be customized via the environment, via a wrap
export LINKTEST_SYSTEM_NODENAME_SUFFIX="i"
if
\[\[
"0" == "$MPI_LOCALRANKID"
\]\]
[
"0" == "$MPI_LOCALRANKID"
](
/cstao-public/linktest/-/wikis/%220%22%20==%20%22%24MPI_LOCALRANKID%22
)
then
export LINKTEST_SYSTEM_HCA_NAME="mlx5_0"
```
plaintext
export LINKTEST_SYSTEM_HCA_NAME="mlx5_0"
export LINKTEST_SYSTEM_HCA_PORT="1"
export LINKTEST_SYSTEM_HCA_PORT="1"
export LINKTEST_SYSTEM_GPU="0"
export LINKTEST_SYSTEM_GPU="0"
```
fi
if
\[\[
"1" == "$MPI_LOCALRANKID"
\]\]
[
"1" == "$MPI_LOCALRANKID"
](
/cstao-public/linktest/-/wikis/%221%22%20==%20%22%24MPI_LOCALRANKID%22
)
then
export LINKTEST_SYSTEM_HCA_NAME="mlx5_1"
```
plaintext
export LINKTEST_SYSTEM_HCA_NAME="mlx5_1"
export LINKTEST_SYSTEM_HCA_PORT="1"
export LINKTEST_SYSTEM_HCA_PORT="1"
export LINKTEST_SYSTEM_GPU="1"
export LINKTEST_SYSTEM_GPU="1"
```
fi
if
\[\[
"2" == "$MPI_LOCALRANKID"
\]\]
[
"2" == "$MPI_LOCALRANKID"
](
/cstao-public/linktest/-/wikis/%222%22%20==%20%22%24MPI_LOCALRANKID%22
)
then
export LINKTEST_SYSTEM_HCA_NAME="mlx5_2"
```
plaintext
export LINKTEST_SYSTEM_HCA_NAME="mlx5_2"
export LINKTEST_SYSTEM_HCA_PORT="1"
export LINKTEST_SYSTEM_HCA_PORT="1"
export LINKTEST_SYSTEM_GPU="2"
export LINKTEST_SYSTEM_GPU="2"
```
fi
if
\[\[
"3" == "$MPI_LOCALRANKID"
\]\]
[
"3" == "$MPI_LOCALRANKID"
](
/cstao-public/linktest/-/wikis/%223%22%20==%20%22%24MPI_LOCALRANKID%22
)
then
export LINKTEST_SYSTEM_HCA_NAME="mlx5_3"
```
plaintext
export LINKTEST_SYSTEM_HCA_NAME="mlx5_3"
export LINKTEST_SYSTEM_HCA_PORT="1"
export LINKTEST_SYSTEM_HCA_PORT="1"
export LINKTEST_SYSTEM_GPU="3"
export LINKTEST_SYSTEM_GPU="3"
```
fi
...
...
...
...
