Changes
Page history
Update Build with disabled Layer defaults.
authored
Aug 26, 2021
by
Yannik Müller
Hide whitespace changes
Inline
Side-by-side
Build.md
View page @
cc91c50a
...
...
@@ -6,7 +6,7 @@
*
The system, compiler and implementations must support silent NaN floating-point values.
*
An MPI implementation (ideally CUDA aware)
*
SIONlib: http://fz-juelich.de/ias/jsc/sionlib
*
If using NVLink (cuda), Omni-Path (psm2), UCX (ucp), or IBVerbs (ibverbs)
*
If using NVLink (cuda), Omni-Path (psm2), UCX (ucp),
TCP(tcp),
or IBVerbs (ibverbs)
*
`minipmi`
: https://github.com/kraused/minipmi
*
If using NVLink (cuda):
*
A standard compliant CUDA compiler
...
...
@@ -19,49 +19,45 @@
In general you have to do the following steps
1.
Install Dependencies
2.
Set-Up Environment, Pahs etc.
3.
Install
`linktest`
with
`make`
,
disable unavail
able transport layers
3.
Install
`linktest`
with
`make`
,
en
able transport layers
other than mpi
4.
[Optional] Install
`linktest-report`
via pip (Update PYTHONPATH or use virtual environments)
5.
[Optional] Install
`pingponganalysis`
with
`make`
Quick-Start-Example installing Linktest with only MPI
and TCP
support. (Might not work on your system)
Quick-Start-Example installing Linktest with only MPI support. (Might not work on your system)
```
bash
mkdir
-p
install
cd
benchmark
ml GCC ParaStationMPI SIONlib SciPy-Stack
# 1 + 2
make
HAVE_MINIPMI
=
0
PREFIX
=
../install clean
install
# 3
make
PREFIX
=
../install clean
install
# 3
python
-m
pip
install
../python/setup.py
--prefix
=
../install
# 4
```
# Transport Layers
Transport Layers can be (de-)activated with
`make HAVE_<LAYER>=(0)1`
# Build Variables / Transport Layers
Transport Layers can be (de-)activated with
`make HAVE_<LAYER>=(0)1`
All Layers other than mpi require the tcp Layer and MiniPMI to be available.
The following transport layers and other variables may be used:
| Layer | Variable | Default | Description |
| ----- | -------- | ------- | ----------- |
|
`mpi`
|
`HAVE_MPI`
| Enabled | MPI |
|
`tcp`
|
`HAVE_TCP`
| Enabled | TCP sockets, if deactivated, layers below will crash at runtime (see chapter
[
Execution
](
Execution
)
) |
| |
`HAVE_MINIPMI`
| Enabled | If deactivated, deactivates all layers below |
|
`ibverbs`
|
`HAVE_IBVERBS`
| Enabled | Verbs-based implementation |
|
`psm2`
|
`HAVE_PSM2`
| Enabled | PSM2 (Omni-Path) |
|
`ucp`
|
`HAVE_UCP`
| Enabled | UCX (using high level api ucp) |
|
`cuda`
|
`HAVE_CUDA`
| Enabled | NVLink (node-internal) |
|
`tcp`
|
`HAVE_TCP`
| Disabled | Automatically enabled if any Layer below is enabled |
|
`ibverbs`
|
`HAVE_IBVERBS`
| Disabled| Verbs-based implementation |
|
`psm2`
|
`HAVE_PSM2`
| Disabled| PSM2 (Omni-Path) |
|
`ucp`
|
`HAVE_UCP`
| Disabled| UCX (using high level api ucp) |
|
`cuda`
|
`HAVE_CUDA`
| Disabled| NVLink (node-internal) |
| |
`CUARCH`
| | Needs to be set if
`cuda`
is enable |
| |
`V`
| | If defined prints detailed steps and additional variables |
| |
`SYSTEM`
|
`generic`
| Use predefined custom affinities for HCAs and GPUs |
| |
`PREFIX`
|
`/usr/local/bin`
| Install directory for binaries |
For
`CUARCH`
values see https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#virtual-architecture-feature-list
# JSC Build Examples
**JUWELS:**
# JSC Build Examples (TODO Update)
| Partitition | Command |
| ----------- | ------- |
| Cluster |
`xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>>`
(uses the
`generic`
system settings) |
| Booster |
`xenv -L CUDA -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>> SYSTEM="JUWELS Booster"`
|
**JURECA:**
| Partitition | Command |
| ----------- | ------- |
| JURECA-DC CPU |
`xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 HAVE_CUDA=0 PREFIX_DIR=<<<PREFIX_DIR>>>`
(uses the
`generic`
system settings) |
| Juwels Cluster and Juwels Booster |
`xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_TCP=1 HAVE_IBVERBS=1 HAVE_UCP=1 PREFIX=<<<DIR>>>`
|
| JURECA-DC CPU |
`xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 HAVE_CUDA=0 PREFIX_DIR=<<<PREFIX_DIR>>>`
(uses the
`generic`
system settings) |
| JURECA-DC CPU |
`xenv -L CUDA -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>> SYSTEM="JURECA-DC GPU"`
|
| Booster |
`xenv -L Architecture/KNL -L Intel -L IntelMPI -L SIONlib make install HAVE_CUDA=0 PREFIX_DIR=<<<PREFIX_DIR>>>`
|
...
...
...
...