Changes

Page history
Update Build with disabled Layer defaults. authored by Yannik Müller's avatar Yannik Müller
Show whitespace changes
Inline Side-by-side
Build.md
View page @ cc91c50a
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
* The system, compiler and implementations must support silent NaN floating-point values. * The system, compiler and implementations must support silent NaN floating-point values.
* An MPI implementation (ideally CUDA aware) * An MPI implementation (ideally CUDA aware)
* SIONlib: http://fz-juelich.de/ias/jsc/sionlib * SIONlib: http://fz-juelich.de/ias/jsc/sionlib
* If using NVLink (cuda), Omni-Path (psm2), UCX (ucp), or IBVerbs (ibverbs) * If using NVLink (cuda), Omni-Path (psm2), UCX (ucp), TCP(tcp), or IBVerbs (ibverbs)
* `minipmi`: https://github.com/kraused/minipmi * `minipmi`: https://github.com/kraused/minipmi
* If using NVLink (cuda): * If using NVLink (cuda):
* A standard compliant CUDA compiler * A standard compliant CUDA compiler
...@@ -19,48 +19,44 @@ ...@@ -19,48 +19,44 @@
In general you have to do the following steps In general you have to do the following steps
1. Install Dependencies 1. Install Dependencies
2. Set-Up Environment, Pahs etc. 2. Set-Up Environment, Pahs etc.
3. Install `linktest` with `make`, disable unavailable transport layers 3. Install `linktest` with `make`, enable transport layers other than mpi
4. [Optional] Install `linktest-report` via pip (Update PYTHONPATH or use virtual environments) 4. [Optional] Install `linktest-report` via pip (Update PYTHONPATH or use virtual environments)
5. [Optional] Install `pingponganalysis` with `make` 5. [Optional] Install `pingponganalysis` with `make`
Quick-Start-Example installing Linktest with only MPI and TCP support. (Might not work on your system) Quick-Start-Example installing Linktest with only MPI support. (Might not work on your system)
```bash ```bash
mkdir -p install mkdir -p install
cd benchmark cd benchmark
ml GCC ParaStationMPI SIONlib SciPy-Stack # 1 + 2 ml GCC ParaStationMPI SIONlib SciPy-Stack # 1 + 2
make HAVE_MINIPMI=0 PREFIX=../install clean install # 3 make PREFIX=../install clean install # 3
python -m pip install ../python/setup.py --prefix=../install # 4 python -m pip install ../python/setup.py --prefix=../install # 4
``` ```
# Transport Layers # Build Variables / Transport Layers
Transport Layers can be (de-)activated with `make HAVE_<LAYER>=(0)1` Transport Layers can be (de-)activated with `make HAVE_<LAYER>=(0)1`
All Layers other than mpi require the tcp Layer and MiniPMI to be available.
The following transport layers and other variables may be used: The following transport layers and other variables may be used:
| Layer | Variable | Default | Description | | Layer | Variable | Default | Description |
| ----- | -------- | ------- | ----------- | | ----- | -------- | ------- | ----------- |
| `mpi` | `HAVE_MPI` | Enabled | MPI | | `mpi` | `HAVE_MPI` | Enabled | MPI |
| `tcp` | `HAVE_TCP` | Enabled | TCP sockets, if deactivated, layers below will crash at runtime (see chapter [Execution](Execution)) | | `tcp` | `HAVE_TCP` | Disabled | Automatically enabled if any Layer below is enabled |
| | `HAVE_MINIPMI` | Enabled | If deactivated, deactivates all layers below | | `ibverbs` | `HAVE_IBVERBS` | Disabled| Verbs-based implementation |
| `ibverbs` | `HAVE_IBVERBS` | Enabled | Verbs-based implementation | | `psm2` | `HAVE_PSM2` | Disabled| PSM2 (Omni-Path) |
| `psm2` | `HAVE_PSM2` | Enabled | PSM2 (Omni-Path) | | `ucp` | `HAVE_UCP` | Disabled| UCX (using high level api ucp) |
| `ucp` | `HAVE_UCP` | Enabled | UCX (using high level api ucp) | | `cuda` | `HAVE_CUDA` | Disabled| NVLink (node-internal) |
| `cuda` | `HAVE_CUDA` | Enabled | NVLink (node-internal) |
| | `CUARCH` | | Needs to be set if `cuda` is enable | | | `CUARCH` | | Needs to be set if `cuda` is enable |
| | `V` | | If defined prints detailed steps and additional variables |
| | `SYSTEM` | `generic` | Use predefined custom affinities for HCAs and GPUs |
| | `PREFIX` | `/usr/local/bin` | Install directory for binaries |
For `CUARCH` values see https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#virtual-architecture-feature-list For `CUARCH` values see https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#virtual-architecture-feature-list
# JSC Build Examples # JSC Build Examples (TODO Update)
**JUWELS:**
| Partitition | Command |
| ----------- | ------- |
| Cluster | `xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>>` (uses the `generic` system settings) |
| Booster | `xenv -L CUDA -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>> SYSTEM="JUWELS Booster"` |
**JURECA:**
| Partitition | Command | | Partitition | Command |
| ----------- | ------- | | ----------- | ------- |
| Juwels Cluster and Juwels Booster | `xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_TCP=1 HAVE_IBVERBS=1 HAVE_UCP=1 PREFIX=<<<DIR>>>`|
| JURECA-DC CPU | `xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 HAVE_CUDA=0 PREFIX_DIR=<<<PREFIX_DIR>>>` (uses the `generic` system settings) | | JURECA-DC CPU | `xenv -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 HAVE_CUDA=0 PREFIX_DIR=<<<PREFIX_DIR>>>` (uses the `generic` system settings) |
| JURECA-DC CPU | `xenv -L CUDA -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>> SYSTEM="JURECA-DC GPU"` | | JURECA-DC CPU | `xenv -L CUDA -L GCC -L ParaStationMPI -L SIONlib make install HAVE_PSM2=0 PREFIX_DIR=<<<PREFIX_DIR>>> SYSTEM="JURECA-DC GPU"` |
| Booster | `xenv -L Architecture/KNL -L Intel -L IntelMPI -L SIONlib make install HAVE_CUDA=0 PREFIX_DIR=<<<PREFIX_DIR>>>` | | Booster | `xenv -L Architecture/KNL -L Intel -L IntelMPI -L SIONlib make install HAVE_CUDA=0 PREFIX_DIR=<<<PREFIX_DIR>>>` |
... ...
......