-
Utz-Uwe Haus authoredUtz-Uwe Haus authored
Prerequisites
You will need a C compiler
capable of understanding C11
, as well as posix threads
.
Also, you will need autoreconf
and libtool
for configuring Maestro build system.
deps/c-timestamp deps/libcyaml deps/libfabric deps/libyaml deps/mamba deps/mio deps/protobuf deps/protobuf-c
Subtree projects and updates
We are including a number of external dependency libraries as subtrees in our tree under ./deps/
and build these versions automatically with our make rules.
Maestro uses the libraries it built from the subtrees and not the versions of the libraries on your system.
Subtree | Remote branch |
---|---|
c-timestamp | git@github.com:chansen/c-timestamp.git |
libcyaml | git@github.com:tlsa/libcyaml.git master |
libfabric | git@github.com:ofiwg/libfabric.git v1.11.x |
libyaml | git@github.com:yaml/libyaml.git master |
mamba | git@gitlab.com:cerl/mamba.git master |
mio | git@gitlab.version.fz-juelich.de:10022/maestro/mio.git master |
protobuf | git@github.com:protocolbuffers/protobuf.git 3.10.x |
protobuf-c | git@github.com:protobuf-c/protobuf-c.git next |
liberl | git@gitlab.com:cerl/liberl.git maestro |
To update a dependency project (e.g. mamba) subtree please do
git subtree pull --prefix=deps/mamba --squash git@gitlab.com:cerl/mamba.git master
If git
refuses to do the subtree update insisting 'repo was never added', create a remote for the repo, e.g.,
git remote add -f ofiwg git@github.com:ofiwg/libfabric.git
and use that repo name for the subtree pull
command.
Building
For CRAY XC systems
Please load the following modules before configuring/building, to ensure libfabric
will be built properly.
module load rdma-credentials
module load gni-headers
Please note that DRC does not permit multiple drc_acquire() calls in the same job allocation. That is typically not a problem in real workflows, since the pool manager will be allocating the DRC token and other processes will only drc_access() it. HOWEVER, when you run the maestro-core tests in a single allocation, only the first one that uses DRC will succeed, and the later ones will all fail with a message like
mstro_drc_init(drc.c:196) Failed to drc_acquire a new credential: -28
The recommended way to run the tests is by allocating a job per test using the LOG_COMPILER
variable of
automake's test infrastructure:
env LOG_COMPILER="srun -n1 --exclusive -t 0:5:0 --pty" make -j2 check
This will start a single-node exclusive job for each individual test, and each will get a different DRC token.
jupiter system
Do not use one of the craype-network-ofi\*
modules at this time.
libfabric
builds nicely with CCE 9.0.
sage prototype
Please load the following modules before configuring/building.
module use $CLIENT_MOD_PATH
module swap gnu GCC/9.3.0
module load Autotools git binutils pkg-config libreadline
Calling autoreconf
autoreconf -ivf
Configuring
./configure
or specify the install directory, e.g.
./configure --prefix=$HOME/maestro
For CRAY systems
Be sure to use CC=cc
on the configure line.
Configuration options
By default, the OFI conductor will be built. To enable the experimental other variants
--enable-mpi-pool-manager
or --enable-smp-pool-manager
at configure time (INCOMPLETE).
We are also including Version 0.1.0 of the Mamba library in deps/mamba, and a snapshot of MIO in deps/mio.
To use MIO you need to add the --with-mio
flag at configure time, and have MERO installed on the system.
Depending on which branch you build, the default configure arguments may include options to enable the
gcc/llvm address sanitizer. To disable, please configure with --disable-asan
Build
Please use
make
If you have the address sanitizer enabled, that may (depending on the compiler) also turn on the leak checker. Since
we are not fully memory leak-free at this point, it may trigger exit-time failures. To disable it, set the environment variable
LSAN_OPTIONS=detect_leaks=0
Install
Please use
make install
Continuous integration
We are running gitlab-CI on the master, devel, and mvp branches. If you notice issues you can run the gitlab CI locally using docker and the gitlab-runner tool.
Install docker and gitlab-runner, then run
docker-machine start
eval $(docker-machine env)
gitlab-runner exec docker build
Configuration is picked up from .gitlab-ci.yml
If docker-machine start
fails try re-provisioning the docker machine, and/or reinstalling virtualbox.