SIONlib issueshttps://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues2022-08-02T09:59:01+02:00https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/233Version 1.7.7 still reports as 1.7.6 & the SIONlib 32-bit Endianness identifi...2022-08-02T09:59:01+02:00Max HolickiVersion 1.7.7 still reports as 1.7.6 & the SIONlib 32-bit Endianness identification flag is 0Morning, could someone try generating a 1.7.7 SIONlib file please and have a look at the binary header. It looks like version 1.7.7:
1.) Does not set the endianness identification bytes.
2.) Still has the version set as 1.7.6.
I kind of...Morning, could someone try generating a 1.7.7 SIONlib file please and have a look at the binary header. It looks like version 1.7.7:
1.) Does not set the endianness identification bytes.
2.) Still has the version set as 1.7.6.
I kind of need those 4 bytes to reliably figure out if the program is little, big or mixed endian.
When I do module load SIONlib/1.7.7 I get this incorrect behavior in produced files.https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/232Memory leak when opening files in read only2022-03-26T12:10:51+01:00Max HolickiMemory leak when opening files in read onlyWhen sanitizing memory access for Linktest we noticed that reading from a SION file with SIONlib 1.7.7 causes a memory leak.
```
Direct leak of 1024 byte(s) in 1 object(s) allocated from:
#0 0x563dbe582378 in __interceptor_malloc (/h...When sanitizing memory access for Linktest we noticed that reading from a SION file with SIONlib 1.7.7 causes a memory leak.
```
Direct leak of 1024 byte(s) in 1 object(s) allocated from:
#0 0x563dbe582378 in __interceptor_malloc (/home/holicki/linktest/SIONtest.exe+0xf5378)
#1 0x563dbe5e0b33 in _sion_get_multi_filename /home/holicki/linktest/SIONlib/build-linux-gomp-openmpi/build/lib/sion_internal.c:915
#2 0x563dbe5c7314 in sion_generic_paropen /home/holicki/linktest/SIONlib/build-linux-gomp-openmpi/build/parlib/sion_generic.c:529
#3 0x563dbe5c40a7 in sion_paropen_mpi /home/holicki/linktest/SIONlib/build-linux-gomp-openmpi/build/parlib/sion_mpi_gen.c:205
```
It has to do with the last parameter of `sion_generic_paropen()`, which is a `char` pointer. The memory that this pointer points to is not deleted. If this is the users responsibility then this should be stated explicitly in the documentation. Otherwise SIONlib should clean this up when closing the file.
Note this problem does not always occur when writing.
This is still present in the newer SIONlib versions.Tom RidleyTom Ridleyhttps://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/231Tests on RB-1.7 insist on installing CMake2021-03-06T01:43:37+01:00Benedikt Steinbuschb.steinbusch@fz-juelich.deTests on RB-1.7 insist on installing CMakeWhy? Stop it!Why? Stop it!Benedikt Steinbuschb.steinbusch@fz-juelich.deBenedikt Steinbuschb.steinbusch@fz-juelich.dehttps://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/230Documentation for scatter callback confuses role of in and out buffers2021-03-05T17:41:01+01:00Benedikt Steinbuschb.steinbusch@fz-juelich.deDocumentation for scatter callback confuses role of in and out buffersReported by @mueller24 via Rocket Chat.Reported by @mueller24 via Rocket Chat.Benedikt Steinbuschb.steinbusch@fz-juelich.deBenedikt Steinbuschb.steinbusch@fz-juelich.dehttps://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/229CUDA aware read/write no longer compile with CUDA 112021-03-06T00:06:44+01:00Benedikt Steinbuschb.steinbusch@fz-juelich.deCUDA aware read/write no longer compile with CUDA 11The `isManaged` and `memoryType` fields of `cudaPointerAttributes` have been removed/renamed in CUDA 11. A backward compatible solution would need to dispatch based on `CUDART_VERSION`.The `isManaged` and `memoryType` fields of `cudaPointerAttributes` have been removed/renamed in CUDA 11. A backward compatible solution would need to dispatch based on `CUDART_VERSION`.https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/228Integrate gcov(r)2021-05-26T10:11:05+02:00Benedikt Steinbuschb.steinbusch@fz-juelich.deIntegrate gcov(r)Specify `CTEST_COVERAGE_COMMAND`? Bonus points for making GitLab pick up the coverage metric.Specify `CTEST_COVERAGE_COMMAND`? Bonus points for making GitLab pick up the coverage metric.https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/227Integrate clang-tidy2021-05-26T10:11:43+02:00Benedikt Steinbuschb.steinbusch@fz-juelich.deIntegrate clang-tidyCMake can run clang-tidy alongside the compiler by, e.g., setting `CMAKE_C_CLANG_TIDY`. Use `find_program(clang-tidy)` or `find_package(Clang)` to find `clang-tidy`'s location, then uncover the eldritch horrors that lurk in the depths.CMake can run clang-tidy alongside the compiler by, e.g., setting `CMAKE_C_CLANG_TIDY`. Use `find_program(clang-tidy)` or `find_package(Clang)` to find `clang-tidy`'s location, then uncover the eldritch horrors that lurk in the depths.https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/226Build documentation through CMake2021-05-06T21:17:53+02:00Benedikt Steinbuschb.steinbusch@fz-juelich.deBuild documentation through CMakeCMake includes a `FindDoxygen` module that provides a convenience function `doxygen_add_docs()`. Use it!CMake includes a `FindDoxygen` module that provides a convenience function `doxygen_add_docs()`. Use it!https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/225src/parlib/sion_lock.c is dead code2021-02-12T09:56:40+01:00Benedikt Steinbuschb.steinbusch@fz-juelich.desrc/parlib/sion_lock.c is dead codeRemove it!Remove it!https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/224OMP and OMPI use OpenMP locks incorrectly2021-02-01T19:13:30+01:00Benedikt Steinbuschb.steinbusch@fz-juelich.deOMP and OMPI use OpenMP locks incorrectlyIn `sion_paropen_omp` and `sion_paropen_ompi`, user defined locks based on OpenMP's `omp_lock_t` are registered via `sion_lock_register_lock_callbacks`. The lock instance is initialized in a global variable and while the access is "prote...In `sion_paropen_omp` and `sion_paropen_ompi`, user defined locks based on OpenMP's `omp_lock_t` are registered via `sion_lock_register_lock_callbacks`. The lock instance is initialized in a global variable and while the access is "protected" against data races by a `master` construct, no effort is made to protect against repeated initialization. (And also `omp_destroy_lock` is never called.) As a consequence, any program that calls `paropen_omp` or `paropen_ompi` more than once does not conform to the OpenMP specification.https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/223Tool version symbol renaming mechanism vs. different name mangling schemes2021-02-01T07:57:04+01:00Benedikt Steinbuschb.steinbusch@fz-juelich.deTool version symbol renaming mechanism vs. different name mangling schemesThe symbol renaming mechanism for the "tool" version relies on the fact that function names become symbol names, e.g. the function `sion_close` becomes a symbol `sion_close`. This does not work on platforms where this is not the case, e....The symbol renaming mechanism for the "tool" version relies on the fact that function names become symbol names, e.g. the function `sion_close` becomes a symbol `sion_close`. This does not work on platforms where this is not the case, e.g. Darwin, where an underscore is prepended, `_sion_close`.https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/222Tool version symbol renaming mechanism vs. ifdef2021-02-01T07:46:43+01:00Benedikt Steinbuschb.steinbusch@fz-juelich.deTool version symbol renaming mechanism vs. ifdefThe symbol renaming mechanism used to construct the "tool" version of SIONlib builds a regular version of the library and then works on a list of symbols produced via `nm`. This can miss symbols which are only available when a certain op...The symbol renaming mechanism used to construct the "tool" version of SIONlib builds a regular version of the library and then works on a list of symbols produced via `nm`. This can miss symbols which are only available when a certain option is turned on, e.g. IME Native integration.
Possible workaround: always include all functions, only `ifdef` function bodies?https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/221GPFS hints do not compile2021-04-13T15:15:55+02:00Benedikt Steinbuschb.steinbusch@fz-juelich.deGPFS hints do not compilesion_hints_gpfs.{h,c} had not been touched in a while and does not compile. Testing is possible with new CMake build system e.g. on JUWELS where GPFS libraries are available.sion_hints_gpfs.{h,c} had not been touched in a while and does not compile. Testing is possible with new CMake build system e.g. on JUWELS where GPFS libraries are available.https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/220Tests buddy__{1,2} and fitest_{1,2} slow on CI2021-01-31T16:52:16+01:00Benedikt Steinbuschb.steinbusch@fz-juelich.deTests buddy__{1,2} and fitest_{1,2} slow on CIAlso in my local docker, so in both cases tests are running on docker in virtualisation. Native runs on macOS seem to be OK.Also in my local docker, so in both cases tests are running on docker in virtualisation. Native runs on macOS seem to be OK.https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/219Fortran bindings Makefile does not honor Fortran compiler flags2020-06-29T12:05:58+02:00Benedikt Steinbuschb.steinbusch@fz-juelich.deFortran bindings Makefile does not honor Fortran compiler flagsBorked in 87e2f395daa90b2e0db8a30e6f7272729d4d3b14, partially fixed (for F77 files) in f4b2b3fa9cacb85b64ccc577aa83108d96c24a14Borked in 87e2f395daa90b2e0db8a30e6f7272729d4d3b14, partially fixed (for F77 files) in f4b2b3fa9cacb85b64ccc577aa83108d96c24a14Benedikt Steinbuschb.steinbusch@fz-juelich.deBenedikt Steinbuschb.steinbusch@fz-juelich.dehttps://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/218Does not build with GFortran 102020-07-01T17:58:19+02:00Benedikt Steinbuschb.steinbusch@fz-juelich.deDoes not build with GFortran 10The diagnostics in gfortran 10 have been made more strict and refuse mismatched types between dummy and actual arguments. A new command line argument `-fallow-argument-mismatch` must be passed to gfortran to turn of this diagnostic. See ...The diagnostics in gfortran 10 have been made more strict and refuse mismatched types between dummy and actual arguments. A new command line argument `-fallow-argument-mismatch` must be passed to gfortran to turn of this diagnostic. See https://gcc.gnu.org/gcc-10/porting_to.htmlBenedikt Steinbuschb.steinbusch@fz-juelich.deBenedikt Steinbuschb.steinbusch@fz-juelich.dehttps://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/217Recursive variables in serial tests Makefile2020-05-13T13:44:03+02:00Benedikt Steinbuschb.steinbusch@fz-juelich.deRecursive variables in serial tests MakefileI'm building SION 1.7.6 and got these message from the regression tests:
```
cd build; make -f RealMakefile test
......
++++++++++ Test Serial ++++++++++++++++++++++++
make[5]: Entering directory '/gpfs/fs1/SHARE/Utils/S...I'm building SION 1.7.6 and got these message from the regression tests:
```
cd build; make -f RealMakefile test
......
++++++++++ Test Serial ++++++++++++++++++++++++
make[5]: Entering directory '/gpfs/fs1/SHARE/Utils/SION/1.7.6/PGI-20.4_CUDA-10.2.89.0_440.33.01_OpenMPI-4.0.4rc1/distro/build-linux-pgi-openmpi-64/build/test/serial'
pgcc -c -c99 -m64 -g -D_SION_LINUX -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D__USE_LARGEFILE64 -D__USE_FILE_OFFSET64 -g -I. -I../../lib -I../../parlib -D_SION_LINUX test_seropn_1.c
Makefile:16: *** Recursive variable 'CXXFLAGS' references itself (eventually). Stop.
make[5]: Leaving directory '/gpfs/fs1/SHARE/Utils/SION/1.7.6/PGI-20.4_CUDA-10.2.89.0_440.33.01_OpenMPI-4.0.4rc1/distro/build-linux-pgi-openmpi-64/build/test/serial'
RealMakefile:92: recipe for target 'all-test-ser' failed
make[4]: [all-test-ser] Error 2 (ignored)
++++++++++ TEST Serial ++++++++++++++++++++++++
make[5]: Entering directory '/gpfs/fs1/SHARE/Utils/SION/1.7.6/PGI-20.4_CUDA-10.2.89.0_440.33.01_OpenMPI-4.0.4rc1/distro/build-linux-pgi-openmpi-64/build/test/serial'
Makefile:16: *** Recursive variable 'CXXFLAGS' references itself (eventually). Stop.
make[5]: Leaving directory '/gpfs/fs1/SHARE/Utils/SION/1.7.6/PGI-20.4_CUDA-10.2.89.0_440.33.01_OpenMPI-4.0.4rc1/distro/build-linux-pgi-openmpi-64/build/test/serial'
RealMakefile:332: recipe for target 'test-ser' failed
make[4]: [test-ser] Error 2 (ignored)
```
The line in question in this
```
16 CXXFLAGS = $(CXXFLAGS) -I../../cppinterface
17 FFLAGS += -g
```
and I think the preferred syntax would be
```
16 CXXFLAGS += -I../../cppinterface
17 FFLAGS += -g
```
similar in structure to the line below it.
Reported by Carl Ponder via e-mail cponder@nvidia.comBenedikt Steinbuschb.steinbusch@fz-juelich.deBenedikt Steinbuschb.steinbusch@fz-juelich.dehttps://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/216Change to CMake build system2021-02-12T09:53:48+01:00Benedikt Steinbuschb.steinbusch@fz-juelich.deChange to CMake build systemReported by Julien Bigot <julien.bigot@cea.fr>:
> While installing SIONlib on my system (debian sid) I faced the following
> issue:
> on line 171, 172 of config/select-mpi.sh , the script is looking for a
> directory ${MLIBDIR}/openmp...Reported by Julien Bigot <julien.bigot@cea.fr>:
> While installing SIONlib on my system (debian sid) I faced the following
> issue:
> on line 171, 172 of config/select-mpi.sh , the script is looking for a
> directory ${MLIBDIR}/openmpi that is absent on my system, instead I have
> ${MLIBDIR}/openmpi3
I have seen different related issues with recent Debian derived distros that have changed the file system layout w.r.t. where library files are placed. A sustainable solution involves moving to a third-party build system (i.e. #99 or similar) that includes auto-detect functionality that we do not have to maintain by ourselves.
**Update:** More reports of the build system not working as expected are coming in.
* JB reports he has to patch the build system to properly handle staged installs.
* JSC's SIONlib EasyConfigs patch the build system to recognize an MPI library that has been in use for over a decade on JSC systems.
* New employees cannot get SIONlib to build out of the box, because they explicitly have to specify an MPI configuration that is there, but is not documented and is not picked up automatically.
#99 is more than six years old now. `DEV-CF-autotools` last had activity more than three years ago and that was a merge from the `main` branch (then `trunk`) and the creation of an empty directory.
I have created `DEV-BS-cmake` to explore the use of a CMake based build system. So far, I have prototyped building the libraries and some (serial and parallel) utilities, building and running the test suite through `ctest`. What is there now is in an early state and certainly a number of things are still missing, such as
* detection and use of optional dependencies (IME, SIONfwd),
* installation,
* source package creation (gotta hide them secrets),
* creation of the `tools` version,
* etc.
On the other hand, it already offers benefits:
* auto-detection for MPI and OpenMP that works and does not have to be maintained by us,
* integration with modern tooling such as `clang-tidy`, `include-what-you-use`, `gcov`, that works and does not have to be maintained by us.https://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/215Select low level API via environment variable2021-01-22T16:13:06+01:00Benedikt Steinbuschb.steinbusch@fz-juelich.deSelect low level API via environment variableAllow selecting what low level API to use via an environment variable `SIONLIB_LOWLEVEL_API={posix|c|ime}`. Similar variables for collector group size, `SION_COLLSIZE`, override the value set in the open function options, so this new mec...Allow selecting what low level API to use via an environment variable `SIONLIB_LOWLEVEL_API={posix|c|ime}`. Similar variables for collector group size, `SION_COLLSIZE`, override the value set in the open function options, so this new mechanism should follow that precedent.2.0.0Benedikt Steinbuschb.steinbusch@fz-juelich.deBenedikt Steinbuschb.steinbusch@fz-juelich.dehttps://gitlab.jsc.fz-juelich.de/cstao-public/SIONlib/SIONlib/-/issues/214sionconfig broken in 1.7.5 when SIONfwd linker flags contain spaces2019-11-20T18:04:25+01:00Benedikt Steinbuschb.steinbusch@fz-juelich.desionconfig broken in 1.7.5 when SIONfwd linker flags contain spacesThe definitions of `SIONFWD_LIBS` etc. in `src/utils/sionconfig.in` should be enclosed in quotes.The definitions of `SIONFWD_LIBS` etc. in `src/utils/sionconfig.in` should be enclosed in quotes.Benedikt Steinbuschb.steinbusch@fz-juelich.deBenedikt Steinbuschb.steinbusch@fz-juelich.de