using SIONlib: ANSI fopen and POSIX buffering
From John Donners:
in a previous email I mentioned the long opening times for a file that is striped across many Lustre OSTs and opened with many tasks (12s for 48 OSTs and 512 MPI tasks). It seems that it's due to the way that SIONlib opens the file for writing: using fopen with the 'w' mode. This truncates the file, which gives a lot of overhead. It seems better to open the file using the 'r+' mode, which doesn't truncate the file and it expects that the file exists (that shouldn't be a problem, since SIONlib just created it). I've modified the routine _sion_file_open_ansi_write_existing in src/lib/sion_file.c and that seems to do the trick. However, you know much better if this doesn't create problems elsewhere.
I've also experimented a lot with the POSIX interface and the internal buffering for SIONlib. The SIONlib buffer seems not flushed at the moment that the file is closed. I've added an extra routine to flush the buffer, which uses the same code as in sion_fwrite. I'm not sure where to insert the code in the different SIONlib close calls.