Documentation for mapped open functions incomplete
Hi,
I have a sion program for 4000 cores, I want to execute the same program on 8000 cores, so whether it is possible to read data from 4000 cores and write to 8000 cores?
Kindly suggest.
Thanks,
Rahul
The best course of action in this scenario is to use SIONlib's "mapped open" functions like sion_paropen_mapped_mpi()
. However, there is no documentation for the public functions from that family, and the documentation for the internal _sion_paropen_mapped_generic()
is incomplete; namely the descriptions of all function arguments that differ in a meaningful way from a regular paropen
function currently read: FIXME
.
This is a first stab at documenting these functions:
\brief Opens a file (or multi-file set) in parallel with a user-defined mapping of file parts to tasks
Each calling task opens nlocaltasks
parts each of which resides in one of the numFiles
files of a multi-file set. A mapping from tasks to local file parts to global file parts can be specified via the arrays globalranks
, chunksizes
, mapping_filenrs
, and mapping_lranks
, each of which contains nlocaltasks
elements.
In write mode, each part i = 0 .. nlocaltasks
opened by a task, is assigned a global number globalranks[i]
which can be used to open a part for reading later on (TODO: also for seeking?). mapping_filenrs[i]
controls in which of the files a part resides, it contains values in the range 0 .. numFiles
. mapping_lranks[i]
assigns a per-file number to every part. Note: pairs of (mapping_filenrs[i], mapping_lranks[i])
must be unique across all tasks (TODO: confirm). Note: local rank numbers are arbitrary (TODO: confirm), but there has to be a part with local rank 0 in every file. Finally chunksizes[]
is used to specify individual chunk sizes for every part.
In read mode, globalranks[]
is used to specify which parts to open from which task. chunksizes
, mapping_filenrs
, and mapping_lranks
are optionally used as output arguments (if not NULL
) to return properties of the file parts selected for opening via their global rank numbers.