maestro rdma transport ignores max_msg_size
According for fi_rma(3)
the length of a transfer is limited by max_msg_size
of the endpoint info fi_ep_attr
field.
Specifying larger values will result in hard errors of the provider; on GNI the size can be 4G (verbs and sockets providers typically have larger limits), and @biddiscombe1 has seen the issue on actual tests.
We need to check that value in locations where an fi_read
or fi_write
could require splitting up the transfer into multiple chunks.
For config block reads we can assert that the value is big enough to read it in one go, but in rdma.c it needs fixing (with multiple completions to be expected before declaring the operation complete).