REFAC: reduce communication in multiprocessing
reason
This is a follow-up of #321 (closed).
idea
#321 (closed) originates that the entire data handler has to be return on multiprocessing which causes errors. Pickle the data handler as tmp file and just return the station name in case of success. Outside the multiprocessing (after calling p.get()
) reload the data handler.
further ideas to consider
Think how to reduce the size of data handler further.