Skip to content

REFAC: reduce communication in multiprocessing

reason

This is a follow-up of #321 (closed).

idea

#321 (closed) originates that the entire data handler has to be return on multiprocessing which causes errors. Pickle the data handler as tmp file and just return the station name in case of success. Outside the multiprocessing (after calling p.get()) reload the data handler.

further ideas to consider

Think how to reduce the size of data handler further.