REFAC: check valid stations behaviour
Current status: For each subset <total>
, train
, val
, test
, train_val
, a separate data collection with new data handlers is build. So the data preprocessing is repeated 5 times. For <total>
, the entire time range is used whereas for the subsets only the set time range is used. During train
, the data transformation is estimated. But in fact, the remaining subsets could be extracted from a general source without reloading, refiltering, ....
REFAC: Check the posibility to create <total>
and estimate transformation by train
. Then, apply this transformation to <total>
and start to extract the remaining subsets from this set (by deep copying ?). Is it faster? In principle it should be sufficient to copy only the data, that is required for get_X
, get_Y
, get_transposed_...
and ...?