REFAC: individual trafo for unfiltered data
Issue
When adding unfiltered data in FIR filter data handlers, this data are transformed with standardization. Reason: filtered data cannot be transformed with log or min/max because data are not following the original distribution and might have negative values. Therefore all data are transformed by standardization. This is valid for the filtered data but not ideal for the unfiltered / raw data.
Solution
Refacorize code to detect if unfiltered data are added and use the original transformation (e.g. log for o3) for this data.
Split unfiltered and filtered data into two distinct data handlers. For unfiltered data, it is quite easy to use the DefaultDataHandlerSingleStation
. Refactorize code, so that the FIR data handlers are able to use two data handlers if unfiltered data are requested to be part of X.
ToDos
-
add all parts of code that need to be adjusted -
remove all add unfiltered functionality from fir data handler -
refac transformation of DefaultDataHandler
to be able to calculate transformation on any given data handler -
add 2nd data handler for unfiltered data and make sure that data is parsed correctly
Outcommings
- Transformation methods are split into several
- removed old code parts that have been changed to comments
- all filter data handler single station are not able anymore to have unfiltered data (must be realized with the 2nd data handler single station)
- when using multiple data handlers single station there is a need to use only intersection of both when returning the observation.