filter dh: differentiate between history offset and information offset

differentiate between history offset and information offset

After closing this issue there are the following parameters:

  • window_history_offset: used to include future values into history / inputs. Example: set window_history_size=48 and window_history_offset=12 to use 2 days as history / input but not on interval t0 + [-48h, 0h] but t0 + [-36h, 12h]. This parameter is especially used together with a 1d target variable (mixed sampling) that measurement interval is not from 0000 to 0000 but 1700 (day before) to 1700 (day of t0) or similar (e.g. dma8eu ozone). Then it is possible to use window_history_size=48+16 and window_history_offset=16 to use 3 days of input without all timesteps starting at 1700 the day before (as they are included in the target).
  • extend_length_opts: use this parameter to use future data in the filter calculation. Example: set extend_length_opts=7*24 to use the observation of the next 7 days to calculate the filtered components. This parameter does not effect the interval of the sample (which is handled by the window_history_offset parameter)

current status and todos

  • The extend_length_opts parameter is somehow used in a wrong way as it also has an impact on the interval length of the history / inputs.
    • change the following lines: expand climate_filter_data according to window_history_offset parameter and not with extend_length_opts
class DataHandlerClimateFirFilterSingleStation(...):
    ...
    def apply_filter(self):
        ...
        if isinstance(self.extend_length_opts, int):
            climate_filter_data = [c.sel({self.window_dim: slice(-self.window_history_size, self.extend_length_opts)})
                             for c in climate_filter.filtered_data]
        else:
            climate_filter_data = []
            for c in climate_filter.filtered_data:
                coll_tmp = []
                for v in c.coords[self.target_dim].values:
                    upper_lim = self.extend_length_opts.get(v, 0)
                    coll_tmp.append(c.sel({self.target_dim: v,
                                           self.window_dim: slice(-self.window_history_size, upper_lim)}))
                climate_filter_data.append(xr.concat(coll_tmp, self.target_dim))
  • The ClimateFIRFilter class has no information about the offset parameter as it uses extend_length_opts for information extension and interval extend.
    • add an offset parameter to the ClimateFIRFilter class (is currently only used in FIRFilter class for plotting)
    • include offset parameter for minimum length calculation
    • trimm data by considering extend_length_opts and offset inside self.clim_filter
    • also trimm data with respect to offset parameter in self.run

testing

to ensure proper working check:

  • extend_length_opts < offset
  • extend_length_opts > offset
  • what about equality?
  • extend_length_opts is None
  • offset is None