Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • MLAir MLAir
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 91
    • Issues 91
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 9
    • Merge requests 9
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Artifacts
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
    • Model experiments
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • esdeesde
  • machine-learning
  • MLAirMLAir
  • Issues
  • #399

climateFIR empty data after sel_opts

Bug

Error description

There is an issue that calculation of climateFIR stops.

Error message

  File "/p/home/jusers/leufen1/juwels/intelliaq/mlair_tf2/mlair/mlair/helpers/time_tracking.py", line 40, in __call__
    return self.__wrapped__(*args, **kwargs)
  File "/p/home/jusers/leufen1/juwels/intelliaq/mlair_tf2/mlair/mlair/data_handler/data_handler_with_filter.py", line 378, in apply_filter
    climate_filter = ClimateFIRFilter(self.input_data.astype("float32"), self.fs, self.filter_order,
  File "/p/home/jusers/leufen1/juwels/intelliaq/mlair_tf2/mlair/mlair/helpers/filter.py", line 210, in __init__
    super().__init__(data, fs, order, cutoff, window, var_dim, time_dim, display_name=display_name,
  File "/p/home/jusers/leufen1/juwels/intelliaq/mlair_tf2/mlair/mlair/helpers/filter.py", line 37, in __init__
    self.run()
  File "/p/home/jusers/leufen1/juwels/intelliaq/mlair_tf2/mlair/mlair/helpers/filter.py", line 221, in run
    diurnal_anomalies = self.create_seasonal_hourly_mean(self.data, self.time_dim, sel_opts=self.sel_opts,
  File "/p/home/jusers/leufen1/juwels/intelliaq/mlair_tf2/mlair/mlair/helpers/filter.py", line 451, in create_seasonal_hourly_mean
    monthly = self.create_monthly_unity_array(data, time_dim) * np.nan
  File "/p/home/jusers/leufen1/juwels/intelliaq/mlair_tf2/mlair/mlair/helpers/filter.py", line 332, in create_monthly_unity_array
    start = coords[time_dim][0].values.astype("datetime64[D]") - np.timedelta64(extend_range, "D")

First guess on error origin

coords must be empty for some reason.

Error origin

If sel_opts is not set properly, no data is remaining. Add method to check if data is available when applying sel_opts

Solution

Add check method and more meaningfull error message:

 class ClimateFIRFilter(FIRFilter):
 ...
    def run(self):
         filtered = []
         h = []
         if self.sel_opts is not None:
             self.sel_opts = self.sel_opts if isinstance(self.sel_opts, dict) else {self.time_dim: self.sel_opts}
+            self._check_sel_opts()
         sampling = {1: "1d", 24: "1H"}.get(int(self.fs))
 ...
+    def _check_sel_opts(self):
+        if len(self.data.sel(**self.sel_opts).coords[self.time_dim]) == 0:
+            raise ValueError(f"Abort {self.__class__.__name__} as no data is available after applying sel_opts to data")
Edited Jul 08, 2022 by lukas leufen
Assignee
Assign to
Time tracking