BUG: error calc fails as no obs is avail

Bug

Error description

When using snapshot loading, there might by an issue as "obs" is not available.

Error message

2022-07-15 18:16:14,095 - ERROR: "['obs'] not found in axis"  [run_environment.py:__exit__:139]
Traceback (most recent call last):
  File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/workflows/abstract_workflow.py", line 31, in run
    stage(**self._registry_kwargs[pos])
  File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 102, in __init__
    self._run()
  File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 117, in _run
    self.estimate_sample_uncertainty(separate_ahead=True)
  File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/helpers/time_tracking.py", line 40, in __call__
    return self.__wrapped__(*args, **kwargs)
  File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 154, in estimate_sample_uncertainty
    block_mse, block_mse_per_station = self.calculate_block_mse(evaluate_competitors=evaluate_competitors,
  File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 238, in calculate_block_mse
    errors = self.create_error_array(combined)
  File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 255, in create_error_array
    errors = data.drop_sel({self.model_type_dim: self.observation_indicator})
  File "/p/project/deepacf/intelliaq/leufen1/demystify-temporal-components/venv_hdfml/lib/python3.9/site-packages/xarray/core/dataarray.py", line 2239, in drop_sel
    ds = self._to_temp_dataset().drop_sel(labels, errors=errors)
  File "/p/project/deepacf/intelliaq/leufen1/demystify-temporal-components/venv_hdfml/lib/python3.9/site-packages/xarray/core/dataset.py", line 4013, in drop_sel
    new_index = index.drop(labels_for_dim, errors=errors)
  File "/p/software/hdfml/stages/2022/software/SciPy-bundle/2021.10-gcccoremkl-11.2.0-2021.4.0/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 6017, in drop
    raise KeyError(f"{labels[mask]} not found in axis")
KeyError: "['obs'] not found in axis"

First guess on error origin

  • check if failing station is available in snapshot creation run -> is removed for val and test subsets
  • check if failing station is used in make prediction calls -> not used, therefore data_collection variable works properly
  • check if failing station is still in stations parameter -> stations is part of the exluded parameters during snapshot loading, but it should be updated at least when copying to the new data store

Error origin

Solution

Issue solved by also updating stations parameter on snapshot loading

Edited by Ghost User