BUG: error calc fails as no obs is avail
Bug
Error description
When using snapshot loading, there might by an issue as "obs" is not available.
Error message
2022-07-15 18:16:14,095 - ERROR: "['obs'] not found in axis" [run_environment.py:__exit__:139]
Traceback (most recent call last):
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/workflows/abstract_workflow.py", line 31, in run
stage(**self._registry_kwargs[pos])
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 102, in __init__
self._run()
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 117, in _run
self.estimate_sample_uncertainty(separate_ahead=True)
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/helpers/time_tracking.py", line 40, in __call__
return self.__wrapped__(*args, **kwargs)
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 154, in estimate_sample_uncertainty
block_mse, block_mse_per_station = self.calculate_block_mse(evaluate_competitors=evaluate_competitors,
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 238, in calculate_block_mse
errors = self.create_error_array(combined)
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 255, in create_error_array
errors = data.drop_sel({self.model_type_dim: self.observation_indicator})
File "/p/project/deepacf/intelliaq/leufen1/demystify-temporal-components/venv_hdfml/lib/python3.9/site-packages/xarray/core/dataarray.py", line 2239, in drop_sel
ds = self._to_temp_dataset().drop_sel(labels, errors=errors)
File "/p/project/deepacf/intelliaq/leufen1/demystify-temporal-components/venv_hdfml/lib/python3.9/site-packages/xarray/core/dataset.py", line 4013, in drop_sel
new_index = index.drop(labels_for_dim, errors=errors)
File "/p/software/hdfml/stages/2022/software/SciPy-bundle/2021.10-gcccoremkl-11.2.0-2021.4.0/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 6017, in drop
raise KeyError(f"{labels[mask]} not found in axis")
KeyError: "['obs'] not found in axis"
First guess on error origin
-
check if failing station is available in snapshot creation run -> is removed for val and test subsets -
check if failing station is used in make prediction calls -> not used, therefore data_collection variable works properly -
check if failing station is still in stations parameter -> stations is part of the exluded parameters during snapshot loading, but it should be updated at least when copying to the new data store
Error origin
Solution
Issue solved by also updating stations parameter on snapshot loading
Edited by Ghost User