<!-- Use this template for a bug in MLAir. --> # Bug ## Error description <!-- Provide a context when the bug / error arises --> Calculate clim skill scores failes on HDFML system. ## Error message <!-- Provide the error log if available --> ``` 2022-02-25 15:34:45,634 - INFO: start calculate_error_metrics [time_tracking.py:__enter__:131] 2022-02-25 15:34:45,634 - DEBUG: get: forecast_path(general)=/p/project/deepacf/intelliaq/leufen1/demystify-temporal-components/experiments/2022-02-24_MBRNN_LT_ST/2022-02-25_15-22-21_network_daily/forecasts [datastore.py:__call__:118] 2022-02-25 15:34:45,635 - DEBUG: get: stations(general)=['DEBB051', 'DEBB053', 'DEBE062', 'DEHH063', 'DEMV001', 'DEMV004', 'DEMV012', 'DEMV017', 'DENI031', 'DENI058', 'DENI059', 'DENI060', 'DENI063', 'DESH001', 'DESH008', 'DEST069', 'DEST089', 'DEUB001', 'DEUB005', 'DEUB006', 'DEUB007', 'DEUB020', 'DEUB022', 'DEUB024', 'DEUB026', 'DEUB027', 'DEUB028', 'DEUB030', 'DEUB034', 'DEUB038', 'DEUB040', 'DEBB028', 'DEBB038', 'DEBB039', 'DEBB040', 'DEBB048', 'DEBB050', 'DEBB063', 'DEBB067', 'DEBE051', 'DEHH005', 'DEHH021', 'DEHH022', 'DEHH030', 'DEHH047', 'DEHH049', 'DEHH050', 'DEMV007', 'DEMV018', 'DENI029', 'DENI052', 'DENI062', 'DESH005', 'DESH006', 'DESH016'] [datastore.py:__call__:118] 2022-02-25 15:34:45,635 - DEBUG: No competitor found for combination 'DEBB051' and 'IntelliO3-ts-v1'. [post_processing.py:load_competitors:273] 2022-02-25 15:34:45,636 - DEBUG: No competitor found for combination 'DEBB051' and 'MB-FCN-LT_ST'. [post_processing.py:load_competitors:273] 2022-02-25 15:34:45,636 - DEBUG: No competitor found for combination 'DEBB051' and 'MB-RNN-LT_ST'. [post_processing.py:load_competitors:273] 2022-02-25 15:34:45,636 - DEBUG: No competitor found for combination 'DEBB051' and 'OLS_hourly'. [post_processing.py:load_competitors:273] 2022-02-25 15:34:45,637 - DEBUG: No competitor found for combination 'DEBB051' and 'RNN_hourly'. [post_processing.py:load_competitors:273] 2022-02-25 15:34:45,637 - DEBUG: No competitor found for combination 'DEBB051' and 'FCN_1449_512_32_4'. [post_processing.py:load_competitors:273] 2022-02-25 15:34:46,661 - INFO: calculate_error_metrics finished after 0:00:02 (hh:mm:ss) [time_tracking.py:__exit__:137] 2022-02-25 15:34:46,661 - ERROR: Reindexing only valid with uniquely valued Index objects [run_environment.py:__exit__:137] Traceback (most recent call last): File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/workflows/abstract_workflow.py", line 31, in run stage(**self._registry_kwargs[pos]) File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 99, in __init__ self._run() File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 129, in _run skill_score_competitive, _, skill_score_climatological, errors = self.calculate_error_metrics() File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 1000, in calculate_error_metrics skill_score_climatological[station] = skill_score.climatological_skill_scores( File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/helpers/statistics.py", line 387, in climatological_skill_scores external_data = self.external_data.sel({self.ahead_dim: iahead, self.type_dim: [self.observation_name]}) File "/p/software/hdfml/stages/2020/software/Jupyter/2021.3.1-gcccoremkl-10.3.0-2021.2.0-Python-3.8.5/lib/python3.8/site-packages/xarray/core/dataarray.py", line 1202, in sel ds = self._to_temp_dataset().sel( File "/p/software/hdfml/stages/2020/software/Jupyter/2021.3.1-gcccoremkl-10.3.0-2021.2.0-Python-3.8.5/lib/python3.8/site-packages/xarray/core/dataset.py", line 2182, in sel pos_indexers, new_indexes = remap_label_indexers( File "/p/software/hdfml/stages/2020/software/Jupyter/2021.3.1-gcccoremkl-10.3.0-2021.2.0-Python-3.8.5/lib/python3.8/site-packages/xarray/core/coordinates.py", line 397, in remap_label_indexers pos_indexers, new_indexes = indexing.remap_label_indexers( File "/p/software/hdfml/stages/2020/software/Jupyter/2021.3.1-gcccoremkl-10.3.0-2021.2.0-Python-3.8.5/lib/python3.8/site-packages/xarray/core/indexing.py", line 270, in remap_label_indexers idxr, new_idx = convert_label_indexer(index, label, dim, method, tolerance) File "/p/software/hdfml/stages/2020/software/Jupyter/2021.3.1-gcccoremkl-10.3.0-2021.2.0-Python-3.8.5/lib/python3.8/site-packages/xarray/core/indexing.py", line 200, in convert_label_indexer indexer = get_indexer_nd(index, label, method, tolerance) File "/p/software/hdfml/stages/2020/software/Jupyter/2021.3.1-gcccoremkl-10.3.0-2021.2.0-Python-3.8.5/lib/python3.8/site-packages/xarray/core/indexing.py", line 103, in get_indexer_nd flat_indexer = index.get_indexer(flat_labels, method=method, tolerance=tolerance) File "/p/software/hdfml/stages/2020/software/SciPy-Stack/2021-gcccoremkl-10.3.0-2021.2.0-Python-3.8.5/lib/python3.8/site-packages/pandas-1.1.0-py3.8-linux-x86_64.egg/pandas/core/indexes/base.py", line 2980, in get_indexer raise InvalidIndexError( pandas.errors.InvalidIndexError: Reindexing only valid with uniquely valued Index objects ``` ## First guess on error origin <!-- Add first ideas where the error could come from --> * is there an empty array? Check shape and coords of `self.external_data` to get more insight :x: * error says that there is no unique index, have a look to all competing models in external data: :heavy_check_mark: ``` array(['MB-RNN-LT_ST', 'persi', 'obs', 'ols', 'IntelliO3-ts-v1', 'MB-FCN-LT_ST', 'MB-RNN-LT_ST', 'OLS_hourly', 'RNN_hourly', 'FCN_1449_512_32_4'], dtype=object) ``` ## Error origin <!-- Fill this up when the bug / error origin has been found --> * Error is caused by a duplicated name in the model_type dimensions (see duplication of 'MB-FCN-LT_ST') * origin: `model_display_name` was specified as `'MB-FCN-LT_ST'` but also a competitor with the same name is added to `competitors` list in the run script. ## Solution <!-- Short description how to solve the error --> * implement a name check in the experiment setup to prevent this duplication and abort experiment run when name is not unique.