MLAir issueshttps://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues2022-12-02T15:30:02+01:00https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/419ResNet model class2022-12-02T15:30:02+01:00Ghost UserResNet model classImplement a model class to create ResNets: `ResNetFromConfig`.Implement a model class to create ResNets: `ResNetFromConfig`.https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/418BUG: wrong valid range in FIR filter2022-12-02T15:29:56+01:00Ghost UserBUG: wrong valid range in FIR filter<!-- Use this template for a bug in MLAir. -->
# Bug
## Error description
<!-- Provide a context when the bug / error arises -->
There are some issues when using FIR Filter as separation frequencies is not working for temperature and ...<!-- Use this template for a bug in MLAir. -->
# Bug
## Error description
<!-- Provide a context when the bug / error arises -->
There are some issues when using FIR Filter as separation frequencies is not working for temperature and pressure.
## Error message
<!-- Provide the error log if available -->
[periodogram_hourly_input_filter_last.pdf](/uploads/cc62b2081ddd03971b65f83db9e95e6d/periodogram_hourly_input_filter_last.pdf)
## First guess on error origin
<!-- Add first ideas where the error could come from -->
A view on a specific station input shows that the ends of the timeseries are affected by boundary effects.
![grafik](/uploads/da1a575aa1beecdb6b750ad9ff2f9859/grafik.png)
It looks that FIR filter does not remove values that are outside the valid filter range (which starts after half of filter window size).
## Error origin
<!-- Fill this up when the bug / error origin has been found -->
## Solution
<!-- Short description how to solve the error -->https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/417proper boolean conversion2022-12-02T15:29:21+01:00Ghost Userproper boolean conversionThere is an issue that boolean variables parsed by argument parser are not indicated as boolean but as string. As consequence, these parameters are always True if provided. * [ ] Add boolean conversion to helpers to ensure boolean variab...There is an issue that boolean variables parsed by argument parser are not indicated as boolean but as string. As consequence, these parameters are always True if provided. * [ ] Add boolean conversion to helpers to ensure boolean variable type.
```
def str2bool(v):
if isinstance(v, bool):
return v
elif isinstance(v, str):
if v.lower() in ('yes', 'true', 't', 'y', '1'):
return True
elif v.lower() in ('no', 'false', 'f', 'n', '0'):
return False
else:
raise argparse.ArgumentTypeError('Boolean value expected.')
else:
raise argparse.ArgumentTypeError('Boolean value expected.')
```
* [ ] update start scripts
```
parser.add_argument('--batch_normalization', default=argparse.SUPPRESS, type=lambda s: str2bool(s))
```https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/416BUG: report model fails on empty list2022-08-16T11:11:51+02:00Ghost UserBUG: report model fails on empty list<!-- Use this template for a bug in MLAir. -->
# Bug
## Error description
<!-- Provide a context when the bug / error arises -->
Model setup fails as list index is out of range.
## Error message
<!-- Provide the error log if availabl...<!-- Use this template for a bug in MLAir. -->
# Bug
## Error description
<!-- Provide a context when the bug / error arises -->
Model setup fails as list index is out of range.
## Error message
<!-- Provide the error log if available -->
```
Traceback (most recent call last):
File "/p/project/deepacf/intelliaq/leufen1/demystify-temporal-components/run_experiment_MBRNN_c+0d_m+inf_fut_cEU.py", line 170, in <module>
main(args)
File "/p/project/deepacf/intelliaq/leufen1/demystify-temporal-components/run_experiment_MBRNN_c+0d_m+inf_fut_cEU.py", line 139, in main
workflow.run()
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/workflows/abstract_workflow.py", line 31, in run
stage(**self._registry_kwargs[pos])
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/model_setup.py", line 70, in __init__
self._run()
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/model_setup.py", line 100, in _run
self.report_model()
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/model_setup.py", line 228, in report_model
if isinstance(v[0], dict):
IndexError: list index out of range
```
## First guess on error origin
<!-- Add first ideas where the error could come from -->
Check which parameter causes issue -> empty list as parameter, which cannot have element `v[0]`
## Error origin
<!-- Fill this up when the bug / error origin has been found -->
## Solution
<!-- Short description how to solve the error -->
Add if statement to check for length of v if it is a list. In case the length is zero, just report empty list.https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/415release v2.2.02022-08-16T12:44:59+02:00Ghost Userrelease v2.2.0<!-- Use this template for a new release of MLAir. -->
# Release
<!-- add your release version here -->
v2.2.0
## checklist
* [x] Create Release Issue
* [x] Create merge request: branch `release_v2.2.0` into `master`
* [x] Merge `deve...<!-- Use this template for a new release of MLAir. -->
# Release
<!-- add your release version here -->
v2.2.0
## checklist
* [x] Create Release Issue
* [x] Create merge request: branch `release_v2.2.0` into `master`
* [x] Merge `develop` into `release_v2.2.0`
* [x] Checkout `release_v2.2.0`
* [x] Adjust `changelog.md` (see template for changelog)
* [x] Update version number in `mlair/__ init__.py`
* [x] Create new dist file: `python3 setup.py sdist bdist_wheel`
* [x] Add new dist file `mlair-2.2.0-py3-none-any.whl` to git
* [x] Update file link `distribution file (current version)` in `README.md`
* [x] Update file link in `docs/_source/installation.rst`
* [x] Commit + push
* [x] Merge `release_v2.2.0` into `master`
* [x] Create new tag with
* [x] distribution file (.whl)
* [x] link to Documentation
* [x] Example Jupyter Notebook
* [x] changelog
## template for changelog
<!-- use this structure for the changelog. Link all issue to at least one item. -->
```
## v2.2.0 - 2022-08-15 - <release description>
### general:
* text
### new features:
* words (issue)
### technical:
*
```https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/414Include CRPS analysis and other ens verif methods or plots2022-08-10T13:34:17+02:00Ghost UserInclude CRPS analysis and other ens verif methods or plotsInclude CRPS analysis and other ens verif methods or plots.
make use of https://pypi.org/project/ensverif/ or related
Contributes to #411Include CRPS analysis and other ens verif methods or plots.
make use of https://pypi.org/project/ensverif/ or related
Contributes to #411https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/413Remove model prediction from normalized loop2022-08-16T11:12:06+02:00Ghost UserRemove model prediction from normalized loopRemove the nn model prediction in post_processing from normalized loop and use deepcopy instead. Modification of function signature `_create_nn_forecast` neededRemove the nn model prediction in post_processing from normalized loop and use deepcopy instead. Modification of function signature `_create_nn_forecast` neededhttps://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/412Create ens. predictions for BNNs2022-08-09T14:45:29+02:00Ghost UserCreate ens. predictions for BNNspreparation for #411preparation for #411https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/411Include postprocessing for BNNs2022-08-10T13:31:21+02:00Ghost UserInclude postprocessing for BNNsExtend post-processing module to work with BNNs
- [x] create ens. predictions #412
- [x] extract learned distribution parameters #412
- [ ] Include CRPS analysis
- [ ] extend time-series plots with uncertainties etc.Extend post-processing module to work with BNNs
- [x] create ens. predictions #412
- [x] extract learned distribution parameters #412
- [ ] Include CRPS analysis
- [ ] extend time-series plots with uncertainties etc.https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/410TECH: reduce CI running time2022-08-16T11:11:49+02:00Ghost UserTECH: reduce CI running time* try to reduce running time of CI
* ?* try to reduce running time of CI
* ?https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/409improve set keras generator speed2022-08-16T11:11:50+02:00Ghost Userimprove set keras generator speed* [ ] check how to improve speed of creating keras generators during Training stage* [ ] check how to improve speed of creating keras generators during Training stagehttps://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/408hypersearch workflow without Postprocessing2022-08-16T11:11:52+02:00Ghost Userhypersearch workflow without Postprocessing* implement a new workflow that can be used for hyperparam search which skips postprocessing for speed-up (as only val loss is relevant)* implement a new workflow that can be used for hyperparam search which skips postprocessing for speed-up (as only val loss is relevant)https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/407TECH: update geos install in CI Test (from Scratch)2022-08-16T11:11:54+02:00Ghost UserTECH: update geos install in CI Test (from Scratch)there is a new geos package, which replaced the old version (which cannot be installed anymore)
v 3.10.3 -> v 3.11.0there is a new geos package, which replaced the old version (which cannot be installed anymore)
v 3.10.3 -> v 3.11.0https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/406REFAC: improve snapshot2022-08-31T10:41:25+02:00Ghost UserREFAC: improve snapshot* [x] add missing skip_args to snapshot loading check
* [x] also use model args similar to data handler requirements* [x] add missing skip_args to snapshot loading check
* [x] also use model args similar to data handler requirementshttps://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/405BUG: error calc fails as no obs is avail2022-08-16T11:11:53+02:00Ghost UserBUG: error calc fails as no obs is avail<!-- Use this template for a bug in MLAir. -->
# Bug
## Error description
<!-- Provide a context when the bug / error arises -->
When using snapshot loading, there might by an issue as "obs" is not available.
## Error message
<!-- Pr...<!-- Use this template for a bug in MLAir. -->
# Bug
## Error description
<!-- Provide a context when the bug / error arises -->
When using snapshot loading, there might by an issue as "obs" is not available.
## Error message
<!-- Provide the error log if available -->
```
2022-07-15 18:16:14,095 - ERROR: "['obs'] not found in axis" [run_environment.py:__exit__:139]
Traceback (most recent call last):
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/workflows/abstract_workflow.py", line 31, in run
stage(**self._registry_kwargs[pos])
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 102, in __init__
self._run()
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 117, in _run
self.estimate_sample_uncertainty(separate_ahead=True)
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/helpers/time_tracking.py", line 40, in __call__
return self.__wrapped__(*args, **kwargs)
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 154, in estimate_sample_uncertainty
block_mse, block_mse_per_station = self.calculate_block_mse(evaluate_competitors=evaluate_competitors,
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 238, in calculate_block_mse
errors = self.create_error_array(combined)
File "/p/home/jusers/leufen1/hdfml/intelliaq/mlair_tf2/mlair/mlair/run_modules/post_processing.py", line 255, in create_error_array
errors = data.drop_sel({self.model_type_dim: self.observation_indicator})
File "/p/project/deepacf/intelliaq/leufen1/demystify-temporal-components/venv_hdfml/lib/python3.9/site-packages/xarray/core/dataarray.py", line 2239, in drop_sel
ds = self._to_temp_dataset().drop_sel(labels, errors=errors)
File "/p/project/deepacf/intelliaq/leufen1/demystify-temporal-components/venv_hdfml/lib/python3.9/site-packages/xarray/core/dataset.py", line 4013, in drop_sel
new_index = index.drop(labels_for_dim, errors=errors)
File "/p/software/hdfml/stages/2022/software/SciPy-bundle/2021.10-gcccoremkl-11.2.0-2021.4.0/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 6017, in drop
raise KeyError(f"{labels[mask]} not found in axis")
KeyError: "['obs'] not found in axis"
```
## First guess on error origin
<!-- Add first ideas where the error could come from -->
* [x] check if failing station is available in snapshot creation run -> is removed for val and test subsets
* [x] check if failing station is used in make prediction calls -> not used, therefore data_collection variable works properly
* [x] check if failing station is still in stations parameter -> stations is part of the exluded parameters during snapshot loading, but it should be updated at least when copying to the new data store
## Error origin
<!-- Fill this up when the bug / error origin has been found -->
## Solution
<!-- Short description how to solve the error -->
Issue solved by also updating stations parameter on snapshot loadinghttps://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/404Skip OLS on request2022-09-14T14:08:43+02:00Ghost UserSkip OLS on requestCurrent status: OLS is always calculated on data. In case only a hyperparameter search is applied, the comparison with the OLS is not that important.
* [x] add parameter to skip OLS calculation
* [x] only estimate OLS if True (should be...Current status: OLS is always calculated on data. In case only a hyperparameter search is applied, the comparison with the OLS is not that important.
* [x] add parameter to skip OLS calculation
* [x] only estimate OLS if True (should be still default)
* [x] check in postprocessing at which point, MLAir expects to have OLS forecast
* [x] Maybe think about using competitor to indicate usage of OLS instead of parameter proposed above? (Adding `OLS` to competitor list would trigger OLS calculation)https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/403TECH: Docs Badge not created2022-08-16T11:12:01+02:00Ghost UserTECH: Docs Badge not created<!-- Use this template for a bug in MLAir. -->
# Bug
## Error description
<!-- Provide a context when the bug / error arises -->
Badges are not created for docs stage during CI pipeline.
## Error message
<!-- Provide the error log if...<!-- Use this template for a bug in MLAir. -->
# Bug
## Error description
<!-- Provide a context when the bug / error arises -->
Badges are not created for docs stage during CI pipeline.
## Error message
<!-- Provide the error log if available -->
There is an error indicating that something is wrong with the curl command.
![grafik](/uploads/223f28202e74b5d3860a9a84af5dfff6/grafik.png)
## First guess on error origin
<!-- Add first ideas where the error could come from -->
## Error origin
<!-- Fill this up when the bug / error origin has been found -->
## Solution
<!-- Short description how to solve the error -->https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/402update HPC environment2022-08-16T11:11:59+02:00Ghost Userupdate HPC environment* [ ] update modules as Stage 2020 currently used is deprecated soon
* [ ] replace by Stage 2022 and update modules
* [ ] update additional requirements
* [ ] maybe also think about updating the local requirements too (to have uniform be...* [ ] update modules as Stage 2020 currently used is deprecated soon
* [ ] replace by Stage 2022 and update modules
* [ ] update additional requirements
* [ ] maybe also think about updating the local requirements too (to have uniform behaviour)https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/401missing datetime import2022-08-16T11:12:02+02:00Ghost Usermissing datetime import<!-- Use this template for a bug in MLAir. -->
# Bug
missing `import datetime as dt` statement in post_processing.py
## Error description
<!-- Provide a context when the bug / error arises -->
## Error message
<!-- Provide the error l...<!-- Use this template for a bug in MLAir. -->
# Bug
missing `import datetime as dt` statement in post_processing.py
## Error description
<!-- Provide a context when the bug / error arises -->
## Error message
<!-- Provide the error log if available -->
## First guess on error origin
<!-- Add first ideas where the error could come from -->
## Error origin
<!-- Fill this up when the bug / error origin has been found -->
## Solution
<!-- Short description how to solve the error -->https://gitlab.jsc.fz-juelich.de/esde/machine-learning/mlair/-/issues/400include TF Probability2023-05-26T11:51:43+02:00Ghost Userinclude TF Probabilitymake tensorflow_probability available within HPC setupmake tensorflow_probability available within HPC setup