Merge branch 'master' into 'develop'

update dev by current master version v1.1.0

See merge request toar/mlair!193

CHANGELOG.md

 # Changelog
 All notable changes to this project will be documented in this file.
 
+## v1.1.0 -  2020-11-18  - hourly resolution support and new data handlers
+
+### general:
+* MLAir can be used with 1H resolution data from JOIN
+* new data handlers to use the Kolmogorov-Zurbenko filter and mixed sampling types
+
+### new features:
+* new data handler `DataHandlerKzFilter` to use Kolmogorov-Zurbenko filter (kz filter) on inputs (#195)
+* new data handler `DataHandlerMixedSampling` that can used mixed sampling types for input and target (#197)
+* new data handler `DataHandlerMixedSamplingWithFilter` that uses kz filter and mixed sampling (#197)
+* new data handler `DataHandlerSeparationOfScales` to filter-depended time steps sizes on filtered inputs using mixed sampling (#196)
+
+### technical:
+* bug fix for very short time series in TimeSeriesPlot (#215)
+* bug fix for variable dictionary when using hourly resolution (#212)
+* variable naming for data from JOIN interface harmonised (#206)
+* transformation setup is now separated for inputs and targets (#202)
+* bug fix in PlotClimatologicalSkillScore if only single station is used (#193)
+* preprocessed data is now stored inside experiment and not in the data folder
+
 ## v1.0.0 -  2020-10-08  - official release of new version 1.0.0
 
 ### general:

README.md

@@ -29,7 +29,7 @@ HPC systems, see [here](#special-instructions-for-installation-on-jülich-hpc-sy
 * Installation of **MLAir**:
     * Either clone MLAir from the [gitlab repository](https://gitlab.version.fz-juelich.de/toar/mlair.git) 
       and use it without installation (beside the requirements) 
-    * or download the distribution file ([current version](https://gitlab.version.fz-juelich.de/toar/mlair/-/blob/master/dist/mlair-1.0.0-py3-none-any.whl)) 
+    * or download the distribution file ([current version](https://gitlab.version.fz-juelich.de/toar/mlair/-/blob/master/dist/mlair-1.1.0-py3-none-any.whl)) 
       and install it via `pip install <dist_file>.whl`. In this case, you can simply import MLAir in any python script 
       inside your virtual environment using `import mlair`.
 * (tf) Currently, TensorFlow-1.13 is mentioned in the requirements. We already tested the TensorFlow-1.15 version and couldn't

docs/_source/get-started.rst

@@ -31,7 +31,7 @@ Installation of MLAir
 * Install all requirements from `requirements.txt <https://gitlab.version.fz-juelich.de/toar/machinelearningtools/-/blob/master/requirements.txt>`_
   preferably in a virtual environment
 * Either clone MLAir from the `gitlab repository <https://gitlab.version.fz-juelich.de/toar/machinelearningtools.git>`_
-* or download the distribution file (`current version <https://gitlab.version.fz-juelich.de/toar/mlair/-/blob/master/dist/mlair-1.0.0-py3-none-any.whl>`_)
+* or download the distribution file (`current version <https://gitlab.version.fz-juelich.de/toar/mlair/-/blob/master/dist/mlair-1.1.0-py3-none-any.whl>`_)
   and install it via :py:`pip install <dist_file>.whl`. In this case, you can simply
   import MLAir in any python script inside your virtual environment using :py:`import mlair`.
 * (tf) Currently, TensorFlow-1.13 is mentioned in the requirements. We already tested the TensorFlow-1.15 version and couldn't

mlair/__init__.py

 __version_info__ = {
     'major': 1,
-    'minor': 0,
+    'minor': 1,
     'micro': 0,
 }
 

mlair/data_handler/data_handler_mixed_sampling.py

@@ -126,7 +126,7 @@ class DataHandlerMixedSamplingWithFilter(DefaultDataHandler):
     _requirements = data_handler.requirements()
 
 
-class DataHandlerMixedSamplingSeparationOfScalesSingleStation(DataHandlerMixedSamplingWithFilterSingleStation):
+class DataHandlerSeparationOfScalesSingleStation(DataHandlerMixedSamplingWithFilterSingleStation):
     """
     Data handler using mixed sampling for input and target. Inputs are temporal filtered and depending on the
     separation frequency of a filtered time series the time step delta for input data is adjusted (see image below).
@@ -194,10 +194,10 @@ class DataHandlerMixedSamplingSeparationOfScalesSingleStation(DataHandlerMixedSa
         return int(max([self.time_delta(est) * self.window_history_size, est]))
 
 
-class DataHandlerMixedSamplingSeparationOfScales(DefaultDataHandler):
+class DataHandlerSeparationOfScales(DefaultDataHandler):
     """Data handler using mixed sampling for input and target. Inputs are temporal filtered and different time step
     sizes are applied in relation to frequencies."""
 
-    data_handler = DataHandlerMixedSamplingSeparationOfScalesSingleStation
-    data_handler_transformation = DataHandlerMixedSamplingSeparationOfScalesSingleStation
+    data_handler = DataHandlerSeparationOfScalesSingleStation
+    data_handler_transformation = DataHandlerSeparationOfScalesSingleStation
     _requirements = data_handler.requirements()

run_mixed_sampling.py

@@ -5,7 +5,7 @@ import argparse
 
 from mlair.workflows import DefaultWorkflow
 from mlair.data_handler.data_handler_mixed_sampling import DataHandlerMixedSampling, DataHandlerMixedSamplingWithFilter, \
-    DataHandlerMixedSamplingSeparationOfScales
+    DataHandlerSeparationOfScales
 
 
 def main(parser_args):
@@ -13,7 +13,7 @@ def main(parser_args):
                 sampling_inputs="hourly",
                 window_history_size=24,
                 **parser_args.__dict__,
-                data_handler=DataHandlerMixedSamplingSeparationOfScales,
+                data_handler=DataHandlerSeparationOfScales,
                 kz_filter_length=[100 * 24, 15 * 24],
                 kz_filter_iter=[4, 5],
                 start="2006-01-01",

supplement/development_guidelines.md

@@ -14,3 +14,19 @@
 * Commit + push
 * Merge `release_vX.Y.Z` into `master`
 * Create new tag
+
+
+## template for release issue
+
+* [ ] Create Release Issue
+* [ ] Create merge request: branch `release_vX.Y.Z` into `master`
+* [ ] Merge `develop` into `release_vX.Y.Z`
+* [ ] Checkout `release_vX.Y.Z`
+* [ ] Adjust `changelog.md`
+* [ ] Update version number in `mlair/__ init__.py`
+* [ ] Create new dist file: `python3 setup.py sdist bdist_wheel`
+* [ ] Update file link `distribution file (current version)` in `README.md`
+* [ ] Update file link in `docs/_source/get-started.rst`
+* [ ] Commit + push
+* [ ] Merge `release_vX.Y.Z` into `master`
+* [ ] Create new tag
\ No newline at end of file