diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 32d8f138020e175ed4e17077713ed7cb26c5c533..4c9e114a55c22341be115484a2f71f303493726a 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -72,7 +72,7 @@ tests (on GPU):
- chmod +x ./CI/update_badge.sh
- ./CI/update_badge.sh > /dev/null
script:
- - pip install -r requirements.txt
+ - pip install -r requirements_gpu.txt
- chmod +x ./CI/run_pytest.sh
- ./CI/run_pytest.sh
after_script:
diff --git a/README.md b/README.md
index 6786f0ddb31e0616bf10161f94e80f09f8afeb1b..c56baeea96c3f7d6f3712a72966f166af4ecf844 100644
--- a/README.md
+++ b/README.md
@@ -10,26 +10,74 @@ docs [here](http://toar.pages.jsc.fz-juelich.de/mlair/docs/).
MLAir is based on several python frameworks. To work properly, you have to install all packages from the
`requirements.txt` file. Additionally to support the geographical plotting part it is required to install geo
-packages built for your operating system. Name names of these package may differ for different systems, we refer
-here to the opensuse / leap OS. The geo plot can be removed from the `plot_list`, in this case there is no need to
-install the geo packages. For special instructions to install MLAir on the Juelich HPC systems, see
-[here](#special-instructions-for-installation-on-jülich-hpc-systems).
-
-* (geo) Install **proj** on your machine using the console. E.g. for opensuse / leap `zypper install proj`
-* (geo) A c++ compiler is required for the installation of the program **cartopy**
-* Install all requirements from [`requirements.txt`](https://gitlab.version.fz-juelich.de/toar/mlair/-/blob/master/requirements.txt)
+packages built for your operating system. Unfortunately, the names of these package may differ for different systems.
+In this instruction, we try to address users of different operating systems namely openSUSE Leap, Ubuntu and macOS.
+If the installation is still not working, we recommend skipping the geographical plot. We have put together a small
+workaround [here](#workaround-to-skip-geographical-plot). For special instructions to install MLAir on the Juelich
+HPC systems, see [here](#special-instructions-for-installation-on-jülich-hpc-systems).
+
+* Make sure to have the **python3.6** version installed.
+* (geo) A **c++ compiler** is required for the installation of the program **cartopy**
+* (geo) Install **proj** and **GEOS** on your machine using the console.
+* Install the **python3.6 develop** libraries.
+* Install all **requirements** from [`requirements.txt`](https://gitlab.version.fz-juelich.de/toar/mlair/-/blob/master/requirements.txt)
preferably in a virtual environment
-* (tf) Currently, TensorFlow-1.13 is mentioned in the requirements. We already tested the TensorFlow-1.15 version and couldn't
- find any compatibility errors. Please note, that tf-1.13 and 1.15 have two distinct branches each, the default branch
- for CPU support, and the "-gpu" branch for GPU support. If the GPU version is installed, MLAir will make use of the GPU
- device.
* Installation of **MLAir**:
* Either clone MLAir from the [gitlab repository](https://gitlab.version.fz-juelich.de/toar/mlair.git)
and use it without installation (beside the requirements)
* or download the distribution file ([current version](https://gitlab.version.fz-juelich.de/toar/mlair/-/blob/master/dist/mlair-0.12.2-py3-none-any.whl))
and install it via `pip install <dist_file>.whl`. In this case, you can simply import MLAir in any python script
inside your virtual environment using `import mlair`.
-
+* (tf) Currently, TensorFlow-1.13 is mentioned in the requirements. We already tested the TensorFlow-1.15 version and couldn't
+ find any compatibility errors. Please note, that tf-1.13 and 1.15 have two distinct branches each, the default branch
+ for CPU support, and the "-gpu" branch for GPU support. If the GPU version is installed, MLAir will make use of the GPU
+ device.
+
+## openSUSE Leap 15.1
+
+* c++ compiler
+
+`sudo zypper install gcc-c++`
+
+* geo packages
+
+`sudo zypper install proj geos-devel`
+
+* depending on the pre-installed packages it could be required to install further packages
+
+`sudo zypper install libproj-devel binutils gdal-devel graphviz`
+
+* python develop libraries
+
+`sudo zypper install python3-devel`
+
+## Ubuntu 20.04.1
+
+* c++ compiler
+
+`sudo apt install build-essential`
+
+* geo packages
+
+`sudo apt install proj-bin libgeos-dev libproj-dev`
+
+* depending on the pre-installed packages it could be required to install further packages
+
+`sudo apt install graphviz libgeos++-dev`
+
+* python develop libraries
+
+`sudo apt install python3.6-dev`
+
+## macOS & windows
+
+The installation on macOS is not tested yet. The following commands are possibly needed:
+
+`brew install geos`
+
+`sudo port install graphviz`
+
+The installation on Windows is not tested yet.
# How to start with MLAir
@@ -421,6 +469,20 @@ class DummyDataHandler(AbstractDataHandler):
# Special Remarks
+## Workaround to skip geographical plot
+
+If it is not possible to install all required geo libraries on your system, a good compromise is to skip the creation
+of the geographical plot. Therefore, it is required to remove the plot from the `plot_list` manually. We recommend to
+use this code snippet as a starting point.
+
+```python
+from mlair.helpers import remove_items
+from mlair.configuration.defaults import DEFAULT_PLOT_LIST
+
+mlair.run(plot_list=remove_items(DEFAULT_PLOT_LIST, "PlotStationMap"))
+
+```
+
## Special instructions for installation on Jülich HPC systems
_Please note, that the HPC setup is customised for JUWELS and HDFML. When using another HPC system, you can use the HPC
@@ -429,10 +491,10 @@ setup files as a skeleton and customise it to your needs._
The following instruction guide you through the installation on JUWELS and HDFML.
* Clone the repo to HPC system (we recommend to place it in `/p/projects/<project name>`).
* Setup venv by executing `source setupHPC.sh`. This script loads all pre-installed modules and creates a venv for
-all other packages. Furthermore, it creates slurm/batch scripts to execute code on compute nodes. <br>
+all other packages. Furthermore, it creates slurm/batch scripts to execute code on compute nodes.
You have to enter the HPC project's budget name (--account flag).
* The default external data path on JUWELS and HDFML is set to `/p/project/deepacf/intelliaq/<user>/DATA/toar_<sampling>`.
-<br>To choose a different location open `run.py` and add the following keyword argument to `ExperimentSetup`:
+To choose a different location open `run.py` and add the following keyword argument to `ExperimentSetup`:
`data_path=<your>/<custom>/<path>`.
* Execute `python run.py` on a login node to download example data. The program will throw an OSerror after downloading.
* Execute either `sbatch run_juwels_develgpus.bash` or `sbatch run_hdfml_batch.bash` to verify that the setup went well.
diff --git a/docs/_source/get-started.rst b/docs/_source/get-started.rst
index 7dda0eccb41281f6d15b4434b7e13ed2aa332d35..67b03ce3644dd65eebc99b864285a8f6aad72d89 100644
--- a/docs/_source/get-started.rst
+++ b/docs/_source/get-started.rst
@@ -15,15 +15,21 @@ packages built for your operating system. Name names of these package may differ
here to the opensuse / leap OS. The geo plot can be removed from the :py:`plot_list`, in this case there is no need to
install the geo packages.
+MLAir is based on several python frameworks. To work properly, you have to install all packages from the
+:py:`requirements.txt` file. Additionally to support the geographical plotting part it is required to install geo
+packages built for your operating system. Unfortunately, the names of these package may differ for different systems.
+In this instruction, we try to address users of different operating systems namely openSUSE Leap, Ubuntu and macOS.
+If the installation is still not working, we recommend skipping the geographical plot. We have put together a small
+workaround :ref:`Workaround to skip geographical plot`. For special instructions to install MLAir on the Juelich
+HPC systems, see ref:`Installation on Jülich HPC systems`.
+
Pre-requirements
~~~~~~~~~~~~~~~~
-* (geo) Install **proj** on your machine using the console. E.g. for opensuse / leap :py:`zypper install proj`
-* (geo) A c++ compiler is required for the installation of the program **cartopy**
-* (tf) Currently, TensorFlow-1.13 is mentioned in the requirements. We already tested the TensorFlow-1.15 version and couldn't
- find any compatibility errors. Please note, that tf-1.13 and 1.15 have two distinct branches each, the default branch
- for CPU support, and the "-gpu" branch for GPU support. If the GPU version is installed, MLAir will make use of the GPU
- device.
+* Make sure to have the **python3.6** version installed.
+* (geo) A **c++ compiler** is required for the installation of the program **cartopy**
+* (geo) Install **proj** and **GEOS** on your machine using the console.
+* Install the **python3.6 develop** libraries.
Installation of MLAir
~~~~~~~~~~~~~~~~~~~~~
@@ -34,6 +40,100 @@ Installation of MLAir
* or download the distribution file (`current version <https://gitlab.version.fz-juelich.de/toar/mlair/-/blob/master/dist/mlair-0.12.2-py3-none-any.whl>`_)
and install it via :py:`pip install <dist_file>.whl`. In this case, you can simply
import MLAir in any python script inside your virtual environment using :py:`import mlair`.
+* (tf) Currently, TensorFlow-1.13 is mentioned in the requirements. We already tested the TensorFlow-1.15 version and couldn't
+ find any compatibility errors. Please note, that tf-1.13 and 1.15 have two distinct branches each, the default branch
+ for CPU support, and the "-gpu" branch for GPU support. If the GPU version is installed, MLAir will make use of the GPU
+ device.
+
+Special Instructions for Installation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+openSUSE Leap 15.1
+""""""""""""""""""
+
+* c++ compiler
+
+:py:`sudo zypper install gcc-c++`
+
+* geo packages
+
+:py:`sudo zypper install proj geos-devel`
+
+* depending on the pre-installed packages it could be required to install further packages
+
+:py:`sudo zypper install libproj-devel binutils gdal-devel graphviz`
+
+* python develop libraries
+
+:py:`sudo zypper install python3-devel`
+
+Ubuntu 20.04.1
+""""""""""""""
+
+* c++ compiler
+
+:py:`sudo apt install build-essential`
+
+* geo packages
+
+:py:`sudo apt install proj-bin libgeos-dev libproj-dev`
+
+* depending on the pre-installed packages it could be required to install further packages
+
+:py:`sudo apt install graphviz libgeos++-dev`
+
+* python develop libraries
+
+:py:`sudo apt install python3.6-dev`
+
+macOS & windows
+"""""""""""""""
+
+The installation on macOS is not tested yet. The following commands are possibly needed:
+
+:py:`brew install geos`
+
+:py:`sudo port install graphviz`
+
+The installation on Windows is not tested yet.
+
+Installation on Jülich HPC systems
+""""""""""""""""""""""""""""""""""
+
+*Please note, that the HPC setup is customised for JUWELS and HDFML. When using another HPC system, you can use the HPC
+setup files as a skeleton and customise it to your needs.*
+
+The following instruction guide you through the installation on JUWELS and HDFML.
+
+* Clone the repo to HPC system (we recommend to place it in :py:`/p/projects/<project name>`).
+* Setup venv by executing :py:`source setupHPC.sh`. This script loads all pre-installed modules and creates a venv for
+ all other packages. Furthermore, it creates slurm/batch scripts to execute code on compute nodes.
+ You have to enter the HPC project's budget name (--account flag).
+* The default external data path on JUWELS and HDFML is set to :py:`/p/project/deepacf/intelliaq/<user>/DATA/toar_<sampling>`.
+* To choose a different location open :py:`run.py` and add the following keyword argument to :py:`ExperimentSetup`:
+ :py:`data_path=<your>/<custom>/<path>`.
+* Execute :py:`python run.py` on a login node to download example data. The program will throw an OSerror after downloading.
+* Execute either :py:`sbatch run_juwels_develgpus.bash` or :py:`sbatch run_hdfml_batch.bash` to verify that the setup
+ went well.
+* Currently cartopy is not working on our HPC system, therefore PlotStations does not create any output.
+
+Note: The method :py:`PartitionCheck` currently only checks if the hostname starts with :py:`ju` or :py:`hdfmll`.
+Therefore, it might be necessary to adopt the :py:`if` statement in :py:`PartitionCheck._run`.
+
+
+Workaround to skip geographical plot
+""""""""""""""""""""""""""""""""""""
+
+If it is not possible to install all required geo libraries on your system, a good compromise is to skip the creation
+of the geographical plot. Therefore, it is required to remove the plot from the :py:`plot_list` manually. We recommend
+to use this code snippet as a starting point.
+
+.. code-block:: python
+
+ from mlair.helpers import remove_items
+ from mlair.configuration.defaults import DEFAULT_PLOT_LIST
+
+ mlair.run(plot_list=remove_items(DEFAULT_PLOT_LIST, "PlotStationMap"))
How to start with MLAir
diff --git a/docs/_source/index.rst b/docs/_source/index.rst
index 5777cf162545d81c11e3834c74a62271d05254d2..705ab91ddc1000edfa7961719db1ff913de26a93 100644
--- a/docs/_source/index.rst
+++ b/docs/_source/index.rst
@@ -6,6 +6,7 @@
Welcome to MLAir's documentation!
================================================
+This is the documation of the `MLAir package <https://gitlab.version.fz-juelich.de/toar/mlair>`_.
.. toctree::
:maxdepth: 2
diff --git a/requirements.txt b/requirements.txt
index 9032c9973ec6fbd28493763e4ebf6dbb4a16ed9a..be76eab5b74797b039682a292ae8890488c058ec 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -65,5 +65,5 @@ xarray==0.15.0
zipp==3.1.0
setuptools~=49.6.0
-Cartopy==0.17.0
+Cartopy==0.18.0
--no-binary shapely Shapely==1.7.0
\ No newline at end of file
diff --git a/requirements_gpu.txt b/requirements_gpu.txt
index fac479d5e724779114006a29eb701944f5ff314f..45e04819f355640dc4a85218440b95164dba280c 100644
--- a/requirements_gpu.txt
+++ b/requirements_gpu.txt
@@ -64,5 +64,5 @@ Werkzeug==1.0.0
xarray==0.15.0
zipp==3.1.0
-#Cartopy==0.17.0
+#Cartopy==0.18.0
--no-binary shapely Shapely==1.7.0
\ No newline at end of file