diff --git a/docs/_source/get-started.rst b/docs/_source/get-started.rst index d51abaf4e1f256a420182c8d93055e99760b6963..7374c9bdc7b3e4ed8a224ed51f6608bf5e9aefb3 100644 --- a/docs/_source/get-started.rst +++ b/docs/_source/get-started.rst @@ -6,130 +6,6 @@ Getting started with MLAir :language: python -Install MLAir -------------- - -MLAir is based on several python frameworks. To work properly, you have to install all packages from the -:py:`requirements.txt` file. Additionally to support the geographical plotting part it is required to install geo -packages built for your operating system. Unfortunately, the names of these package may differ for different systems. -In this instruction, we try to address users of different operating systems namely openSUSE Leap, Ubuntu and macOS. -If the installation is still not working, we recommend skipping the geographical plot. We have put together a small -workaround :ref:`here<Workaround to skip geographical plot>`. For special instructions to install MLAir on the Juelich -HPC systems, see section :ref:`Installation on Jülich HPC systems`. - -Pre-requirements -~~~~~~~~~~~~~~~~ - -* Make sure to have the **python3.6** version installed. -* (geo) A **c++ compiler** is required for the installation of the program **cartopy** -* (geo) Install **proj** and **GEOS** on your machine using the console. -* Install the **python3.6 develop** libraries. - -Installation of MLAir -~~~~~~~~~~~~~~~~~~~~~ - -* Install all requirements from `requirements.txt <https://gitlab.version.fz-juelich.de/toar/machinelearningtools/-/blob/master/requirements.txt>`_ - preferably in a virtual environment -* Either clone MLAir from the `gitlab repository <https://gitlab.version.fz-juelich.de/toar/machinelearningtools.git>`_ -* or download the distribution file (`current version <https://gitlab.version.fz-juelich.de/toar/mlair/-/blob/master/dist/mlair-1.2.0-py3-none-any.whl>`_) - and install it via :py:`pip install <dist_file>.whl`. In this case, you can simply - import MLAir in any python script inside your virtual environment using :py:`import mlair`. -* (tf) Currently, TensorFlow-1.13 is mentioned in the requirements. We already tested the TensorFlow-1.15 version and couldn't - find any compatibility errors. Please note, that tf-1.13 and 1.15 have two distinct branches each, the default branch - for CPU support, and the "-gpu" branch for GPU support. If the GPU version is installed, MLAir will make use of the GPU - device. - -Special Instructions for Installation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -openSUSE Leap 15.1 -"""""""""""""""""" - -* c++ compiler - -:py:`sudo zypper install gcc-c++` - -* geo packages - -:py:`sudo zypper install proj geos-devel` - -* depending on the pre-installed packages it could be required to install further packages - -:py:`sudo zypper install libproj-devel binutils gdal-devel graphviz graphviz-gnome` - -* python develop libraries - -:py:`sudo zypper install python3-devel` - -Ubuntu 20.04.1 -"""""""""""""" - -* c++ compiler - -:py:`sudo apt install build-essential` - -* geo packages - -:py:`sudo apt install proj-bin libgeos-dev libproj-dev` - -* depending on the pre-installed packages it could be required to install further packages - -:py:`sudo apt install graphviz libgeos++-dev` - -* python develop libraries - -:py:`sudo apt install python3.6-dev` - -macOS & windows -""""""""""""""" - -The installation on macOS is not tested yet. The following commands are possibly needed: - -:py:`brew install geos` - -:py:`sudo port install graphviz` - -The installation on Windows is not tested yet. - -Installation on Jülich HPC systems -"""""""""""""""""""""""""""""""""" - -*Please note, that the HPC setup is customised for JUWELS and HDFML. When using another HPC system, you can use the HPC -setup files as a skeleton and customise it to your needs.* - -The following instruction guide you through the installation on JUWELS and HDFML. - -* Clone the repo to HPC system (we recommend to place it in :py:`/p/projects/<project name>`). -* Setup venv by executing :py:`source setupHPC.sh`. This script loads all pre-installed modules and creates a venv for - all other packages. Furthermore, it creates slurm/batch scripts to execute code on compute nodes. - You have to enter the HPC project's budget name (--account flag). -* The default external data path on JUWELS and HDFML is set to :py:`/p/project/deepacf/intelliaq/<user>/DATA/toar_<sampling>`. -* To choose a different location open :py:`run.py` and add the following keyword argument to :py:`ExperimentSetup`: - :py:`data_path=<your>/<custom>/<path>`. -* Execute :py:`python run.py` on a login node to download example data. The program will throw an OSerror after downloading. -* Execute either :py:`sbatch run_juwels_develgpus.bash` or :py:`sbatch run_hdfml_batch.bash` to verify that the setup - went well. -* Currently cartopy is not working on our HPC system, therefore PlotStations does not create any output. - -Note: The method :py:`PartitionCheck` currently only checks if the hostname starts with :py:`ju` or :py:`hdfmll`. -Therefore, it might be necessary to adopt the :py:`if` statement in :py:`PartitionCheck._run`. - - -Workaround to skip geographical plot -"""""""""""""""""""""""""""""""""""" - -If it is not possible to install all required geo libraries on your system, a good compromise is to skip the creation -of the geographical plot. Therefore, it is required to remove the plot from the :py:`plot_list` manually. We recommend -to use this code snippet as a starting point. - -.. code-block:: python - - from mlair.helpers import remove_items - from mlair.configuration.defaults import DEFAULT_PLOT_LIST - - mlair.run(plot_list=remove_items(DEFAULT_PLOT_LIST, "PlotStationMap")) - - How to start with MLAir ----------------------- diff --git a/docs/_source/index.rst b/docs/_source/index.rst index e0087b0e69fa2c63d1d976e4d0b076747e007d5f..b45b5db0546df02704fe4b24e5c7ce0a1d278376 100644 --- a/docs/_source/index.rst +++ b/docs/_source/index.rst @@ -8,10 +8,18 @@ Welcome to MLAir's documentation! This is the documentation of the `MLAir package <https://gitlab.version.fz-juelich.de/toar/mlair>`_. +.. figure:: ../logo/MLAir_Logo.png + + MLAir Logo + +MLAir (Machine Learning on Air data) is an environment that simplifies and accelerates the creation of new machine +learning (ML) models for the analysis and forecasting of meteorological and air quality time series. + .. toctree:: :maxdepth: 2 :caption: Contents: + installation get-started customise defaults diff --git a/docs/_source/installation.rst b/docs/_source/installation.rst new file mode 100644 index 0000000000000000000000000000000000000000..7578d9abf49b9e4b67dac19b6263c4bc05110eea --- /dev/null +++ b/docs/_source/installation.rst @@ -0,0 +1,125 @@ +.. role:: py(code) + :language: python + +Install MLAir +------------- + +MLAir is based on several python frameworks. To work properly, you have to install all packages from the +:py:`requirements.txt` file. Additionally to support the geographical plotting part it is required to install geo +packages built for your operating system. Unfortunately, the names of these package may differ for different systems. +In this instruction, we try to address users of different operating systems namely openSUSE Leap, Ubuntu and macOS. +If the installation is still not working, we recommend skipping the geographical plot. We have put together a small +workaround :ref:`here<Workaround to skip geographical plot>`. For special instructions to install MLAir on the Juelich +HPC systems, see section :ref:`Installation on Jülich HPC systems`. + +Pre-requirements +~~~~~~~~~~~~~~~~ + +* Make sure to have the **python3.6** version installed. +* (geo) A **c++ compiler** is required for the installation of the program **cartopy** +* (geo) Install **proj** and **GEOS** on your machine using the console. +* Install the **python3.6 develop** libraries. + +Installation of MLAir +~~~~~~~~~~~~~~~~~~~~~ + +* Install all requirements from `requirements.txt <https://gitlab.version.fz-juelich.de/toar/machinelearningtools/-/blob/master/requirements.txt>`_ + preferably in a virtual environment +* Either clone MLAir from the `gitlab repository <https://gitlab.version.fz-juelich.de/toar/machinelearningtools.git>`_ +* or download the distribution file (`current version <https://gitlab.version.fz-juelich.de/toar/mlair/-/blob/master/dist/mlair-1.2.0-py3-none-any.whl>`_) + and install it via :py:`pip install <dist_file>.whl`. In this case, you can simply + import MLAir in any python script inside your virtual environment using :py:`import mlair`. +* (tf) Currently, TensorFlow-1.13 is mentioned in the requirements. We already tested the TensorFlow-1.15 version and couldn't + find any compatibility errors. Please note, that tf-1.13 and 1.15 have two distinct branches each, the default branch + for CPU support, and the "-gpu" branch for GPU support. If the GPU version is installed, MLAir will make use of the GPU + device. + +Special Instructions for Installation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +openSUSE Leap 15.1 +"""""""""""""""""" + +* c++ compiler + +:py:`sudo zypper install gcc-c++` + +* geo packages + +:py:`sudo zypper install proj geos-devel` + +* depending on the pre-installed packages it could be required to install further packages + +:py:`sudo zypper install libproj-devel binutils gdal-devel graphviz graphviz-gnome` + +* python develop libraries + +:py:`sudo zypper install python3-devel` + +Ubuntu 20.04.1 +"""""""""""""" + +* c++ compiler + +:py:`sudo apt install build-essential` + +* geo packages + +:py:`sudo apt install proj-bin libgeos-dev libproj-dev` + +* depending on the pre-installed packages it could be required to install further packages + +:py:`sudo apt install graphviz libgeos++-dev` + +* python develop libraries + +:py:`sudo apt install python3.6-dev` + +macOS & windows +""""""""""""""" + +The installation on macOS is not tested yet. The following commands are possibly needed: + +:py:`brew install geos` + +:py:`sudo port install graphviz` + +The installation on Windows is not tested yet. + +Installation on Jülich HPC systems +"""""""""""""""""""""""""""""""""" + +*Please note, that the HPC setup is customised for JUWELS and HDFML. When using another HPC system, you can use the HPC +setup files as a skeleton and customise it to your needs.* + +The following instruction guide you through the installation on JUWELS and HDFML. + +* Clone the repo to HPC system (we recommend to place it in :py:`/p/projects/<project name>`). +* Setup venv by executing :py:`source setupHPC.sh`. This script loads all pre-installed modules and creates a venv for + all other packages. Furthermore, it creates slurm/batch scripts to execute code on compute nodes. + You have to enter the HPC project's budget name (--account flag). +* The default external data path on JUWELS and HDFML is set to :py:`/p/project/deepacf/intelliaq/<user>/DATA/toar_<sampling>`. +* To choose a different location open :py:`run.py` and add the following keyword argument to :py:`ExperimentSetup`: + :py:`data_path=<your>/<custom>/<path>`. +* Execute :py:`python run.py` on a login node to download example data. The program will throw an OSerror after downloading. +* Execute either :py:`sbatch run_juwels_develgpus.bash` or :py:`sbatch run_hdfml_batch.bash` to verify that the setup + went well. +* Currently cartopy is not working on our HPC system, therefore PlotStations does not create any output. + +Note: The method :py:`PartitionCheck` currently only checks if the hostname starts with :py:`ju` or :py:`hdfmll`. +Therefore, it might be necessary to adopt the :py:`if` statement in :py:`PartitionCheck._run`. + + +Workaround to skip geographical plot +"""""""""""""""""""""""""""""""""""" + +If it is not possible to install all required geo libraries on your system, a good compromise is to skip the creation +of the geographical plot. Therefore, it is required to remove the plot from the :py:`plot_list` manually. We recommend +to use this code snippet as a starting point. + +.. code-block:: python + + from mlair.helpers import remove_items + from mlair.configuration.defaults import DEFAULT_PLOT_LIST + + mlair.run(plot_list=remove_items(DEFAULT_PLOT_LIST, "PlotStationMap"))