1. The source code does not include any data to process. Instead, it will check if data are available on your local machine
and will download data from the [here](https://join.fz-juelich.de). The data set is roughly 1.6 GB. We did not implement
a default `data_path` as we want to allow you to choose where exactly the data should be stored.
Consequently, you have to pass a custom data path to `ExperimentSetup` in `run.py` (see example below).
If all required data are already locally available, the program will not download any new data.
2. Please note that cartopy may cause errors on run time. If cartopy raises an error you can try the following
(in activated venv):
*`pip uninstall shapely`
*`pip uninstall cartopy`
*`pip install --upgrade numpy`
*`pip install --no-binary shapely shapely`
*`pip install cartopy`
Catropy is needed only to create one plot showing the station locations and does not effect
the neural network itself. If the procedure above does not solve the problem, you can force the workflow to ignore
cartopy by adding the first two characters of your hostname (`echo $HOSTNAME`) as a list containing a string to the
keyword argument `hpc_hosts` in `run.py`.
Thee example below, assumes that the output of `echo $HOSTNAME` is "your_hostname".
Please also consult the [installation instruction](https://scitools.org.uk/cartopy/docs/latest/installing.html#installing)
of the cartopy package itself.
Example for all remarks given above:
```
import [...]
[...]
def main(parser_args):
[...]
with RunEnvironment():
ExperimentSetup(parser_args,
data_path="<your>/<custom>/<path>" # <- Remark 1
hpc_hosts=["yo"], # <- Remark 2
[...]
```
## HPC - JUWELS and HDFML setup
The following instruction guide you throug the installation on JUWELS and HDFML.
The following instruction guide you through the installation on JUWELS and HDFML.
* Clone the repo to HPC system (we recommend to place it in `/p/projects/<project name>`.
* Setup venv by executing `source setupHPC.sh`. This script loads all pre-installed modules and creates a venv for
all other packages. Furthermore, it creates slurm/batch scripts to execute code on compute nodes. <br>
...
...
@@ -28,7 +76,8 @@ You have to enter the HPC project's budget name (--account flag).
* Currently cartopy is not working on our HPC system, therefore PlotStations does not create any output.
### HPC JUWELS and HDFML remarks
Please note, that the HPC setup is customised for JUWELS and HDFML. When using another HPC system, you can use the HPC setup files as a skeleton and customise it to your needs.
Please note, that the HPC setup is customised for JUWELS and HDFML. When using another HPC system, you can use the HPC
setup files as a skeleton and customise it to your needs.
Note: The method `PartitionCheck` currently only checks if the hostname starts with `ju` or `hdfmll`.
Therefore, it might be necessary to adopt the `if` statement in `PartitionCheck._run`.
