Exploring Wilderness Using Explainable Machine Learning in Satellite Imagery

This code belongs to the research article "Exploring Wilderness Using Explainable Machine Learning in Satellite Imagery" (2022) by Timo T. Stomberg, Taylor Stone, Johannes Leonhardt, Immanuel Weber, and Ribana Roscher; https://doi.org/10.48550/arXiv.2203.00379. Please cite our article, if you use this code, the model or the dataset.

The AnthroProtect dataset which has been used for the experiments can be found here: https://phenoroam.phenorob.de/geonetwork/srv/eng/catalog.search#/metadata/6b1b0977-9bc0-4bf3-944e-bc825e466435.

The trained model from the main experiments of our publication can be downloaded here: http://rs.ipb.uni-bonn.de/downloads/asos/.

Content of the ReadMe:

• Code Structure
• Methodology
• Setup this Repository
• Getting Started: Predict Attribution Maps using the Trained Model from the Article
• Train Your Own Model
• Install via pip

Code Structure

The code to run the methodology can be found at:

• tjects/exps/sefas

It is is based on the following three sub-libraries within the library tlib:

• tlib/tutils: basic utils
• tlib/tgeo: tools for GeoTIF files, KML files, etc.
• tlib/ttorch: machine learning tools for PyTorch

Methodology

If you apply an attribution method such as Grad-CAM to the input layer, you will obtain low-level features. If you apply it to the last convolutional layer, high-level features will be considered, but in low resolution. We propose a neural network (NN) architecture to obtain both high-level features in high resolution. It consists of an image-to-image network (U-Net) and a task-specific head (regressor or classifier):

Further, we harmonize the attributions across the training data to obtain consistent results across large and different scenes.

Our methodology works with a wide range of attribution methods.

We use our methodology to find patterns of protected and anthropogenic areas as proxies for naturalness and human influence, respectively. To this end, we use the AnthroProtect dataset which is built to discover the appearance of wilderness and anthropogenic areas in Fennoscandia using multispectral satellite imagery. It consists of 23,919 Sentinel-2 images.

Click here for an interactive map: http://rs.ipb.uni-bonn.de/html/anthroprotect_datset_locations.html [Copernicus Sentinel data 2020; Plotly Technologies Inc. (2015), Carto and OpenStreetMap contributors]

In the end, we are able to predict attribution maps for scenes in Fennoscandia.

Setup this Repository

Please download or clone this repository on your machine. Use the environment.yml file to setup a virtual environment using Anaconda; or use the requirements.txt file to setup a virtual environment using pip.

If you work on an Ubuntu system and use Anaconda, you can easily use the setup.sh file to set up everything including the Python paths. Just run the following command in your terminal:

$ source setup.sh

Choose "1) install venv from yml". Next time you can choose "2) activate venv".

Before you run setup.sh the first time, make sure that you have installed Anaconda and the following packages or run the following line:

$ sudo apt-get install libgl1-mesa-glx libegl1-mesa libxrandr2 libxrandr2 libxss1 libxcursor1 libxcomposite1 libasound2 libxi6 libxtst6

Getting Started: Predict Attribution Maps using the Trained Model from the Article

• Download or clone this repository.

• Set up the repository as described in "Setup and Requirements", e.g. running $ source setup.sh in your terminal.

• Download the AnthroProtect dataset. You find a link at the beginning of this readme. Unzip the dataset.

• Download the trained model (link also at the beginning of this readme). Unzip the zip file and locate the "logs" folder in a working directory of your choice. Within this working directory, other files (figures etc.) might be saved later on.

• Open the file "tjects/config.py" and set the configurations as described in this file.

• Open Jupyter Lab or Jupyter Notebook, e.g. running $ jupyter lab or $ jupyter notebook in your terminal.

• Open the notebook "tjects/exps/sefas/41_analyze_samples.ipynb". You have several options to load data and to plot it.

Train Your Own Model

• Run $ python tjects/exps/sefas/21_train.py in your terminal to start the training of a new model. You can set hyperparameters etc. in this file.

• While training, you can view the loggings in tensorboard running $ tensorboard --logdir working_dir/ttorch_logs where working_dir is your working directory.

• Folder "version_x" in "ttorch_logs" in your working directory must be moved to working directory and be renamed to "logs".

• Open the "31_sefas.ipynb" notebook to harmonize the attributions. This might take significantly longer than training the model.

• With the other notebooks you can analyze the results.

Install via pip

You can also install this repository via pip. This way you can access the tlib library, but not the tjects library. If you want to reproduce the results from our article, we recommend to clone this repository, so that you can use the notebooks from the tjects library! To install the repository, you have to install earthengine-api first.

$ conda create --name asos python=3.9
$ conda activate asos

$ conda install -c conda-forge earthengine-api
$ pip install git+https://gitlab.jsc.fz-juelich.de/kiste/asos@main

After installing the repository, you can import it as tlib:

import tlib

Note: Typing 'pip install tlib' will install another library which is not this one.