Newer
Older
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Get Dataset from request\n",
"\n",
"This cell imports all required packages and sets up the logging as well as the required information for the requests to the TOAR-DB."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from datetime import datetime as dt\n",
"from pathlib import Path\n",
"\n",
"import pandas as pd\n",
"import numpy as np\n",
"\n",
"from toargridding.grids import RegularGrid\n",
"from toargridding.toar_rest_client import (\n",
" AnalysisServiceDownload,\n",
" STATION_LAT,\n",
" STATION_LON,\n",
")\n",
"from toargridding.metadata import Metadata, TimeSample, AnalysisRequestResult, Coordinates\n",
"from toargridding.variables import Coordinate\n",
"\n",
"from toargridding.contributors import contributions_manager_by_id\n",
"import logging\n",
"from toargridding.defaultLogging import toargridding_defaultLogging\n",
"#setup of logging\n",
"logger = toargridding_defaultLogging()\n",
"logger.addShellLogger(logging.DEBUG)\n",
"logger.logExceptions()\n",
"\n",
"endpoint = \"https://toar-data.fz-juelich.de/api/v2/analysis/statistics/\"\n",
"#maybe adopt the toargridding_base_path for your machine.\n",
"toargridding_base_path = Path(\".\")\n",
"cache_dir = toargridding_base_path / \"cache\"\n",
"data_download_dir = toargridding_base_path / \"results\"\n",
"data_download_dir.mkdir(exist_ok=True)\n",
"analysis_service = AnalysisServiceDownload(endpoint, cache_dir, data_download_dir, use_downloaded=True)\n",
"my_grid = RegularGrid(1.9, 2.5)\n",
"\n",
"time = TimeSample(dt(2016,1,1), dt(2016,2,28), \"daily\")\n",
"metadata = Metadata.construct(\"mole_fraction_of_ozone_in_air\", time, \"mean\")\n"
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the next step we want to download the data and store them to disc. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"data = analysis_service.get_data(metadata)\n",
"\n",
"# create contributors endpoint and write result to metadata\n",
"contrib = contributions_manager_by_id(metadata.get_id(), data_download_dir)\n",
"contrib.extract_contributors_from_data_frame(data.stations_data)\n",
"metadata.contributors_metadata_field = contrib.setup_contributors_endpoint_for_metadata()\n",
"ds = my_grid.as_xarray(data)\n",
"#store dataset\n",
"ds.to_netcdf(data_download_dir / f\"{metadata.get_id()}_by_names_inline_{my_grid.get_id()}.nc\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Visual inspection\n",
"We now clean the station metadata. Therefore we remove all stations which have invalid coordinates"
"metadata": {},
"outputs": [],
"source": [
"#especially separation of coordinates with results and without results.\n",
"import cartopy.crs as ccrs\n",
"import matplotlib.pyplot as plt\n",
"import matplotlib.ticker as mticker\n",
"\n",
"mean_data = ds[\"mean\"]\n",
"clean_coords = data.stations_coords\n",
"all_na = data.stations_data.isna().all(axis=1)\n",
"clean_coords = all_na.to_frame().join(clean_coords)[[\"latitude\", \"longitude\"]]\n",
"all_na_coords = clean_coords[all_na]\n",
"not_na_coords = clean_coords[~all_na]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the next step we prepare a function for plotting the gridded data to a world map. The flag *discrete* influences the creation of the color bar. The *plot_stations* flag allows including the station positions into the map."
]
},
"metadata": {},
"outputs": [],
"source": [
"import matplotlib as mpl\n",
"\n",
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
"\n",
"def plot_cells(data, stations, na_stations, discrete=True, plot_stations=False):\n",
" fig = plt.figure(figsize=(9, 18))\n",
"\n",
" ax = plt.axes(projection=ccrs.PlateCarree())\n",
" ax.coastlines()\n",
" gl = ax.gridlines(draw_labels=True)\n",
" gl.top_labels = False\n",
" gl.left_labels = False\n",
" gl.xlocator = mticker.FixedLocator(data.longitude.values)\n",
" gl.ylocator = mticker.FixedLocator(data.latitude.values)\n",
"\n",
" cmap = mpl.cm.viridis\n",
"\n",
" if discrete:\n",
" print(np.unique(data.values))\n",
" bounds = np.arange(8)\n",
" norm = mpl.colors.BoundaryNorm(bounds, cmap.N, extend=\"both\")\n",
" ticks = np.arange(bounds.size + 1)[:-1] + 0.5\n",
" ticklables = bounds\n",
" \n",
" im = plt.pcolormesh(\n",
" data.longitude,\n",
" data.latitude,\n",
" data,\n",
" transform=ccrs.PlateCarree(),\n",
" cmap=cmap,\n",
" shading=\"nearest\",\n",
" norm=norm,\n",
" )\n",
" cb = fig.colorbar(im, ax=ax, shrink=0.2, aspect=25)\n",
" cb.set_ticks(ticks)\n",
" cb.set_ticklabels(ticklables)\n",
" im = plt.pcolormesh(\n",
" data.longitude,\n",
" data.latitude,\n",
" data,\n",
" transform=ccrs.PlateCarree(),\n",
" cmap=cmap,\n",
" shading=\"nearest\",\n",
" norm=norm,\n",
" )\n",
" else:\n",
" im = plt.pcolormesh(\n",
" data.longitude,\n",
" data.latitude,\n",
" data,\n",
" transform=ccrs.PlateCarree(),\n",
" cmap=cmap,\n",
" shading=\"nearest\",\n",
" )\n",
"\n",
" cb = fig.colorbar(im, ax=ax, shrink=0.2, aspect=25)\n",
" \n",
"\n",
" if plot_stations:\n",
" plt.scatter(na_stations[\"longitude\"], na_stations[\"latitude\"], s=1, c=\"k\")\n",
" plt.scatter(stations[\"longitude\"], stations[\"latitude\"], s=1, c=\"r\")\n",
" plt.title(f\"global ozon at {data.time.values} {data.time.units}\")"
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we do the actual plotting. We select a single time from the dataset. To obtain two maps: 1) the mean ozone concentration per grid point and second the number of stations contributing to a grid point."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#example visualization for two time points\n",
"time = ds.time[timestep]\n",
"data = ds.sel(time=time)\n",
"\n",
"plot_cells(data[\"mean\"], not_na_coords, all_na_coords, discrete=False, plot_stations=True)\n",
"plt.show()\n",
"\n",
"plot_cells(data[\"n\"], not_na_coords, all_na_coords, discrete=True)\n",
"plt.show()\n",
"\n",
"n_observations = ds[\"n\"].sum([\"latitude\", \"longitude\"])\n",
"plt.plot(ds.time, n_observations)\n",
"print(np.unique(ds[\"n\"]))"
]
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Last but not least: We print the data and metadata of the dataset. Especially a look into the metadata can be interesting."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
}
],
"metadata": {
"kernelspec": {
"display_name": "toargridding-g-KQ1Hyq-py3.10",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
}
},
"nbformat": 4,
"nbformat_minor": 2
}