Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • MLAir MLAir
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 90
    • Issues 90
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 12
    • Merge requests 12
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
  • Monitor
    • Monitor
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • esdeesde
  • machine-learning
  • MLAirMLAir
  • Issues
  • #396
Closed
Open
Issue created Jun 30, 2022 by lukas leufen@leufen1👻Owner10 of 10 checklist items completed10/10 checklist items

Load data from ToarDB V2

Load data from REST API that uses Toar DB V2.

Path is: https://toar-data.fz-juelich.de/api/v2/

todos

  • get station name
  • load variables ids
  • get all variables of station
  • load data for station
  • make use of meta data
  • choose between different roles (replacing network parameter)
  • remove network parameter from MLAir (make it a data handler parameter)
  • old AIRBASE data is not available in V2 for now: implement module to merge data from v1 and v2
  • check for timezone of data, could be that v1 is on local time but v2 in UTC -> v1 always uses UTC without indicating, v2 uses UTC indicated by tz_info
  • as old and new AIRBASE data will be provided as distinct time series in V2, there should be a module merging these series -> all available series are merged (in case UBA and EEA are given this results in a merge of all UBA and EEA beeing merged, if only UBA is given, only UBA data is merged)

collection of sample links

station meta data

Ask for station meta data

https://toar-data.fz-juelich.de/api/v2/stationmeta/?codes=DEBW107

variable codes

Ask for variable code

https://toar-data.fz-juelich.de/api/v2/variables/o3

timeseries ids

Search for timeseries id with given variable id and station id

https://toar-data.fz-juelich.de/api/v2/search/?station_id=88&variable_id=5

or Search for timeseries id with given variable id and station codes

https://toar-data.fz-juelich.de/api/v2/search/?codes=DEBW107&variable_id=5

variable name cannot be used in search

Both commands could return multiple series, i.e. for UBA and EEA. To select the right series id, the meta data needs to be inspected.

load timeseries data

Load data of this timeseries with

https://toar-data.fz-juelich.de/api/v2/data/timeseries/18650?format=csv

loading aggregated timeseries data

To load statistics like dma8eu

https://toar-data.fz-juelich.de/statistics/api/v1/?timeseries_id=18650&names=dma8eu&sampling=daily

Note that this uses v1 (which still uses the new DB). Multiple statistics can be requested by using a comma separated list names=dma8eu,max

Edited Jul 07, 2022 by lukas leufen
Assignee
Assign to
Time tracking