Skip to content
Snippets Groups Projects
Commit 291d6ba4 authored by Sabine Schröder's avatar Sabine Schröder
Browse files

#11 and #3 (and also #8): provided an API to the controlled vocabulary (as... 

#11 and #3 (and also #8): provided an API to the controlled vocabulary (as long as no ontology service is in place), also wrote first pytest, but not yet working (settings have to be established in test database), prepared scripts to create pages also documenting the controlled vocabulary (pages: index can still show the source code, which is wanted)
parent b983f3fe
Branches
No related tags found
No related merge requests found
Pipeline #42743 passed
Stage: deploy
Stage: pages
Hide whitespace changes
Inline Side-by-side
CI/prepare_documentation_precommit.sh
+ 18
− 2
View file @ 291d6ba4
...@@ -7,6 +7,14 @@ python3 prepare_openapi_json.py toardb_fastapi.yaml ...@@ -7,6 +7,14 @@ python3 prepare_openapi_json.py toardb_fastapi.yaml
# convert OpenAPI format to Markdown # convert OpenAPI format to Markdown
openapi-markdown --skip-info -i sample.json openapi-markdown --skip-info -i sample.json
# also get controlled_vocabulary
curl "http://127.0.0.1:8000/controlled_vocabulary" >controlled_vocabulary.json
# giving credit to torsimany (although it was not working, it was a good starting point)
torsimany controlled_vocabulary.json
# some more pretty printing for the "tables"
cat controlled_vocabulary.markdown | sed -E 's/\], \[/\], \n \[/g' | sed -E 's/\[\[/\[ \n \[/g' | sed -E 's/#/####/g' >controlled_vocabulary.md
# last pretty print changes for pages # last pretty print changes for pages
( (
cat <<'EOF' cat <<'EOF'
...@@ -26,6 +34,14 @@ EOF ...@@ -26,6 +34,14 @@ EOF
) > sample_rest_header.md ) > sample_rest_header.md
sed '1,/^### Models$/d' sample.md >sample_models.md sed '1,/^### Models$/d' sample.md >sample_models.md
sed '/^### Models$/Q' sample.md >sample_rest.md sed '/^### Models$/Q' sample.md >sample_rest.md
(
cat <<'EOF'
## controlled vocabulary
EOF
) > controlled_vocabulary_header.md
cat sample_header.md sample_models.md sample_rest_header.md sample_rest.md >../docs/toardb_fastapi.md cat sample_header.md sample_models.md controlled_vocabulary_header.md controlled_vocabulary.md sample_rest_header.md sample_rest.md >../docs/toardb_fastapi.md
rm sample* toardb_fastapi.yaml rm sample* controlled* toardb_fastapi.yaml
docs/toardb_fastapi.md
+ 180
− 0
View file @ 291d6ba4
...@@ -236,8 +236,188 @@ ...@@ -236,8 +236,188 @@
| chemical_formula | string | Chemical formula. Chemical formula of variable(if applicable) | Yes | | chemical_formula | string | Chemical formula. Chemical formula of variable(if applicable) | Yes |
| id | integer | | Yes | | id | integer | | Yes |
## controlled vocabulary
#### Rc_Vocabulary ####
* RC_vocabulary: [
[0, 'PointOfContact', 'point of contact'],
[1, 'PrincipalInvestigator', 'principal investigator'],
[2, 'Originator', 'originator'],
[3, 'Contributor', 'contributor'],
[4, 'Collaborator', 'collaborator'],
[5, 'ResourceProvider', 'resource provider']]
#### Rs_Vocabulary ####
* RS_vocabulary: [
[0, 'active', 'active'],
[1, 'inactive', 'inactive'],
[2, 'unknown', 'unknown']]
#### Ok_Vocabulary ####
* OK_vocabulary: [
[1, 'Government', 'government'],
[2, 'Research', 'research'],
[3, 'International', 'international'],
[4, 'NonProfit', 'non-profit'],
[5, 'Commercial', 'commercial'],
[6, 'Individual', 'individual'],
[7, 'Other', 'other']]
#### Da_Vocabulary ####
* DA_vocabulary: [
[0, 'ByAttribution', 'by attribution'],
[1, 'ShareAlike', 'share alike'],
[2, 'Restricted', 'restricted']]
#### Sf_Vocabulary ####
* SF_vocabulary: [
[0, 'Hourly', 'hourly'],
[1, 'ThreeHourly', '3-hourly'],
[2, 'SixHourly', '6-hourly'],
[3, 'Daily', 'daily'],
[4, 'Weekly', 'weekly'],
[5, 'Monthly', 'monthly'],
[6, 'Yearly', 'yearly'],
[7, 'Irregular', 'irregular data samples of constant length'],
[8, 'Irregular2', 'irregular data samples of varying length']]
#### At_Vocabulary ####
* AT_vocabulary: [
[0, 'Mean', 'mean'],
[1, 'Mean1Of2', 'mean of two values'],
[2, 'Mean1OfWeek', 'weekly mean'],
[3, 'Mean4Samples', 'mean out of 4 samples'],
[4, 'MeanMonth', 'monthly mean'],
[5, 'None', 'none'],
[6, 'Unknown', 'unknown']]
#### Ds_Vocabulary ####
* DS_vocabulary: [
[0, 'Model', 'model'],
[1, 'Measurement', 'measurement']]
#### Mm_Vocabulary ####
* MM_vocabulary: [
[0, 'UVAbsorption', 'UV absorption'],
[1, 'UnknownInstrument', 'unknown instrument']]
#### Cz_Vocabulary ####
* CZ_vocabulary: [
[-1, 'Undefined', 'undefined'],
[0, 'Unclassified', 'unclassified'],
[1, 'WarmTemperateMoist', 'warm temperate moist'],
[2, 'WarmTemperateDry', 'warm temperate dry'],
[3, 'CoolTemperateMoist', 'cool temperate moist'],
[4, 'CoolTemperateDry', 'cool temperate dry'],
[5, 'PolarMoist', 'polar moist'],
[6, 'PolarDry', 'polar dry'],
[7, 'BorealMoist', 'boreal moist'],
[8, 'BorealDry', 'boreal dry'],
[9, 'TropicalMontane', 'tropical montane'],
[10, 'TropicalWet', 'tropical wet'],
[11, 'TropicalMoist', 'tropical moist'],
[12, 'TropicalDry', 'tropical dry']]
#### Cv_Vocabulary ####
* CV_vocabulary: [
[0, 'NotChecked', 'not checked'],
[1, 'Verified', 'verified'],
[2, 'Plausible', 'plausible'],
[3, 'Doubtful', 'doubtful'],
[4, 'Unverifyable', 'not verifyable']]
#### St_Vocabulary ####
* ST_vocabulary: [
[0, 'Unknown', 'unknown'],
[1, 'Background', 'background'],
[2, 'Traffic', 'traffic'],
[3, 'Industrial', 'industrial'],
[4, 'Other', 'other']]
#### Ta_Vocabulary ####
* TA_vocabulary: [
[0, 'Unknown', 'unknown'],
[1, 'Urban', 'urban'],
[2, 'Suburban', 'suburban'],
[3, 'Rural', 'rural'],
[4, 'Remote', 'remote']]
#### Tc_Vocabulary ####
* TC_vocabulary: [
[-1, 'Unknown', 'unknown'],
[0, 'Unclassified', 'unclassified'],
[1, 'RuralLowElevation', 'rural low elevation'],
[2, 'RuralHighElevation', 'rural high elevation'],
[3, 'Urban', 'urban']]
#### Tr_Vocabulary ####
* TR_vocabulary: [
[-1, 'HTAPTier1Undefined', '-1 (undefined)'],
[1, 'HTAPTier1World', '1 (World)'],
[2, 'HTAPTier1OCN', '2 (OCN Non-arctic/Antarctic Ocean)'],
[3, 'HTAPTier1NAM', '3 (NAM US+Canada (upto 66 N; polar circle))'],
[4, 'HTAPTier1EUR', '4 (EUR Western + Eastern EU+Turkey (upto 66 N polar circle))'],
[5, 'HTAPTier1SAS', '5 (SAS South Asia: India, Nepal, Pakistan, Afghanistan, Bangadesh, Sri Lanka)'],
[6, 'HTAPTier1EAS', '6 (EAS East Asia: China, Korea, Japan)'],
[7, 'HTAPTier1SEA', '7 (SEA South East Asia)'],
[8, 'HTAPTier1PAN', '8 (PAN Pacific, Australia+ New Zealand)'],
[9, 'HTAPTier1NAF', '9 (NAF Northern Africa+Sahara+Sahel)'],
[10, 'HTAPTier1SAF', '10 (SAF Sub Saharan/sub Sahel Africa)'],
[11, 'HTAPTier1MDE', '11 (MDE Middle East: S. Arabia, Oman, etc, Iran, Iraq)'],
[12, 'HTAPTier1MCA', '12 (MCA Mexico, Central America, Caribbean, Guyanas, Venezuela, Columbia)'],
[13, 'HTAPTier1SAM', '13 (SAM S. America)'],
[14, 'HTAPTier1RBU', '14 (RBU Russia, Belarussia, Ukraine)'],
[15, 'HTAPTier1CAS', '15 (CAS Central Asia)'],
[16, 'HTAPTier1NPO', '16 (NPO Arctic Circle (North of 66 N) + Greenland)'],
[17, 'HTAPTier1SPO', '17 (SPO Antarctic)']]
#### Dl_Vocabulary ####
* DL_vocabulary: [
[-1, 'Undefined', '-1 (undefined)'],
[0, 'Water', '0 (Water)'],
[1, 'EGNeedleleaf', '1 (Evergreen Needleleaf forest)'],
[2, 'EGBroadleaf', '2 (Evergreen Broadleaf forest)'],
[3, 'DCNeedleleaf', '3 (Deciduous Needleleaf forest)'],
[4, 'DCBroadleaf', '4 (Deciduous Broadleaf forest)'],
[5, 'MixedForest', '5 (Mixed forest)'],
[6, 'ClosedShrublands', '6 (Closed shrublands)'],
[7, 'OpenShrublands', '7 (Open shrublands)'],
[8, 'WoodySavannas', '8 (Woody savannas)'],
[9, 'Savannas', '9 (Savannas)'],
[10, 'Grasslands', '10 (Grasslands)'],
[11, 'Wetlands', '11 (Permanent wetlands)'],
[12, 'Croplands', '12 (Croplands)'],
[13, 'Urban', '13 (Urban and built-up)'],
[14, 'Mosaic', '14 (Cropland/Natural vegetation mosaic)'],
[15, 'Snow', '15 (Snow and ice)'],
[16, 'Barren', '16 (Barren or sparsely vegetated)'],
[255, 'Fill', '255 (Fill Value/Unclassified)']]
#### Rt_Vocabulary ####
* RT_vocabulary: [
[0, 'String', 'str'],
[1, 'Integer', 'int'],
[2, 'Float', 'float']]
#### Df_Vocabulary ####
* DF_vocabulary: [
[0, 'OK', 'OK'],
[1, 'OKPreliminary', 'OK preliminary'],
[2, 'OKModified', 'OK modified'],
[3, 'OKPreliminaryModified', 'OK preliminary modified'],
[4, 'Inconsistent', 'inconsistent'],
[5, 'InconsistentPreliminary', 'inconsistent preliminary'],
[6, 'Doubtful', 'doubtful'],
[7, 'DoubtfulPreliminary', 'doubtful preliminary'],
[8, 'DoubtfulModified', 'doubtful modified'],
[9, 'DoubtfulPreliminaryModified', 'doubtful preliminary modified'],
[10, 'Wrong', 'wrong'],
[11, 'WrongPreliminary', 'wrong preliminary'],
[12, 'NotCheckedPreliminary', 'not checked preliminary'],
[13, 'Changed', 'changed'],
[14, 'Estimated', 'estimated'],
[15, 'MissingValue', 'missing value']]
## REST API ## REST API
### /controlled_vocabulary
#### GET
##### Summary:
Info
##### Responses
| Code | Description |
| ---- | ----------- |
| 200 | Successful Response |
### /variables/ ### /variables/
#### GET #### GET
......
docs/toardb_v2.pdf
+ 0
− 0
View file @ 291d6ba4
No preview for this file type
toardb/test_toardb.py 0 → 100644
+ 151
− 0
View file @ 291d6ba4
import pytest
import json
# Required imports 'create_test_database'
from toardb.test_base import (
client,
get_test_db,
create_test_database,
url,
get_test_engine,
test_db_session as db,
)
class TestApps:
def setup(self):
self.application_url = "/controlled_vocabulary"
def test_get_controlled_vocabulary(self, client, db):
response = client.get("/controlled_vocabulary")
expected_status_code = 200
assert response.status_code == expected_status_code
expected_resp = {"RC_vocabulary":[[0,"PointOfContact","point of contact"],
[1,"PrincipalInvestigator","principal investigator"],
[2,"Originator","originator"],
[3,"Contributor","contributor"],
[4,"Collaborator","collaborator"],
[5,"ResourceProvider","resource provider"]],
"RS_vocabulary":[[0,"active","active"],
[1,"inactive","inactive"],
[2,"unknown","unknown"]],
"OK_vocabulary":[[1,"Government","government"],
[2,"Research","research"],
[3,"International","international"],
[4,"NonProfit","non-profit"],
[5,"Commercial","commercial"],
[6,"Individual","individual"],
[7,"Other","other"]],
"DA_vocabulary":[[0,"ByAttribution","by attribution"],
[1,"ShareAlike","share alike"],
[2,"Restricted","restricted"]],
"SF_vocabulary":[[0,"Hourly","hourly"],
[1,"ThreeHourly","3-hourly"],
[2,"SixHourly","6-hourly"],
[3,"Daily","daily"],
[4,"Weekly","weekly"],
[5,"Monthly","monthly"],
[6,"Yearly","yearly"],
[7,"Irregular","irregular data samples of constant length"],
[8,"Irregular2","irregular data samples of varying length"]],
"AT_vocabulary":[[0,"Mean","mean"],
[1,"Mean1Of2","mean of two values"],
[2,"Mean1OfWeek","weekly mean"],
[3,"Mean4Samples","mean out of 4 samples"],
[4,"MeanMonth","monthly mean"],
[5,"None","none"],
[6,"Unknown","unknown"]],
"DS_vocabulary":[[0,"Model","model"],
[1,"Measurement","measurement"]],
"MM_vocabulary":[[0,"UVAbsorption","UV absorption"],
[1,"UnknownInstrument","unknown instrument"]],
"CZ_vocabulary":[[-1,"Undefined","undefined"],
[0,"Unclassified","unclassified"],
[1,"WarmTemperateMoist","warm temperate moist"],
[2,"WarmTemperateDry","warm temperate dry"],
[3,"CoolTemperateMoist","cool temperate moist"],
[4,"CoolTemperateDry","cool temperate dry"],
[5,"PolarMoist","polar moist"],
[6,"PolarDry","polar dry"],
[7,"BorealMoist","boreal moist"],
[8,"BorealDry","boreal dry"],
[9,"TropicalMontane","tropical montane"],
[10,"TropicalWet","tropical wet"],
[11,"TropicalMoist","tropical moist"],
[12,"TropicalDry","tropical dry"]],
"CV_vocabulary":[[0,"NotChecked","not checked"],
[1,"Verified","verified"],
[2,"Plausible","plausible"],
[3,"Doubtful","doubtful"],
[4,"Unverifyable","not verifyable"]],
"ST_vocabulary":[[0,"Unknown","unknown"],
[1,"Background","background"],
[2,"Traffic","traffic"],
[3,"Industrial","industrial"],
[4,"Other","other"]],
"TA_vocabulary":[[0,"Unknown","unknown"],
[1,"Urban","urban"],
[2,"Suburban","suburban"],
[3,"Rural","rural"],
[4,"Remote","remote"]],
"TC_vocabulary":[[-1,"Unknown","unknown"],
[0,"Unclassified","unclassified"],
[1,"RuralLowElevation","rural low elevation"],
[2,"RuralHighElevation","rural high elevation"],
[3,"Urban","urban"]],
"TR_vocabulary":[[-1,"HTAPTier1Undefined","-1 (undefined)"],
[1,"HTAPTier1World","1 (World)"],
[2,"HTAPTier1OCN","2 (OCN Non-arctic/Antarctic Ocean)"],
[3,"HTAPTier1NAM","3 (NAM US+Canada (upto 66 N; polar circle))"],
[4,"HTAPTier1EUR","4 (EUR Western + Eastern EU+Turkey (upto 66 N polar circle))"],
[5,"HTAPTier1SAS","5 (SAS South Asia: India, Nepal, Pakistan, Afghanistan, Bangadesh, Sri Lanka)"],
[6,"HTAPTier1EAS","6 (EAS East Asia: China, Korea, Japan)"],
[7,"HTAPTier1SEA","7 (SEA South East Asia)"],
[8,"HTAPTier1PAN","8 (PAN Pacific, Australia+ New Zealand)"],
[9,"HTAPTier1NAF","9 (NAF Northern Africa+Sahara+Sahel)"],
[10,"HTAPTier1SAF","10 (SAF Sub Saharan/sub Sahel Africa)"],
[11,"HTAPTier1MDE","11 (MDE Middle East: S. Arabia, Oman, etc, Iran, Iraq)"],
[12,"HTAPTier1MCA","12 (MCA Mexico, Central America, Caribbean, Guyanas, Venezuela, Columbia)"],
[13,"HTAPTier1SAM","13 (SAM S. America)"],
[14,"HTAPTier1RBU","14 (RBU Russia, Belarussia, Ukraine)"],
[15,"HTAPTier1CAS","15 (CAS Central Asia)"],
[16,"HTAPTier1NPO","16 (NPO Arctic Circle (North of 66 N) + Greenland)"],
[17,"HTAPTier1SPO","17 (SPO Antarctic)"]],
"DL_vocabulary":[[-1,"Undefined","-1 (undefined)"],
[0,"Water","0 (Water)"],
[1,"EGNeedleleaf","1 (Evergreen Needleleaf forest)"],
[2,"EGBroadleaf","2 (Evergreen Broadleaf forest)"],
[3,"DCNeedleleaf","3 (Deciduous Needleleaf forest)"],
[4,"DCBroadleaf","4 (Deciduous Broadleaf forest)"],
[5,"MixedForest","5 (Mixed forest)"],
[6,"ClosedShrublands","6 (Closed shrublands)"],
[7,"OpenShrublands","7 (Open shrublands)"],
[8,"WoodySavannas","8 (Woody savannas)"],
[9,"Savannas","9 (Savannas)"],
[10,"Grasslands","10 (Grasslands)"],
[11,"Wetlands","11 (Permanent wetlands)"],
[12,"Croplands","12 (Croplands)"],
[13,"Urban","13 (Urban and built-up)"],
[14,"Mosaic","14 (Cropland/Natural vegetation mosaic)"],
[15,"Snow","15 (Snow and ice)"],
[16,"Barren","16 (Barren or sparsely vegetated)"],
[255,"Fill","255 (Fill Value/Unclassified)"]],
"RT_vocabulary":[[0,"String","str"],
[1,"Integer","int"],
[2,"Float","float"]],
"DF_vocabulary":[[0,"OK","OK"],
[1,"OKPreliminary","OK preliminary"],
[2,"OKModified","OK modified"],
[3,"OKPreliminaryModified","OK preliminary modified"],
[4,"Inconsistent","inconsistent"],
[5,"InconsistentPreliminary","inconsistent preliminary"],
[6,"Doubtful","doubtful"],
[7,"DoubtfulPreliminary","doubtful preliminary"],
[8,"DoubtfulModified","doubtful modified"],
[9,"DoubtfulPreliminaryModified","doubtful preliminary modified"],
[10,"Wrong","wrong"],
[11,"WrongPreliminary","wrong preliminary"],
[12,"NotCheckedPreliminary","not checked preliminary"],
[13,"Changed","changed"],
[14,"Estimated","estimated"],
[15,"MissingValue","missing value"]]}
assert response.json() == expected_resp
toardb/toardb.py
+ 68
− 1
View file @ 291d6ba4
...@@ -15,10 +15,32 @@ from toardb.timeseries import timeseries ...@@ -15,10 +15,32 @@ from toardb.timeseries import timeseries
from toardb.data import data from toardb.data import data
from collections import namedtuple from collections import namedtuple
from pydantic import BaseSettings
app = FastAPI() app = FastAPI()
@app.get("/controlled_vocabulary")
async def info():
return {
"RC_vocabulary": settings.RC_vocab,
"RS_vocabulary": settings.RS_vocab,
"OK_vocabulary": settings.OK_vocab,
"DA_vocabulary": settings.DA_vocab,
"SF_vocabulary": settings.SF_vocab,
"AT_vocabulary": settings.AT_vocab,
"DS_vocabulary": settings.DS_vocab,
"MM_vocabulary": settings.MM_vocab,
"CZ_vocabulary": settings.CZ_vocab,
"CV_vocabulary": settings.CV_vocab,
"ST_vocabulary": settings.ST_vocab,
"TA_vocabulary": settings.TA_vocab,
"TC_vocabulary": settings.TC_vocab,
"TR_vocabulary": settings.TR_vocab,
"DL_vocabulary": settings.DL_vocab,
"RT_vocabulary": settings.RT_vocab,
"DF_vocabulary": settings.DF_vocab,
}
# Dependency # Dependency
def get_db(): def get_db():
try: try:
...@@ -73,3 +95,48 @@ async def startup_event(): ...@@ -73,3 +95,48 @@ async def startup_event():
RT_vocabulary = __get_enum_dict(fake_cur, "rt_vocabulary") RT_vocabulary = __get_enum_dict(fake_cur, "rt_vocabulary")
DF_vocabulary = __get_enum_dict(fake_cur, "df_vocabulary") DF_vocabulary = __get_enum_dict(fake_cur, "df_vocabulary")
Enumdict=namedtuple("Dict",["value","string","display_str"])
fake_conn = engine.raw_connection()
fake_cur = fake_conn.cursor()
RC_vocabulary = __get_enum_dict(fake_cur, "rc_vocabulary")
RS_vocabulary = __get_enum_dict(fake_cur, "rs_vocabulary")
OK_vocabulary = __get_enum_dict(fake_cur, "ok_vocabulary")
DA_vocabulary = __get_enum_dict(fake_cur, "da_vocabulary")
SF_vocabulary = __get_enum_dict(fake_cur, "sf_vocabulary")
AT_vocabulary = __get_enum_dict(fake_cur, "at_vocabulary")
DS_vocabulary = __get_enum_dict(fake_cur, "ds_vocabulary")
MM_vocabulary = __get_enum_dict(fake_cur, "mm_vocabulary")
CZ_vocabulary = __get_enum_dict(fake_cur, "cz_vocabulary")
CV_vocabulary = __get_enum_dict(fake_cur, "cv_vocabulary")
ST_vocabulary = __get_enum_dict(fake_cur, "st_vocabulary")
TA_vocabulary = __get_enum_dict(fake_cur, "ta_vocabulary")
TC_vocabulary = __get_enum_dict(fake_cur, "tc_vocabulary")
TR_vocabulary = __get_enum_dict(fake_cur, "tr_vocabulary")
DL_vocabulary = __get_enum_dict(fake_cur, "dl_vocabulary")
RT_vocabulary = __get_enum_dict(fake_cur, "rt_vocabulary")
DF_vocabulary = __get_enum_dict(fake_cur, "df_vocabulary")
class Settings(BaseSettings):
Enumdict=namedtuple("Dict",["value","string","display_str"])
RC_vocab = RC_vocabulary
RS_vocab = RS_vocabulary
OK_vocab = OK_vocabulary
DA_vocab = DA_vocabulary
SF_vocab = SF_vocabulary
AT_vocab = AT_vocabulary
DS_vocab = DS_vocabulary
MM_vocab = MM_vocabulary
CZ_vocab = CZ_vocabulary
CV_vocab = CV_vocabulary
ST_vocab = ST_vocabulary
TA_vocab = TA_vocabulary
TC_vocab = TC_vocabulary
TR_vocab = TR_vocabulary
DL_vocab = DL_vocabulary
RT_vocab = RT_vocabulary
DF_vocab = DF_vocabulary
settings = Settings()
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment