1. 14 Aug, 2022 1 commit
  2. 18 Aug, 2021 1 commit
  3. 27 Jul, 2021 1 commit
  4. 16 Jul, 2021 1 commit
    • Sabine Schröder's avatar
      Squashed 'autoqc/' changes from 709a018..a909812 · 58ca2016
      Sabine Schröder authored
      a909812 Added statistics on the results to the returned dictionary. Updated the 'pandas' version.
      3fd2719 Added an argparser to the 'demo_program.py'.
      6faf783 Updated package versions in 'requirements.txt'.
      0c04cea Updated package versions in 'requirements.txt'.
      fd3b3b2 The coordinates will now be printed in the degrees minutes seconds format by the 'Plotter' class.
      90757e9 Removed the 'create_qc_score_plot' function from the 'Plotter' class in 'plotter.py'.
      9a3d46d Made some changes to the docstrings, comments, and error messages in 'tests.py'.
      0047e97 Changed the error messages in 'data_getter.py'.
      205bac9 Added metadata retrieval functions to the DataGetter classes in 'data_getter.py'.
      96c878f Made some changes to the docstrings and comments in 'plotter.py'.
      9b61f3c Made some changes to the docstrings in 'data_getter.py'.
      b88e1d2 Made some changes to the docstrings and comments in 'checks.py'. Furthermore, changed the error messages in 'checks.py'. Changed 'test_checks.py' according to the changes in the error messages.
      590db83 Changed the line breaks in 'autoqc_runner.py' so that they break before binary operations.
      3cda5f3 Made some changes to the docstrings and comments in 'autoqc_runner.py'.
      eaa4fe9 Made some changes to the docstrings and comments in 'autoqc_config_reader.py'.
      aea5709 Removed the comments from the 'YAML' files in the 'configs' folder as they were unneeded.
      c26e065 Changed the docstring in 'test_checks.py'. Added docstrings to each test in that file.
      53f6605 Rewrote the 'demo_program.py'. The functionality however did not change.
      b86667d Made some changes and additions to the 'README.md'.
      2667340 Removed 'pytest', 'pytest-cov', and 'Sphinx' from the 'requirements.txt'. Changed the 'README.md' accordingly.
      7707558 Added a test to 'test_checks.py'. This will test if the 'check_limits' function will return the correct limits in the edge case of the limits being set to 0 and 1 respectively.
      0af6b1c Removed the PYTHONPATH manipulation from 'test_checks.py'. The 'README.md' now reflects the new working way to execute the 'pytest' command.
      df4136f Split the tests in 'test_checks.py' into different functions. Added a test with correct values for the OK limit and the questionable limit.
      f4c6dd8 Moved all checks of parameters and settings to the 'checks.py' file. With these changes the 'test_autoqc_runner.py' was renamed to 'test_checks.py'. Also, the tests in 'test_checks.py' were changed to work with the new version of the checks. Additionally, the version of the 'pytest-cov' package was updated in the 'requirements.txt' file.
      b9e76ce Added a missing point to the header comment of the 'test_autoqc_runner.py'.
      6425b4d Made a few minor changes to the 'README.md'.
      8ab7cd4 The 'demo_program.py' now chooses its configuration by the given variable in the metadata if they are found.
      cb0ed70 Added the 'checks.py' and the 'plotter.py' to the documentation.
      4a746d5 Added a 'checks.py' file. This file holds functions to check all the different inputs into the framework. For now moved the 'check_config' function there. Each test class got a static function which returns the parameters needed for each test and their respective data types.
      1d2fae6 Added the latitude and longitude to the plot title. Slightly changed the format of the text in the cells in the statistics tables.
      f474f23 Changed the 'DataGetter'. They do not remove the time zone information anymore. Additionally, unified the header comments in the source files.
      
      git-subtree-dir: autoqc
      git-subtree-split: a9098124d97c70801ea45a8764a165d7692c9893
      58ca2016
  5. 25 May, 2021 3 commits
    • Sabine Schröder's avatar
      Squashed 'autoqc/' changes from a9cce39..177c4d1 · e8bb6c62
      Sabine Schröder authored
      177c4d1 The Automated Data Quality Control will now return a dictionary that contains the QC-Score, the flags and the settings used in the run. Furthermore, it contains the results per group of tests and per test. The 'demo_program.py' was changed accordingly as well as the 'README.md'. The 'Plotter' class takes advantage of the extra information to save some calculations.
      
      git-subtree-dir: autoqc
      git-subtree-split: 177c4d1b857a428c526b3741ea362565a77d1086
      e8bb6c62
    • Sabine Schröder's avatar
      0df880ae
    • Sabine Schröder's avatar
      Squashed 'autoqc/' changes from de18f11..a9cce39 · 4489d489
      Sabine Schröder authored
      a9cce39 Added some tables to the 'Plotter'. They will show the number of points in the categories 'OK', 'Questionable', and 'Erroneous' per year. This required some changes to the existing 'Plotter' class. Removed the 'psutil' package from 'requirements.txt' as it is not needed.
      71bb6b8 The Automated Data Quality Control can now handle metadata. Currently these will only be used to create titles for the plots. The 'demo_program.py' was changed accordingly. The version of 'Sphinx' was updated in the 'requirements.txt'.
      8bd3ba0 Created a standalone Plotter. Removed all plotting routines from 'autoqc_runner.py'. Added a switch to toggle if plots are created or not. Renamed the keyword argument 'plot_output' to 'outfile' in the 'run_autoqc' interface. All plotting is now done with the 'plotly' package. Therefore, 'matplotlib' was removed from 'requirements.txt' and 'psutil' and 'kaleido' were added to enable static image export.
      4212f77 Updated package versions in 'requirements.txt'.
      508f909 Updated package versions in 'requirements.txt'.
      608bd2a Changed the format of the input files used in the 'demo_program.py'.
      2322bf5 Changed the value of 'sigma' for the 'SigmaTest' in the O3 'standard' configuration. This change was done in both the YAML as well as the JSON version of the configuration.
      ffec376 Added a '.gitignore'. It will ignore any Python byte code files, so all files with 'pyc' extension.
      d07c323 Added the 'autoqc_config_reader'. Created a 'config' directory to which configurations will be saved. Added a first configuration for ozone time series with the name 'standard' once in YAML and once in JSON format. This includes all currently implemented tests. Added an abstract about the reader in the 'README.md'. Expanded the 'index.rst' for documenting the reader with Sphinx. Moved the 'check_config' function to 'autoqc_config_reader.py'. The demo program now uses the config reader.
      5b66703 Changed the 'requirements.txt'.
      80709dd Added a 'min_periods' parameter to the 'rolling' operation in the 'run' method of the 'PositiveSpikeTest' class and the 'NegativeSpikeTest' class.
      REVERT: de18f11 Added a 'min_periods' parameter to the 'PositiveSpikeTest' class and the 'NegativeSpikeTest' class.
      
      git-subtree-dir: autoqc
      git-subtree-split: a9cce39be22ff78b59b9e0865e1493c4d92c59dc
      4489d489
  6. 29 Apr, 2021 2 commits
    • Sabine Schröder's avatar
      Squashed 'autoqc/' changes from b891849..de18f11 · 8670e1c8
      Sabine Schröder authored
      de18f11 Added a 'min_periods' parameter to the 'PositiveSpikeTest' class and the 'NegativeSpikeTest' class.
      eb871c7 Added the 'BeforeNaNTest' and the 'AfterNaNTest' to 'tests.py'. Added these two tests to the 'demo_program.py'.
      8d15200 Added information on setting the limits in 'README.md'
      
      git-subtree-dir: autoqc
      git-subtree-split: de18f11ff5b7b59230960615942c93a7a308022d
      8670e1c8
    • Sabine Schröder's avatar
      Squashed 'autoqc/' changes from fda9716..b891849 · 5e471020
      Sabine Schröder authored
      b891849 Added a new keyword argument to be able to choose what the return values are. Now either the flags, the QC-Score or both can be returned.
      19bdd1d Changed the workflow inside the class 'AutoQCWorkflow'. Now the original index of the time series will be unchanged in terms of order and time zone. The result will have the exact same index as the given time series. Moreover, all statistical tests will now return a NaN value for a single point in the time series if the time series has a NaN value there.
      97d8f5f Added a 'requirements.txt'
      
      git-subtree-dir: autoqc
      git-subtree-split: b891849692e21bbd684bfc50e17f93e807451ab6
      5e471020
  7. 18 Apr, 2021 1 commit
  8. 01 Apr, 2021 1 commit
  9. 30 Mar, 2021 2 commits
  10. 16 Mar, 2021 1 commit
  11. 07 Mar, 2021 1 commit
  12. 07 Jul, 2020 1 commit
  13. 31 May, 2020 1 commit
  14. 26 May, 2020 1 commit