better understanding why station is removed in preprocessing

added Doing enhancement technical labels

created merge request !276 (merged) to address this issue

mentioned in merge request !276 (merged)

should add warning if two or more target vars are used

New proposal, the code above will grab the error origin, but we are interested in the error location in MLAir.


def f_proc(data_handler, station, name_affix, store, **kwargs):
    """
    Try to create a data handler for given arguments. If build fails, this station does not fulfil all requirements and
    therefore f_proc will return None as indication. On a successfull build, f_proc returns the built data handler and
    the station that was used. This function must be implemented globally to work together with multiprocessing.
    """
    try:
        res = data_handler.build(station, name_affix=name_affix, store_processed_data=store, **kwargs)
    except (AttributeError, EmptyQueryResult, KeyError, requests.ConnectionError, ValueError) as e:
        formatted_lines = traceback.format_exc().splitlines()
        logging.info(f"remove station {station} because it raised an error: {e} -> {' | '.join(f_inspect_error(formatted_lines))}")
        res = None
    return res, station


def f_inspect_error(formatted):
    for i in range(len(formatted) - 1, -1, -1):
        if "mlair/mlair" not in formatted[i]:
            return formatted[i - 3:i]
    return formatted[-3:0]

new error log looks like this:

2021-03-18 14:28:53,115 - INFO: DefaultWorkflow started  [run_environment.py:__init__:103]
2021-03-18 14:28:53,115 - INFO: ExperimentSetup started  [run_environment.py:__init__:103]
2021-03-18 14:28:53,118 - INFO: Experiment path is: /home/felix/PycharmProjects/mlair/testrun_network_hourly  [experiment_setup.py:__init__:265]
2021-03-18 14:28:53,128 - INFO: ExperimentSetup finished after 0:00:01 (hh:mm:ss)  [run_environment.py:__del__:118]
2021-03-18 14:28:53,128 - INFO: PreProcessing started  [run_environment.py:__init__:103]
2021-03-18 14:28:53,128 - INFO: check valid stations started (preprocessing)  [pre_processing.py:validate_station:239]
2021-03-18 14:28:53,134 - INFO: use parallel validate station approach  [pre_processing.py:validate_station:251]
2021-03-18 14:28:53,157 - INFO: running 4 processes in parallel  [pre_processing.py:validate_station:254]
2021-03-18 14:28:57,718 - INFO: set dimensions as coordinates  [data_handler_wrf_chem.py:_set_dims_as_coords:150]
2021-03-18 14:28:57,784 - INFO: set dimensions as coordinates  [data_handler_wrf_chem.py:_set_dims_as_coords:150]
2021-03-18 14:28:57,858 - INFO: set dimensions as coordinates  [data_handler_wrf_chem.py:_set_dims_as_coords:150]
2021-03-18 14:28:57,883 - INFO: set dimensions as coordinates  [data_handler_wrf_chem.py:_set_dims_as_coords:150]
2021-03-18 14:28:57,893 - INFO: setup external coords  [data_handler_wrf_chem.py:_apply_external_coordinates:240]
2021-03-18 14:28:57,960 - INFO: setup external coords  [data_handler_wrf_chem.py:_apply_external_coordinates:240]
2021-03-18 14:28:58,063 - INFO: setup external coords  [data_handler_wrf_chem.py:_apply_external_coordinates:240]
2021-03-18 14:28:58,087 - INFO: setup external coords  [data_handler_wrf_chem.py:_apply_external_coordinates:240]
2021-03-18 14:28:59,765 - INFO: setup_samples finished after 0:00:07 (hh:mm:ss)  [time_tracking.py:__exit__:131]
2021-03-18 14:28:59,773 - INFO: setup_samples finished after 0:00:07 (hh:mm:ss)  [time_tracking.py:__exit__:131]
2021-03-18 14:28:59,780 - INFO: setup_samples finished after 0:00:07 (hh:mm:ss)  [time_tracking.py:__exit__:131]
2021-03-18 14:28:59,787 - INFO: setup_samples finished after 0:00:07 (hh:mm:ss)  [time_tracking.py:__exit__:131]
2021-03-18 14:29:00,371 - ERROR: Error sending result: '(coords__50_7333__7_1000, 'coords__50_7333__7_1000')'. Reason: 'AttributeError("Can't pickle local object 'rolling_window.<locals>.func'",)'  [run_environment.py:__exit__:137]
Traceback (most recent call last):
  File "/home/felix/PycharmProjects/mlair/mlair/workflows/abstract_workflow.py", line 30, in run
    stage(**self._registry_kwargs[pos])
  File "/home/felix/PycharmProjects/mlair/mlair/run_modules/pre_processing.py", line 58, in __init__
    self._run()
  File "/home/felix/PycharmProjects/mlair/mlair/run_modules/pre_processing.py", line 64, in _run
    "preprocessing")  # , store_processed_data=False)
  File "/home/felix/PycharmProjects/mlair/mlair/run_modules/pre_processing.py", line 259, in validate_station
    dh, s = p.get()
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
multiprocessing.pool.MaybeEncodingError: Error sending result: '(coords__50_7333__7_1000, 'coords__50_7333__7_1000')'. Reason: 'AttributeError("Can't pickle local object 'rolling_window.<locals>.func'",)'
2021-03-18 14:29:00,500 - INFO: DefaultWorkflow finished after 0:00:08 (hh:mm:ss)  [run_environment.py:__del__:118]
2021-03-18 14:29:02,284 - INFO: Copy tracker file to /home/felix/PycharmProjects/mlair/testrun_network_hourly/logging/tracking_002.json  [run_environment.py:__save_tracking:155]
2021-03-18 14:29:02,285 - INFO: Move log file to /home/felix/PycharmProjects/mlair/testrun_network_hourly/logging/logging_002.log  [run_environment.py:__move_log_file:143]
Traceback (most recent call last):
  File "/home/felix/PycharmProjects/mlair/run.py", line 80, in <module>
    main(args)
  File "/home/felix/PycharmProjects/mlair/run.py", line 72, in main
    workflow.run()
  File "/home/felix/PycharmProjects/mlair/mlair/workflows/abstract_workflow.py", line 30, in run
    stage(**self._registry_kwargs[pos])
  File "/home/felix/PycharmProjects/mlair/mlair/run_modules/pre_processing.py", line 58, in __init__
    self._run()
  File "/home/felix/PycharmProjects/mlair/mlair/run_modules/pre_processing.py", line 64, in _run
    "preprocessing")  # , store_processed_data=False)
  File "/home/felix/PycharmProjects/mlair/mlair/run_modules/pre_processing.py", line 259, in validate_station
    dh, s = p.get()
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
multiprocessing.pool.MaybeEncodingError: Error sending result: '(coords__50_7333__7_1000, 'coords__50_7333__7_1000')'. Reason: 'AttributeError("Can't pickle local object 'rolling_window.<locals>.func'",)'
Exception ignored in: <bound method RunEnvironment.__del__ of <mlair.run_modules.pre_processing.PreProcessing object at 0x7fed262a3860>>
Traceback (most recent call last):
  File "/home/felix/PycharmProjects/mlair/mlair/run_modules/run_environment.py", line 118, in __del__
  File "/usr/lib/python3.6/logging/__init__.py", line 1902, in info
  File "/usr/lib/python3.6/logging/__init__.py", line 1308, in info
  File "/usr/lib/python3.6/logging/__init__.py", line 1444, in _log
  File "/usr/lib/python3.6/logging/__init__.py", line 1454, in handle
  File "/usr/lib/python3.6/logging/__init__.py", line 1516, in callHandlers
  File "/usr/lib/python3.6/logging/__init__.py", line 865, in handle
  File "/usr/lib/python3.6/logging/__init__.py", line 1071, in emit
  File "/usr/lib/python3.6/logging/__init__.py", line 1061, in _open
NameError: name 'open' is not defined

Process finished with exit code 1

added merged in Develop label and removed Doing label

closed

removed merged in Develop label

better understanding why station is removed in preprocessing

Designs

Child items ...

Activity