Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
MLAir
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container registry
Model registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
esde
machine-learning
MLAir
Merge requests
!318
Resolve "release v1.4.0"
Code
Review changes
Check out branch
Download
Patches
Plain diff
Merged
Resolve "release v1.4.0"
release_v1.4.0
into
master
Overview
0
Commits
229
Pipelines
3
Changes
3
Merged
Ghost User
requested to merge
release_v1.4.0
into
master
3 years ago
Overview
0
Commits
229
Pipelines
3
Changes
3
Expand
Closes
#317 (closed)
Edited
3 years ago
by
Ghost User
0
0
Merge request reports
Viewing commit
6917b04a
Prev
Next
Show latest version
3 files
+
51
−
0
Inline
Compare changes
Side-by-side
Inline
Show whitespace changes
Show one file at a time
Files
3
Search (e.g. *.vue) (Ctrl+P)
6917b04a
MLAir can store parameters during preprocessing from train subset when using store_attributes
· 6917b04a
leufen1
authored
4 years ago
mlair/data_handler/abstract_data_handler.py
+
26
−
0
Options
@@ -11,6 +11,7 @@ from mlair.helpers import remove_items
class
AbstractDataHandler
:
_requirements
=
[]
_store_attributes
=
[]
def
__init__
(
self
,
*
args
,
**
kwargs
):
pass
@@ -32,6 +33,31 @@ class AbstractDataHandler:
list_of_args
=
arg_spec
.
args
+
arg_spec
.
kwonlyargs
return
remove_items
(
list_of_args
,
[
"
self
"
]
+
list
(
args
))
@classmethod
def
store_attributes
(
cls
):
"""
Let MLAir know that some data should be stored in the data store. This is used for calculations on the train
subset that should be applied to validation and test subset.
To work properly, add a class variable cls._store_attributes to your data handler. If your custom data handler
is constructed on different data handlers (e.g. like the DefaultDataHandler), it is required to overwrite the
get_store_attributs method in addition to return attributes from the corresponding subclasses. This is not
required, if only attributes from the main class are to be returned.
Note, that MLAir will store these attributes with the data handler
'
s identification. This depends on the custom
data handler setting. When loading an attribute from the data handler, it is therefore required to extract the
right information by using the class identification. In case of the DefaultDataHandler this can be achieved to
convert all keys of the attribute to string and compare these with the station parameter.
"""
return
list
(
set
(
cls
.
_store_attributes
))
def
get_store_attributes
(
self
):
"""
Returns all attribute names and values that are indicated by the store_attributes method.
"""
attr_dict
=
{}
for
attr
in
self
.
store_attributes
():
attr_dict
[
attr
]
=
self
.
__getattribute__
(
attr
)
return
attr_dict
@classmethod
def
transformation
(
cls
,
*
args
,
**
kwargs
):
return
None
Loading