Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
MLAir
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container registry
Model registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
GitLab community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
esde
machine-learning
MLAir
Commits
dd825dba
Commit
dd825dba
authored
5 years ago
by
lukas leufen
Browse files
Options
Downloads
Patches
Plain Diff
docstrings for run environment
parent
423d59cd
Branches
Branches containing commit
Tags
Tags containing commit
3 merge requests
!125
Release v0.10.0
,
!124
Update Master to new version v0.10.0
,
!91
WIP: Resolve "create sphinx docu"
Pipeline
#35439
passed
5 years ago
Stage: test
Stage: docs
Stage: pages
Stage: deploy
Changes
2
Pipelines
1
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
src/run_modules/run_environment.py
+79
-7
79 additions, 7 deletions
src/run_modules/run_environment.py
src/run_modules/training.py
+1
-1
1 addition, 1 deletion
src/run_modules/training.py
with
80 additions
and
8 deletions
src/run_modules/run_environment.py
+
79
−
7
View file @
dd825dba
"""
Implementation of run environment.
"""
__author__
=
"
Lukas Leufen
"
__date__
=
'
2019-11-25
'
...
...
@@ -14,38 +16,107 @@ from src.helpers import TimeTracking
class
RunEnvironment
(
object
):
"""
basic run class to measure execution time. Either call this class calling it by
'
with
'
or delete the class instance
after finishing the measurement. The duration result is logged.
Basic run class to measure execution time.
Either call this class by
'
with
'
statement or delete the class instance after finishing the measurement. The
duration result is logged.
.. code-block:: python
>>>
with
RunEnvironment
():
<
your
code
>
INFO
:
RunEnvironment
started
...
INFO
:
RunEnvironment
finished
after
00
:
00
:
04
(
hh
:
mm
:
ss
)
If you want to embed your custom module in a RunEnvironment, you can easily call it inside the with statement. If
you want to exchange between different modules in addition, create your module as inheritance of the RunEnvironment
and call it after you initialised the RunEnvironment itself.
.. code-block:: python
class CustomClass(RunEnvironment):
def __init__(self):
super().__init__()
...
...
>>>
with
RunEnvironment
():
CustomClass
()
INFO
:
RunEnvironment
started
INFO
:
CustomClass
started
INFO
:
CustomClass
finished
after
00
:
00
:
04
(
hh
:
mm
:
ss
)
INFO
:
RunEnvironment
finished
after
00
:
00
:
04
(
hh
:
mm
:
ss
)
All data that is stored in the data store will be available for all other modules that inherit from RunEnvironment
as long the RunEnvironemnt base class is running. If the base class is deleted either by hand or on exit of the with
statement, this storage is cleared.
.. code-block:: python
class CustomClassA(RunEnvironment):
def __init__(self):
super().__init__()
self.data_store.set(
"
testVar
"
, 12)
class CustomClassB(RunEnvironment):
def __init__(self):
super().__init__()
self.test_var = self.data_store.get(
"
testVar
"
)
logging.info(f
"
testVar = {self.test_var}
"
)
>>>
with
RunEnvironment
():
CustomClassA
()
CustomClassB
()
INFO
:
RunEnvironment
started
INFO
:
CustomClassA
started
INFO
:
CustomClassA
finished
after
00
:
00
:
01
(
hh
:
mm
:
ss
)
INFO
:
CustomClassB
started
INFO
:
testVar
=
12
INFO
:
CustomClassB
finished
after
00
:
00
:
02
(
hh
:
mm
:
ss
)
INFO
:
RunEnvironment
finished
after
00
:
00
:
03
(
hh
:
mm
:
ss
)
"""
# set data store and logger (both are mutable!)
del_by_exit
=
False
data_store
=
DataStoreObject
()
logger
=
Logger
()
def
__init__
(
self
):
"""
Starts time tracking automatically and logs as info.
"""
"""
Start time tracking automatically and logs as info.
"""
self
.
time
=
TimeTracking
()
logging
.
info
(
f
"
{
self
.
__class__
.
__name__
}
started
"
)
def
__del__
(
self
):
"""
This is the class finalizer. The code is not executed if already called by exit method to prevent duplicated
logging (__exit__ is always executed before __del__) it this class was used in a with statement.
Finalise class.
Only stop time tracking, if not already called by exit method to prevent duplicated logging (__exit__ is always
executed before __del__) it this class was used in a with statement. If instance is called as base class and
not as inheritance from this class, log file is copied and data store is cleared.
"""
if
not
self
.
del_by_exit
:
self
.
time
.
stop
()
logging
.
info
(
f
"
{
self
.
__class__
.
__name__
}
finished after
{
self
.
time
}
"
)
self
.
del_by_exit
=
True
# copy log file and clear data store only if called as base class and not as super class
if
self
.
__class__
.
__name__
==
"
RunEnvironment
"
:
self
.
__copy_log_file
()
self
.
data_store
.
clear_data_store
()
def
__enter__
(
self
):
"""
Enter run environment.
"""
return
self
def
__exit__
(
self
,
exc_type
,
exc_val
,
exc_tb
):
"""
Exit run environment.
"""
if
exc_type
:
logging
.
error
(
exc_val
,
exc_info
=
(
exc_type
,
exc_val
,
exc_tb
))
self
.
__del__
()
...
...
@@ -65,4 +136,5 @@ class RunEnvironment(object):
@staticmethod
def
do_stuff
(
length
=
2
):
"""
Just a placeholder method for testing without any sense.
"""
time
.
sleep
(
length
)
This diff is collapsed.
Click to expand it.
src/run_modules/training.py
+
1
−
1
View file @
dd825dba
...
...
@@ -31,7 +31,7 @@ class Training(RunEnvironment):
* `batch_size` [model]
* `epochs` [model]
* `callbacks` [model]
* `model_name
*
[model]
* `model_name
`
[model]
* `experiment_name` [.]
* `experiment_path` [.]
* `trainable` [.]
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment