Skip to content
Snippets Groups Projects
Select Git revision
  • 48785ddbc94e0830ad278d678dbb12dd05ef585b
  • master default protected
  • tf2
  • tf2_pytorch
  • issue_3
  • issue_2
  • 2019a
  • juwels_2019a
  • jureca_2019_a
9 results

train_lenet.py

Blame
    • Background Jobs.ipynb 15.57 KiB

    Simple interactive bacgkround jobs with IPython

    We start by loading the backgroundjobs library and defining a few trivial functions to illustrate things with.

    In [1]: 
    from IPython.lib import backgroundjobs as bg

import sys
import time

def sleepfunc(interval=2, *a, **kw):
    args = dict(interval=interval,
                args=a,
                kwargs=kw)
    time.sleep(interval)
    return args

def diefunc(interval=2, *a, **kw):
    time.sleep(interval)
    raise Exception("Dead job with interval %s" % interval)

def printfunc(interval=1, reps=5):
    for n in range(reps):
        time.sleep(interval)
        print('In the background... %i' % n)
        sys.stdout.flush()
    print('All done!')
    sys.stdout.flush()

    Now, we can create a job manager (called simply jobs) and use it to submit new jobs.

    Run the cell below, it will show when the jobs start. Wait a few seconds until you see the 'all done' completion message:

    In [2]: 
    jobs = bg.BackgroundJobManager()

# Start a few jobs, the first one will have ID # 0
jobs.new(sleepfunc, 4)
jobs.new(sleepfunc, kw={'reps':2})
jobs.new('printfunc(1,3)')
    Out [2]: 
    Starting job # 0 in a separate thread.
Starting job # 2 in a separate thread.
Starting job # 3 in a separate thread.
    Out [2]: 
    <BackgroundJob #3: printfunc(1,3)>
    Out [2]: 
    In the background... 0
In the background... 1
In the background... 2
All done!

    You can check the status of your jobs at any time:

    In [3]: 
    jobs.status()
    Out [3]: 
    Completed jobs:
0 : <function sleepfunc at 0x10521f2f0>
2 : <function sleepfunc at 0x10521f2f0>
3 : printfunc(1,3)

    For any completed job, you can get its result easily:

    In [4]: 
    jobs[0].result
    Out [4]: 
    {'args': (), 'interval': 4, 'kwargs': {}}

    Errors and tracebacks

    The jobs manager tries to help you with debugging:

    In [5]: 
    # This makes a couple of jobs which will die.  Let's keep a reference to
# them for easier traceback reporting later
diejob1 = jobs.new(diefunc, 1)
diejob2 = jobs.new(diefunc, 2)
    Out [5]: 
    Starting job # 4 in a separate thread.
Starting job # 5 in a separate thread.

    You can get the traceback of any dead job. Run the line below again interactively until it prints a traceback (check the status of the job):

    In [6]: 
    print("Status of diejob1: %s" % diejob1.status)
diejob1.traceback()  # jobs.traceback(4) would also work here, with the job number
    Out [6]: 
    Status of diejob1: Dead (Exception), call jobs.traceback() for details
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
/Users/minrk/dev/ip/mine/IPython/lib/backgroundjobs.py in call(self)
    489 
    490     def call(self):
--> 491         return self.func(*self.args, **self.kwargs)

<ipython-input-1-169e49434ce0> in diefunc(interval, *a, **kw)
     13 def diefunc(interval=2, *a, **kw):
     14     time.sleep(interval)
---> 15     raise Exception("Dead job with interval %s" % interval)
     16 
     17 def printfunc(interval=1, reps=5):

Exception: Dead job with interval 1

    This will print all tracebacks for all dead jobs:

    In [7]: 
    jobs.traceback()
    Out [7]: 
    Traceback for: <BackgroundJob #4: <function diefunc at 0x10521f7b8>>
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
/Users/minrk/dev/ip/mine/IPython/lib/backgroundjobs.py in call(self)
    489 
    490     def call(self):
--> 491         return self.func(*self.args, **self.kwargs)

<ipython-input-1-169e49434ce0> in diefunc(interval, *a, **kw)
     13 def diefunc(interval=2, *a, **kw):
     14     time.sleep(interval)
---> 15     raise Exception("Dead job with interval %s" % interval)
     16 
     17 def printfunc(interval=1, reps=5):

Exception: Dead job with interval 1

Traceback for: <BackgroundJob #5: <function diefunc at 0x10521f7b8>>
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
/Users/minrk/dev/ip/mine/IPython/lib/backgroundjobs.py in call(self)
    489 
    490     def call(self):
--> 491         return self.func(*self.args, **self.kwargs)

<ipython-input-1-169e49434ce0> in diefunc(interval, *a, **kw)
     13 def diefunc(interval=2, *a, **kw):
     14     time.sleep(interval)
---> 15     raise Exception("Dead job with interval %s" % interval)
     16 
     17 def printfunc(interval=1, reps=5):

Exception: Dead job with interval 2

    The job manager can be flushed of all completed jobs at any time:

    In [8]: 
    jobs.flush()
    Out [8]: 
    Flushing 3 Completed jobs.
Flushing 2 Dead jobs.

    After that, the status is simply empty:

    In [9]: 
    jobs.status()

    Jobs have a .join method that lets you wait on their thread for completion:

    In [10]: 
    j = jobs.new(sleepfunc, 2)
j.join?
    Out [10]: 
    Starting job # 0 in a separate thread.

    Exercise

    1. Start a new job that calls sleepfunc with a 5-second wait
    2. Print a short message that indicates you are waiting (note: you'll need to flush stdout to see that print output appear).
    3. Wait on the job and then print its result.