Commit e64aa9c6 authored by Jens Henrik Goebbert's avatar Jens Henrik Goebbert
Browse files

clear output

parent 9c2ba215
%% Cell type:markdown id: tags:
# Simple interactive bacgkround jobs with IPython
We start by loading the `backgroundjobs` library and defining a few trivial functions to illustrate things with.
%% Cell type:code id: tags:
``` python
from IPython.lib import backgroundjobs as bg
import sys
import time
def sleepfunc(interval=2, *a, **kw):
args = dict(interval=interval,
args=a,
kwargs=kw)
time.sleep(interval)
return args
def diefunc(interval=2, *a, **kw):
time.sleep(interval)
raise Exception("Dead job with interval %s" % interval)
def printfunc(interval=1, reps=5):
for n in range(reps):
time.sleep(interval)
print('In the background... %i' % n)
sys.stdout.flush()
print('All done!')
sys.stdout.flush()
```
%% Cell type:markdown id: tags:
Now, we can create a job manager (called simply `jobs`) and use it to submit new jobs.
Run the cell below, it will show when the jobs start. Wait a few seconds until you see the 'all done' completion message:
%% Cell type:code id: tags:
``` python
jobs = bg.BackgroundJobManager()
# Start a few jobs, the first one will have ID # 0
jobs.new(sleepfunc, 4)
jobs.new(sleepfunc, kw={'reps':2})
jobs.new('printfunc(1,3)')
```
%% Output
Starting job # 0 in a separate thread.
Starting job # 2 in a separate thread.
Starting job # 3 in a separate thread.
<BackgroundJob #3: printfunc(1,3)>
In the background... 0
In the background... 1
In the background... 2
All done!
%% Cell type:markdown id: tags:
You can check the status of your jobs at any time:
%% Cell type:code id: tags:
``` python
jobs.status()
```
%% Output
Completed jobs:
0 : <function sleepfunc at 0x10521f2f0>
2 : <function sleepfunc at 0x10521f2f0>
3 : printfunc(1,3)
%% Cell type:markdown id: tags:
For any completed job, you can get its result easily:
%% Cell type:code id: tags:
``` python
jobs[0].result
```
%% Output
{'args': (), 'interval': 4, 'kwargs': {}}
%% Cell type:markdown id: tags:
## Errors and tracebacks
%% Cell type:markdown id: tags:
The jobs manager tries to help you with debugging:
%% Cell type:code id: tags:
``` python
# This makes a couple of jobs which will die. Let's keep a reference to
# them for easier traceback reporting later
diejob1 = jobs.new(diefunc, 1)
diejob2 = jobs.new(diefunc, 2)
```
%% Output
Starting job # 4 in a separate thread.
Starting job # 5 in a separate thread.
%% Cell type:markdown id: tags:
You can get the traceback of any dead job. Run the line
below again interactively until it prints a traceback (check the status
of the job):
%% Cell type:code id: tags:
``` python
print("Status of diejob1: %s" % diejob1.status)
diejob1.traceback() # jobs.traceback(4) would also work here, with the job number
```
%% Output
Status of diejob1: Dead (Exception), call jobs.traceback() for details
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
/Users/minrk/dev/ip/mine/IPython/lib/backgroundjobs.py in call(self)
 489 
 490 def call(self):
--> 491 return self.func(*self.args, **self.kwargs)

<ipython-input-1-169e49434ce0> in diefunc(interval, *a, **kw)
 13 def diefunc(interval=2, *a, **kw):
 14 time.sleep(interval)
---> 15 raise Exception("Dead job with interval %s" % interval)
 16 
 17 def printfunc(interval=1, reps=5):
Exception: Dead job with interval 1
%% Cell type:markdown id: tags:
This will print all tracebacks for all dead jobs:
%% Cell type:code id: tags:
``` python
jobs.traceback()
```
%% Output
Traceback for: <BackgroundJob #4: <function diefunc at 0x10521f7b8>>
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
/Users/minrk/dev/ip/mine/IPython/lib/backgroundjobs.py in call(self)
 489 
 490 def call(self):
--> 491 return self.func(*self.args, **self.kwargs)

<ipython-input-1-169e49434ce0> in diefunc(interval, *a, **kw)
 13 def diefunc(interval=2, *a, **kw):
 14 time.sleep(interval)
---> 15 raise Exception("Dead job with interval %s" % interval)
 16 
 17 def printfunc(interval=1, reps=5):
Exception: Dead job with interval 1
Traceback for: <BackgroundJob #5: <function diefunc at 0x10521f7b8>>
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
/Users/minrk/dev/ip/mine/IPython/lib/backgroundjobs.py in call(self)
 489 
 490 def call(self):
--> 491 return self.func(*self.args, **self.kwargs)

<ipython-input-1-169e49434ce0> in diefunc(interval, *a, **kw)
 13 def diefunc(interval=2, *a, **kw):
 14 time.sleep(interval)
---> 15 raise Exception("Dead job with interval %s" % interval)
 16 
 17 def printfunc(interval=1, reps=5):
Exception: Dead job with interval 2
%% Cell type:markdown id: tags:
The job manager can be flushed of all completed jobs at any time:
%% Cell type:code id: tags:
``` python
jobs.flush()
```
%% Output
Flushing 3 Completed jobs.
Flushing 2 Dead jobs.
%% Cell type:markdown id: tags:
After that, the status is simply empty:
%% Cell type:code id: tags:
``` python
jobs.status()
```
%% Cell type:markdown id: tags:
Jobs have a `.join` method that lets you wait on their thread for completion:
%% Cell type:code id: tags:
``` python
j = jobs.new(sleepfunc, 2)
j.join?
```
%% Output
Starting job # 0 in a separate thread.
%% Cell type:markdown id: tags:
## Exercise
1. Start a new job that calls `sleepfunc` with a 5-second wait
2. Print a short message that indicates you are waiting (note: you'll need to flush stdout to see that print output appear).
3. Wait on the job and then print its result.
......