Commit e64aa9c6 authored by Jens Henrik Goebbert's avatar Jens Henrik Goebbert
Browse files

clear output

parent 9c2ba215
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Simple interactive bacgkround jobs with IPython # Simple interactive bacgkround jobs with IPython
We start by loading the `backgroundjobs` library and defining a few trivial functions to illustrate things with. We start by loading the `backgroundjobs` library and defining a few trivial functions to illustrate things with.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
from IPython.lib import backgroundjobs as bg from IPython.lib import backgroundjobs as bg
import sys import sys
import time import time
def sleepfunc(interval=2, *a, **kw): def sleepfunc(interval=2, *a, **kw):
args = dict(interval=interval, args = dict(interval=interval,
args=a, args=a,
kwargs=kw) kwargs=kw)
time.sleep(interval) time.sleep(interval)
return args return args
def diefunc(interval=2, *a, **kw): def diefunc(interval=2, *a, **kw):
time.sleep(interval) time.sleep(interval)
raise Exception("Dead job with interval %s" % interval) raise Exception("Dead job with interval %s" % interval)
def printfunc(interval=1, reps=5): def printfunc(interval=1, reps=5):
for n in range(reps): for n in range(reps):
time.sleep(interval) time.sleep(interval)
print('In the background... %i' % n) print('In the background... %i' % n)
sys.stdout.flush() sys.stdout.flush()
print('All done!') print('All done!')
sys.stdout.flush() sys.stdout.flush()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Now, we can create a job manager (called simply `jobs`) and use it to submit new jobs. Now, we can create a job manager (called simply `jobs`) and use it to submit new jobs.
Run the cell below, it will show when the jobs start. Wait a few seconds until you see the 'all done' completion message: Run the cell below, it will show when the jobs start. Wait a few seconds until you see the 'all done' completion message:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
jobs = bg.BackgroundJobManager() jobs = bg.BackgroundJobManager()
# Start a few jobs, the first one will have ID # 0 # Start a few jobs, the first one will have ID # 0
jobs.new(sleepfunc, 4) jobs.new(sleepfunc, 4)
jobs.new(sleepfunc, kw={'reps':2}) jobs.new(sleepfunc, kw={'reps':2})
jobs.new('printfunc(1,3)') jobs.new('printfunc(1,3)')
``` ```
%% Output
Starting job # 0 in a separate thread.
Starting job # 2 in a separate thread.
Starting job # 3 in a separate thread.
<BackgroundJob #3: printfunc(1,3)>
In the background... 0
In the background... 1
In the background... 2
All done!
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
You can check the status of your jobs at any time: You can check the status of your jobs at any time:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
jobs.status() jobs.status()
``` ```
%% Output
Completed jobs:
0 : <function sleepfunc at 0x10521f2f0>
2 : <function sleepfunc at 0x10521f2f0>
3 : printfunc(1,3)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
For any completed job, you can get its result easily: For any completed job, you can get its result easily:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
jobs[0].result jobs[0].result
``` ```
%% Output
{'args': (), 'interval': 4, 'kwargs': {}}
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Errors and tracebacks ## Errors and tracebacks
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The jobs manager tries to help you with debugging: The jobs manager tries to help you with debugging:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# This makes a couple of jobs which will die. Let's keep a reference to # This makes a couple of jobs which will die. Let's keep a reference to
# them for easier traceback reporting later # them for easier traceback reporting later
diejob1 = jobs.new(diefunc, 1) diejob1 = jobs.new(diefunc, 1)
diejob2 = jobs.new(diefunc, 2) diejob2 = jobs.new(diefunc, 2)
``` ```
%% Output
Starting job # 4 in a separate thread.
Starting job # 5 in a separate thread.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
You can get the traceback of any dead job. Run the line You can get the traceback of any dead job. Run the line
below again interactively until it prints a traceback (check the status below again interactively until it prints a traceback (check the status
of the job): of the job):
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
print("Status of diejob1: %s" % diejob1.status) print("Status of diejob1: %s" % diejob1.status)
diejob1.traceback() # jobs.traceback(4) would also work here, with the job number diejob1.traceback() # jobs.traceback(4) would also work here, with the job number
``` ```
%% Output
Status of diejob1: Dead (Exception), call jobs.traceback() for details
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
/Users/minrk/dev/ip/mine/IPython/lib/backgroundjobs.py in call(self)
 489 
 490 def call(self):
--> 491 return self.func(*self.args, **self.kwargs)

<ipython-input-1-169e49434ce0> in diefunc(interval, *a, **kw)
 13 def diefunc(interval=2, *a, **kw):
 14 time.sleep(interval)
---> 15 raise Exception("Dead job with interval %s" % interval)
 16 
 17 def printfunc(interval=1, reps=5):
Exception: Dead job with interval 1
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
This will print all tracebacks for all dead jobs: This will print all tracebacks for all dead jobs:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
jobs.traceback() jobs.traceback()
``` ```
%% Output
Traceback for: <BackgroundJob #4: <function diefunc at 0x10521f7b8>>
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
/Users/minrk/dev/ip/mine/IPython/lib/backgroundjobs.py in call(self)
 489 
 490 def call(self):
--> 491 return self.func(*self.args, **self.kwargs)

<ipython-input-1-169e49434ce0> in diefunc(interval, *a, **kw)
 13 def diefunc(interval=2, *a, **kw):
 14 time.sleep(interval)
---> 15 raise Exception("Dead job with interval %s" % interval)
 16 
 17 def printfunc(interval=1, reps=5):
Exception: Dead job with interval 1
Traceback for: <BackgroundJob #5: <function diefunc at 0x10521f7b8>>
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
/Users/minrk/dev/ip/mine/IPython/lib/backgroundjobs.py in call(self)
 489 
 490 def call(self):
--> 491 return self.func(*self.args, **self.kwargs)

<ipython-input-1-169e49434ce0> in diefunc(interval, *a, **kw)
 13 def diefunc(interval=2, *a, **kw):
 14 time.sleep(interval)
---> 15 raise Exception("Dead job with interval %s" % interval)
 16 
 17 def printfunc(interval=1, reps=5):
Exception: Dead job with interval 2
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The job manager can be flushed of all completed jobs at any time: The job manager can be flushed of all completed jobs at any time:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
jobs.flush() jobs.flush()
``` ```
%% Output
Flushing 3 Completed jobs.
Flushing 2 Dead jobs.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
After that, the status is simply empty: After that, the status is simply empty:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
jobs.status() jobs.status()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Jobs have a `.join` method that lets you wait on their thread for completion: Jobs have a `.join` method that lets you wait on their thread for completion:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
j = jobs.new(sleepfunc, 2) j = jobs.new(sleepfunc, 2)
j.join? j.join?
``` ```
%% Output
Starting job # 0 in a separate thread.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Exercise ## Exercise
1. Start a new job that calls `sleepfunc` with a 5-second wait 1. Start a new job that calls `sleepfunc` with a 5-second wait
2. Print a short message that indicates you are waiting (note: you'll need to flush stdout to see that print output appear). 2. Print a short message that indicates you are waiting (note: you'll need to flush stdout to see that print output appear).
3. Wait on the job and then print its result. 3. Wait on the job and then print its result.
......