Skip to content
Snippets Groups Projects
Select Git revision
  • master
  • integration
2 results

Cell Magics.ipynb

Blame
  • Forked from jupyter4jsc / j4j_notebooks
    142 commits behind, 1 commit ahead of the upstream repository.
    Cell Magics.ipynb 17.12 KiB

    Cell Magics in IPython

    IPython has a system of commands we call 'magics' that provide a mini command language that is orthogonal to the syntax of Python and is extensible by the user with new commands. Magics are meant to be typed interactively, so they use command-line conventions, such as using whitespace for separating arguments, dashes for options and other conventions typical of a command-line environment.

    Magics come in two kinds:

    • Line magics: these are commands prepended by one % character and whose arguments only extend to the end of the current line.
    • Cell magics: these use two percent characters as a marker (%%), and they receive as argument both the current line where they are declared and the whole body of the cell. Note that cell magics can only be used as the first line in a cell, and as a general principle they can't be 'stacked' (i.e. you can only use one cell magic per cell). A few of them, because of how they operate, can be stacked, but that is something you will discover on a case by case basis.

    The %lsmagic magic is used to list all available magics, and it will show both line and cell magics currently defined:

    In [1]:
    %lsmagic
    Out [1]:
    Available line magics:
    %alias  %alias_magic  %autoawait  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %colors  %conda  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %man  %matplotlib  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %pip  %popd  %pprint  %precision  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode
    
    Available cell magics:
    %%!  %%HTML  %%SVG  %%bash  %%capture  %%debug  %%file  %%html  %%javascript  %%js  %%latex  %%markdown  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile
    
    Automagic is ON, % prefix IS NOT needed for line magics.

    Since in the introductory section we already covered the most frequently used line magics, we will focus here on the cell magics, which offer a great amount of power.

    Let's load matplotlib and numpy so we can use numerics/plotting at will later on.

    In [2]:
    %matplotlib inline
    import numpy as np
    import matplotlib.pyplot as plt

    Some simple cell magics

    Timing the execution of code; the 'timeit' magic exists both in line and cell form:

    In [3]:
    %timeit np.linalg.eigvals(np.random.rand(100,100))
    Out [3]:
    4.54 ms ± 72.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
    
    In [4]:
    %%timeit a = np.random.rand(100, 100)
    np.linalg.eigvals(a)
    Out [4]:
    4.69 ms ± 156 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
    

    The %%capture magic can be used to capture the stdout/err of any block of python code, either to discard it (if it's noise to you) or to store it in a variable for later use:

    In [5]:
    %%capture capt
    from __future__ import print_function
    import sys
    print('Hello stdout')
    print('and stderr', file=sys.stderr)
    In [6]:
    capt.stdout, capt.stderr
    Out [6]:
    ('Hello stdout\n', 'and stderr\n')
    In [7]:
    capt.show()
    Out [7]:
    Hello stdout
    
    Out [7]:
    and stderr
    

    The %%writefile magic is a very useful tool that writes the cell contents as a named file:

    In [8]:
    %%writefile foo.py
    print('Hello world')
    Out [8]:
    Writing foo.py
    
    In [9]:
    %run foo
    Out [9]:
    Hello world
    

    Magics for running code under other interpreters

    IPython has a %%script cell magic, which lets you run a cell in a subprocess of any interpreter on your system, such as: bash, ruby, perl, zsh, R, etc.

    It can even be a script of your own, which expects input on stdin.

    To use it, simply pass a path or shell command to the program you want to run on the %%script line, and the rest of the cell will be run by that script, and stdout/err from the subprocess are captured and displayed.

    In [10]:
    %%script python2
    import sys
    print 'hello from Python %s' % sys.version
    Out [10]:
    hello from Python 2.7.5 (default, Jun 20 2019, 20:27:34) 
    [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
    
    In [11]:
    %%script python3
    import sys
    print('hello from Python: %s' % sys.version)
    Out [11]:
    hello from Python: 3.6.8 (default, Oct 17 2019, 14:24:17) 
    [GCC 8.3.0]
    

    IPython also creates aliases for a few common interpreters, such as bash, ruby, perl, etc.

    These are all equivalent to %%script <name>

    In [14]:
    %%ruby
    puts "Hello from Ruby #{RUBY_VERSION}"
    In [15]:
    %%bash
    echo "hello from $BASH"
    Out [15]:
    hello from /usr/bin/bash
    

    Capturing output

    You can also capture stdout/err from these subprocesses into Python variables, instead of letting them go directly to stdout/err

    In [16]:
    %%bash
    echo "hi, stdout"
    echo "hello, stderr" >&2
    
    Out [16]:
    hi, stdout
    
    Out [16]:
    hello, stderr
    
    In [17]:
    %%bash --out output --err error
    echo "hi, stdout"
    echo "hello, stderr" >&2
    In [18]:
    print(error)
    print(output)
    Out [18]:
    hello, stderr
    
    hi, stdout
    
    

    Background Scripts

    These scripts can be run in the background, by adding the --bg flag.

    When you do this, output is discarded unless you use the --out/err flags to store output as above.

    In [19]:
    %%ruby --bg --out ruby_lines
    for n in 1...10
        sleep 1
        puts "line #{n}"
        STDOUT.flush
    end
    Out [19]:
    Couldn't find program: 'ruby'
    

    When you do store output of a background thread, these are the stdout/err pipes, rather than the text of the output.

    In [20]:
    #ruby_lines
    In [22]:
    #print(ruby_lines.read().decode('utf8'))

    Cleanup

    In [1]:
    !rm -f foo.py