00_Introduction to IPython.ipynb

 
 Introduction to IPython and Jupyter Notebook
 
12 June 2023 | Jan H. Meinke
 Jupyter notebooks offer a flexible way to combine python (and other language), scripts, text (such as this one), plots, and other things into one document.
There are two modi: command und edit. You enter edit mode by pressing Enter. You can exit it by pressing Esc. To see the shortcuts for each modus press Esc followed by h if you are using Jupyter Notebook. Look in the Commands tab on the left hand side if you are using JupyterLab.
If you want a new input above your current cell, press a in command mode. If you would like a new cell below your current cell press b.
In the course, we'll use the IPython kernel within the Jupyter notebook. IPython is also available within a regular terminal (ipython) or in its own special terminal-like window called Jupyter QtConsole.
 IPython Shell & QtConsole
 
 Magic and Help
 IPyhon extends the normal Python shell with magic commands. You can get a list of available magic commands using the magic command %lsmagic. Give it a try. Use Shift+Enter to execute the cell. Just pressing Enter only gives you a new line.
 %lsmagic
 
 Not all the magic commands may be obvious to you, so it's a good thing that IPython also includes a help system. If you want to know, for example, what %dhist does im comparison to %hist, you call first %dhist? and then %hist?. Give it a try.
 
 
 You saw that %hist is just an alias for %history.
Some of the magic commands, e.g., %less, %ls, and %man are substitutes for their shell equivalents, others, e.g., %load, %save, %%writefile deal with reading and writing from (to) files. We'll encounter more magic commands as we go along. 
The question mark doesn't just work for magic functions but gives you access to the documentation included with most Python modules. If you want to know, for example, what the random module does, you use random?. Give it a try.
 
 
 If you didn't get the documentation but only Object `random` not found. Try importing the module first. Start by typing im and hit the tab key. Then type r and hit tab again. You'll get a dropdown box with available modules. You can continue typing until your choice is unique or select an item from the list. Give it a try. 
Try random? after importing the module. If you use ?? instead of ? you get the source code.
 
 
 Timing with %timeit
 Let's see how fast random.uniform generates random numbers and compare it with numpy's random.uniform. We'll use the timeit magics for that (more about those later).
 %%timeit 
n = 1
[random.uniform(0, 1) for i in range(n)]
 
 import numpy
 
 n = 1
t = %timeit -o numpy.random.uniform(0, 1, size=n)
print(n * 1. / t.best)
 
 Plotting
 We won't go into the details of using matplotlib for plotting in this course, but a quick introduction can't hurt.
We first need to initialize the interactive plotting routines. The easiest way to do this is to use some magic. 
The command %matplotlib inline sets up interactive plotting support without importing any modules. This allows us to import the modules that we want:
%matplotlib inline
import matplotlib.pyplot as plt

Alternatively, the command %pylab inline sets up interactive plotting and pulls all functions and modules from numpy and matplotlib.pyplot into the namespace.
 %matplotlib inline  
# widget is an interactive alternative to inline
import matplotlib.pyplot as plt
import numpy
 
 Output format
 We can set the output format of matplotlib, too. The following commands change the default format from png to pdf.
 import matplotlib_inline.backend_inline
matplotlib_inline.backend_inline.set_matplotlib_formats('pdf', 'svg')
 
 Plots and labels
 x = numpy.linspace(0, 2 * numpy.pi, 120)
y = numpy.sin(x)
 
 plt.plot(x, y, label=r"$\sin(x)$")
plt.xlim([0, 2 * numpy.pi])
plt.grid(1)
plt.xticks(numpy.linspace(0, 2 * numpy.pi, 9), [0, r"$\frac{\pi}{4}$", 
                                          r"$\frac{\pi}{4}$", r"$\frac{3\pi}{4}$", r"$\pi$", 
                                          r"$\frac{5\pi}{4}$", r"$\frac{3\pi}{2}$", r"$\frac{7\pi}{4}$"
                                          , r"$2\pi$"])
plt.xlabel('x')
plt.legend()
 
 Saving and Loading Files
 Let's say, we want to use the script that we wrote in the section Plotting later from outside the notebook. We can use the %save magic to do that. Each cell has a number In [60] for example. The %save command takes a file name and a space separated list of input cells. The content of the input cells is written to a file. You can give ranges and even reorder things, e.g.,
%save plotsin.py 60 61 62

saves the cells with index 60, 61, and 62 to plotsin.py. I could have also written 60-62 to give the range. Look at the cell indices in the previous section and write them to a file. Give it a try.
 
 
 We can load the file using %load. Try loading the file you just wrote to disk. 
 
 
 The entire content of the file is written to a single cell. We can split (and join) cells (see the Edit menu) if we don't like this. For files that are contained in a single cell, we can use the %%writefile magic.
Join the next three cells and write them to a file called gaussian.py by adding %%writefile filename to the top of the cell.
 
Highlight the next three cells (use the Shift-key to highlight multiple cells)
See the Commands (Ctrl+Shift+C) tab on the left (or press h to check wich key is used for merging cells if you are using a classic Jupyter notebook).
Merge the cells
Write %%writefile gaussian.py at the top of cell
 r = numpy.random.normal(size=1000000)
 
 print("The average is %.3f and the standard deviation is %.3f" % (r.mean(), r.std()))
 
 n, b, p = plt.hist(r, bins=100, density=True)
 
 If you try to run gaussian.py from the command line it won't work because it's missing the imports, but you can run it using ipython --pylab=auto gaussian.py. You can also run the script from IPython using the %run magic, but you have to use -i to run the script within the current namespace. Give it a try.
 %run -i gaussian.py
 
 Load can also load scripts from a URL. Give it a try. Look at the script loaded and then execute it, too.
Note, since our compute nodes don't have internet access, I executed the command for you.
 # %load https://matplotlib.org/stable/_downloads/0c69e8950c767c2d95108979a24ace2f/surface3d_simple.py

"""
=====================
3D surface
=====================

See `~mpl_toolkits.mplot3d.axes3d.Axes3D.plot_surface`.
"""
import matplotlib.pyplot as plt
from matplotlib import cm
import numpy as np

# plt.style.use('_mpl-gallery')

# Make data
X = np.arange(-5, 5, 0.25)
Y = np.arange(-5, 5, 0.25)
X, Y = np.meshgrid(X, Y)
R = np.sqrt(X**2 + Y**2)
Z = np.sin(R)

# Plot the surface
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
ax.plot_surface(X, Y, Z, vmin=Z.min() * 2, cmap=cm.Blues)

ax.set(xticklabels=[],
       yticklabels=[],
       zticklabels=[])

plt.show()
 
 
 
 Moving Around
 IPython has two ways of moving around in the directory tree: %cd and %pushd/%popd. Both retain their history. %cd's history is available through %dhist whereas %dirs shows the directory stack of %pushd/%popd. The %cd  command has some nifty options, for example, %cd -2 gets you to the second to last visited directory. You can also set %bookmarks and use them with %cd.
Make a new sub directory called scripts/mandelbrot using %mkdir -p scripts/mandelbrot. Change into the directory scripts/mandelbrot using %cd. Go one level up using %cd ... Look at the history using %dhist. Finally use %cd -0 to get back to where you started from.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 A Little Bit of History
 Every output is stored to the Out array and can be accessed through its index. It can even be assigned.
 numpy.random.random([4,4])
 
 a = _ # Assigns the output of the previous cell
 
 a = Out[13] # Assign Out[?] to a (replace with index from two cells above)
a
 
 IPython keeps your history. It's accessed through the %history magic. By default it prints the history from the current session. If you have edited a cell multiple times, %history gives you all the versions. You can add line numbers to the output with the -n option, give ranges of lines that you want to see, and even perform some pattern matching. Pattern matching is done for your complete IPython history, not just the current session. For details see %history?.
If you want to log your work to a file (possibly with time stamps.) you can use IPython's logging facilities. %logstart -t testlog rotate creates a rotating log with timestamps and starts logging. %logoff pauses your log and %logstop stops the logging and closes the file.
 Some Introspection
 After an extended IPython session you might have declared many variables and functions. IPython provides some commands to help you to keep track of those. The first two are %who and %whos. %who lists the names of functions and variables in the default namespace. %whos provides more details and includes the type of the variable. Give it a try.
 %who
 
 %whos
 
 from math import pi
 
 %who
 
 If the namespace gets too crowded you can search for a particular object using %psearch. Remember, though, auto completion is your friend.
Sometimes it may become necessary to reset the namespace, for example, if Out[] contains some large objects that are no longer needed. %reset has several options to reset variables including the history of directories, inputs, and output. Please, check %reset? for details.
 Entering special symbols
  $IPython allows us to enter special symbols. To type , for example, you start by typing \\alpha followed by the Tab key.$ 
 \alpha
 
 IPython has several other interesting features. To learn more check out IPython's documentation.