Skip to content
Snippets Groups Projects
Select Git revision
  • ae6b6f49ef5dc3a19e9d64a35836b7cdf3676c4a
  • master default protected
  • enxhi_issue460_remove_TOAR-I_access
  • michael_issue459_preprocess_german_stations
  • sh_pollutants
  • develop protected
  • release_v2.4.0
  • michael_issue450_feat_load-ifs-data
  • lukas_issue457_feat_set-config-paths-as-parameter
  • lukas_issue454_feat_use-toar-statistics-api-v2
  • lukas_issue453_refac_advanced-retry-strategy
  • lukas_issue452_bug_update-proj-version
  • lukas_issue449_refac_load-era5-data-from-toar-db
  • lukas_issue451_feat_robust-apriori-estimate-for-short-timeseries
  • lukas_issue448_feat_load-model-from-path
  • lukas_issue447_feat_store-and-load-local-clim-apriori-data
  • lukas_issue445_feat_data-insight-plot-monthly-distribution
  • lukas_issue442_feat_bias-free-evaluation
  • lukas_issue444_feat_choose-interp-method-cams
  • 414-include-crps-analysis-and-other-ens-verif-methods-or-plots
  • lukas_issue384_feat_aqw-data-handler
  • v2.4.0 protected
  • v2.3.0 protected
  • v2.2.0 protected
  • v2.1.0 protected
  • Kleinert_etal_2022_initial_submission
  • v2.0.0 protected
  • v1.5.0 protected
  • v1.4.0 protected
  • v1.3.0 protected
  • v1.2.1 protected
  • v1.2.0 protected
  • v1.1.0 protected
  • IntelliO3-ts-v1.0_R1-submit
  • v1.0.0 protected
  • v0.12.2 protected
  • v0.12.1 protected
  • v0.12.0 protected
  • v0.11.0 protected
  • v0.10.0 protected
  • IntelliO3-ts-v1.0_initial-submit
41 results

pre_processing.py

Blame
  • 3-JupyterLab-magics.ipynb 24.50 KiB

    JupyterLab Tour

    This is the first time you are using JupyterLab? Let us have a look at the user interface and some general concepts.


    What are Magics?

    Build-in commands of the running kernel.

    • Line magics, prepended by %, are single-line-commands
    • Cell magics, prepended by %%, receive as argument both the current line where they are declared and the whole body of the cell.

    Whether Magics are available on a kernel is a decision that is made by the kernel developer on a per-kernel basis. Magics specific to and provided by the IPython kernel: https://ipython.readthedocs.io/en/stable/interactive/magics.html


    Examples of Line Magics

    Exercise 1:

    List currently available magic functions - %lsmagic

    In [1]:
    %lsmagic
    Out [1]:
    Available line magics:
    %alias  %alias_magic  %autoawait  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %colors  %conda  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %man  %matplotlib  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %pip  %popd  %pprint  %precision  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode
    
    Available cell magics:
    %%!  %%HTML  %%SVG  %%bash  %%capture  %%debug  %%file  %%html  %%javascript  %%js  %%latex  %%markdown  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile
    
    Automagic is ON, % prefix IS NOT needed for line magics.

    Exercise:

    Timing the execution of code - %timeit

    In [2]:
    import numpy as np
    %timeit np.linalg.eigvals(np.random.rand(100,100))
    Out [2]:
    9.25 ms ± 4.29 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
    

    Exercise 2:

    Set up matplotlib to work interactively - %matplotlib

    • List matplotlib backends - %matplotlib -l
    • Enable the inline backend for usage with the IPython Notebook - %matplotlib inline
    In [3]:
    %matplotlib -l
    Out [3]:
    Available matplotlib backends: ['tk', 'gtk', 'gtk3', 'wx', 'qt4', 'qt5', 'qt', 'osx', 'nbagg', 'notebook', 'agg', 'svg', 'pdf', 'ps', 'inline', 'ipympl', 'widget']
    
    In [4]:
    %matplotlib inline

    Exercise 3:

    Automatic reload Python packages

    • %load_ext - Load an IPython extension by its module name.
      • autoreload - more
        • %load_ext autoload 2 - Reload all modules (except those excluded by %aimport) every time before executing the Python code typed.
    • %xmode - Switch modes for the exception handlers.

    Exercise 4:

    Tracebacks can sometimes get very longish in Python without much use.
    Switch modes for the exception handlers - %xmode -more

    • Valid modes: Plain, Context, Verbose, and Minimal.

    Examples of Cell Magics

    Exercise 5:

    Run cells with bash in a subprocess - %%bash

    • shortcut for %%script bash
    • !! attention: one bash subprocess per cell !!
    In [5]:
    %%bash
    cd $HOME
    ls -lt
    Out [5]:
    total 92084
    drwxr-sr-x  8 jovyan users     4096 May 25 11:48 jupyterlab-demo
    -rw-r--r--  1 jovyan users    28366 May 25 11:37 filename.png
    drwxr-sr-x  6 jovyan users     4096 May 25 11:20 CoE-2021.05-jupyter4hpc
    drwxr-sr-x  8 jovyan users     4096 May 25 11:20 prace-2021.04
    drwxrwsr-x  7 jovyan users     4096 Apr 22 06:23 trash
    drwxrwsr-x 16 jovyan users     4096 Apr 21 19:02 miniconda3
    lrwxrwxrwx  1 jovyan users       24 Apr 16 12:43 JUST_HOMEs_readonly -> /mnt/JUST_HOMEs_readonly
    drwxrwsr-x  3 jovyan users     4096 Apr 16 12:43 previous_files
    -rw-rw-r--  1 jovyan users 94235922 Nov 23  2020 Miniconda3.sh
    

    Exercise 6:

    Write the contents of the cell to a file - %%writefile

    In [11]:
    %%writefile test.out
    This is text in the file test.out
    Out [11]:
    Writing test.out
    

    Exercise 7:

    capture stdout/err from subprocesses into Python variables

    In [6]:
    %%bash --out output --err error
    echo "hi, stdout"
    echo "hello, stderr" >&2
    In [7]:
    print(error)
    print(output)
    Out [7]:
    hello, stderr
    
    hi, stdout