update

Create_JupyterKernel.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# - Create your own Jupyter Kernel -"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "## Building your own Jupyter kernel is a three step process\n",
+    "1. Create/Pimp new virtual Python environment\n",
+    "   * venv\n",
+    "2. Create/Edit launch script for the Jupyter kernel\n",
+    "   * kernel.sh\n",
+    "3. Create/Edit Jupyter kernel configuration\n",
+    "   * kernel.json"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Settings"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* Set new kernel name\n",
+    "  - change if you like"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "export KERNEL_NAME=${USER}_kernel\n",
+    "echo ${KERNEL_NAME}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* Set directory for kernels virtual environment\n",
+    "  - change if you like"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "export KERNEL_VENVS_DIR=${PROJECT}/${USER}/jupyter/kernels/\n",
+    "mkdir -p ${KERNEL_VENVS_DIR}\n",
+    "echo ${KERNEL_VENVS_DIR}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* Set location of kernel spec\n",
+    "  - select one:\n",
+    "    - personal kernel = \"\\${HOME}/.local/\"  \n",
+    "    - project kernel  = \"\\${PROJECT}/.local/\"  "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "export KERNEL_SPECS_DIR=${HOME}/.local/\n",
+    "#export KERNEL_SPECS_DIR=${PROJECT}/.local/\n",
+    "echo ${KERNEL_SPECS_DIR}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Create/Pimp new virual Python environment"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* 1.1 - Load required modules"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "module -q purge\n",
+    "module -q use $OTHERSTAGES        \n",
+    "module -q load Stages/Devel-2019a 2> /dev/null\n",
+    "module -q load GCCcore/.8.3.0     2> /dev/null\n",
+    "module -q load Jupyter"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* 1.2 - Load extra modules you need for your kernel"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# module load <module you need>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* 1.3 - Create and activate a virtual environment for the kernel  \n",
+    "and ensure python packages installed in the virtual environment are always prefered"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "python -m venv --system-site-packages ${KERNEL_VENVS_DIR}/${KERNEL_NAME}\n",
+    "source ${KERNEL_VENVS_DIR}/${KERNEL_NAME}/bin/activate\n",
+    "export PYTHONPATH=${VIRTUAL_ENV}/lib/python3.6/site-packages:${PYTHONPATH}\n",
+    "echo ${VIRTUAL_ENV}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* 1.4 - Install Python libraries required for communication with Jupyter"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "pip install --ignore-installed ipykernel"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* 1.5 - Install whatever else you need in your Python virtual environment (using pip)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#pip install <python-package you need>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Create/Edit launch script for the Jupyter kernel"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* 2.1 - Create launch script, which loads your Python virtual environment and starts the ipykernel process inside:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "echo '#!/bin/bash\n",
+    "\n",
+    "# Load required modules\n",
+    "module purge\n",
+    "module load $OTHERSTAGES'\"\n",
+    "module load Stages/Devel-2019a\n",
+    "module load GCCcore/.8.3.0\n",
+    "module load Jupyter\n",
+    "\n",
+    "# Load extra modules you need for your kernel (as you did in step 1.2)\n",
+    "#module load <module you need>\n",
+    "    \n",
+    "# Activate your Python virtual environment\n",
+    "source ${KERNEL_VENVS_DIR}/${KERNEL_NAME}/bin/activate\n",
+    "    \n",
+    "# Ensure python packages installed in the virtual environment are always prefered\n",
+    "export PYTHONPATH=${VIRTUAL_ENV}/lib/python3.6/site-packages:\"'${PYTHONPATH}'\"\n",
+    "    \n",
+    "exec python -m ipykernel \"'$@' > ${VIRTUAL_ENV}/kernel.sh\n",
+    "chmod +x ${VIRTUAL_ENV}/kernel.sh"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. Create/Edit Jupyter kernel configuration"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* 3.1 - Create Jupyter kernel configuration directory and files"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "python -m ipykernel install --name=${KERNEL_NAME} --prefix ${KERNEL_SPECS_DIR}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "* 3.2 - Adjust kernel.json file"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "mv ${KERNEL_SPECS_DIR}/share/jupyter/kernels/${KERNEL_NAME}/kernel.json ${KERNEL_SPECS_DIR}/share/jupyter/kernels/${KERNEL_NAME}/kernel.json.orig\n",
+    "\n",
+    "echo '\n",
+    "{\n",
+    "  \"argv\": [\n",
+    "    \"'${KERNEL_VENVS_DIR}/${KERNEL_NAME}/kernel.sh'\",\n",
+    "    \"-m\",\n",
+    "    \"ipykernel_launcher\",\n",
+    "    \"-f\",\n",
+    "    \"{connection_file}\"\n",
+    "  ],\n",
+    "  \"display_name\": \"'${KERNEL_NAME}'\",\n",
+    "  \"language\": \"python\"\n",
+    "}' > ${KERNEL_SPECS_DIR}/share/jupyter/kernels/${KERNEL_NAME}/kernel.json"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Bash",
+   "language": "bash",
+   "name": "bash"
+  },
+  "language_info": {
+   "codemirror_mode": "shell",
+   "file_extension": ".sh",
+   "mimetype": "text/x-sh",
+   "name": "bash"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}