Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found
Select Git revision
  • documentation
  • integration
  • master
  • pre_update
4 results

Target

Select target project
  • jupyter4jsc/j4j_notebooks
  • kreuzer1/j4j_notebooks
  • goebbert1/j4j_notebooks
3 results
Select Git revision
  • integration
  • master
  • pre_update
3 results
Show changes

Commits on Source 15

Expand all Show whitespace changes
Inline Side-by-side
Showing
with 5665 additions and 91 deletions
01-Introduction/01-JSC.ipynb
View file @ 227ad4ab
%% Cell type:markdown id:ca8705ef-351e-411b-b988-6658cc06a450 tags:
![header.png](attachment:f66d472c-49c1-47d2-8d28-99749aa5c770.png)
<h5 style="text-align: right">Author: <a href="mailto:t.kreuzer@fz-juelich.de?subject=Jupyter-JSC%20documentation">Tim Kreuzer</a></h5>
<h5><a href="../index.ipynb">Index</a></h5>
<h1 style="text-align: center">JSC</h1>
%% Cell type:markdown id:d510e4f8-4eaf-4261-a15e-358a6646a2ea tags:
# Overview
%% Cell type:markdown id:9e0f3e32-6fe8-424c-80db-b2fbf8f90721 tags:
- [About JSC](https://fz-juelich.de/ias/jsc/EN/AboutUs/Profile/profile.html)
- [User Portal JuDoor](https://fz-juelich.de/ias/jsc/EN/Expertise/Supercomputers/NewUsageModel/JuDoor.html)
- [Usage Model](https://fz-juelich.de/ias/jsc/EN/Expertise/Supercomputers/NewUsageModel/NewUsageModel_node.html)
%% Cell type:markdown id:7a7af86c-eeab-42f0-a4bc-7f247061882d tags:
# Systems
%% Cell type:markdown id:fa1151a4-609d-40b0-9e30-f2211012a535 tags:
## HPC-Systems
- [JUWELS](https://fz-juelich.de/ias/jsc/EN/Expertise/Supercomputers/JUWELS/JUWELS_node.html)
- [JUWELS](https://www.fz-juelich.de/en/ias/jsc/systems/supercomputers/juwels)
- [User documentation](https://apps.fz-juelich.de/jsc/hps/juwels/index.html)
- [JURECA](https://fz-juelich.de/ias/jsc/EN/Expertise/Supercomputers/JURECA/JURECA_node.html)
- [JURECA](https://www.fz-juelich.de/en/ias/jsc/systems/supercomputers/jureca)
- [User documentation](https://apps.fz-juelich.de/jsc/hps/jureca/index.html)
- [JUSUF](https://fz-juelich.de/ias/jsc/EN/Expertise/Supercomputers/JUSUF/JUSUF_node.html)
- [JUSUF](https://www.fz-juelich.de/en/ias/jsc/systems/supercomputers/jusuf)
- [User documentation](https://apps.fz-juelich.de/jsc/hps/jusuf/index.html)
- [DEEP](https://fz-juelich.de/ias/jsc/EN/Expertise/Supercomputers/DEEP-EST/_node.html)
- [Project](https://www.deep-projects.eu/)
- [User documentation](https://deeptrac.zam.kfa-juelich.de:8443/trac/wiki/Public/User_Guide)
- HDFML
## Cloud-Systems
- [HDF-Cloud](https://www.fz-juelich.de/ias/jsc/EN/Expertise/SciCloudServices/HDFCloud/_node.html)
- [Jupyter-JSC configuration on HDF-Cloud](https://docs.jupyter-jsc.fz-juelich.de/github/kreuzert/jupyter-jsc-notebooks/blob/documentation/02-Configuration/details/HDFCloud.ipynb)
- [HDF-Cloud](https://www.fz-juelich.de/en/ias/jsc/systems/scientific-clouds/hdf-cloud)
- [Jupyter-JSC configuration on HDF-Cloud](https://docs.jupyter-jsc.fz-juelich.de/github/FZJ-JSC/jupyter-jsc-notebooks/blob/documentation/02-Configuration/details/HDFCloud.ipynb)
%% Cell type:markdown id:6c71b6b4-9c25-49a1-97d1-0639dcddf0c4 tags:
# File-Systems
%% Cell type:markdown id:b6006d25-132c-448c-b8b5-b35d4fdcc28d tags:
- [HPC-Systems](https://apps.fz-juelich.de/jsc/hps/just/filesystems.html)
- Cloud-Systems: Hosted centralized as NFS server. Each service will mount specific user directory.
......
01-Introduction/02-Jupyter-JSC.ipynb
View file @ 227ad4ab
%% Cell type:markdown id:ca8705ef-351e-411b-b988-6658cc06a450 tags:
![header.png](attachment:f66d472c-49c1-47d2-8d28-99749aa5c770.png)
<h5 style="text-align: right">Author: <a href="mailto:t.kreuzer@fz-juelich.de?subject=Jupyter-JSC%20documentation">Tim Kreuzer</a></h5>
<h5><a href="../index.ipynb">Index</a></h5>
<h1 style="text-align: center">Jupyter-JSC</h1>
%% Cell type:markdown id:d510e4f8-4eaf-4261-a15e-358a6646a2ea tags:
# About
%% Cell type:markdown id:469fe0ed-79f3-412e-b1e9-c1dbf7f8ae99 tags:
Jupyter-JSC is designed to provide the rich high performance computing (HPC) ecosystem to the world's most popular software: web browsers. JupyterLab is a web-based interactive development environment for Jupyter notebooks, code, and data. JupyterLab is flexible to support a wide range of workflows in data science, scientific computing, and machine learning.
%% Cell type:markdown id:46238f4c-42ca-4dcc-b6cc-b0ae5ae8b61c tags:
# Setup
%% Cell type:markdown id:79c71dd6-9279-44e7-86fb-307a511dad74 tags:
## Webservice
Jupyter-JSC is a customized [JupyterHub](https://jupyter.org/hub) running on the [HDF-Cloud](https://www.fz-juelich.de/ias/jsc/EN/Expertise/SciCloudServices/HDFCloud/_node.html). It is deployed in a kubernetes cluster.
Jupyter-JSC is a customized [JupyterHub](https://jupyter.org/hub) running on the [HDF-Cloud](https://www.fz-juelich.de/en/ias/jsc/systems/scientific-clouds/hdf-cloud). It is deployed in a kubernetes cluster.
There are four basic components to serve Jupyter-JSC:
- JuypterHub
- Interacts with the user. Handles authentication and communication between JupyterHub and the user's services.
- Backend
- Starting / stopping services for the user on any system. Called by JupyterHub [Spawner](https://jupyterhub.readthedocs.io/en/stable/reference/spawners.html). Communicating with [UNICORE](https://unicore.eu) and UserLab Manager.
- Tunneling
- Secure port forwarding between JupyterHub and the user's service. Required to reach compute nodes without access to the internet.
- UserLab Manager
- Starting / stopping services on the cloud systems, where [UNICORE](https://unicore.eu) is not an option.
%% Cell type:markdown id:19ccfc9d-c95d-4f60-887e-2fba9c63f8ac tags:
## Authentication
If you visit Jupyter-JSC and hit the Login button, you will be redirected to our authentication service. This webservice, called [Unity-IdM](https://www.unity-idm.eu), is connected to the JSC WebLDAP and the [Helmholtz AAI](https://aai.helmholtz.de/). You can use your JSC account, also used at [JuDoor](https://judoor.fz-juelich.de/), [GitLab](https://gitlab.jsc.fz-juelich.de/) and more, or the Helmholtz AAI, to sign in with your local identity provider (IdP). After your successful login, you will be redirected to Jupyter-JSC.
![jupyter-jsc-authentication.png](attachment:f0958653-c22c-436d-88b3-2afd3f02d4eb.png)
### [2-Factor-Authorization](https://docs.jupyter-jsc.fz-juelich.de/github/kreuzert/jupyter-jsc-notebooks/blob/documentation/02-Configuration/2-Factor-Authentication.ipynb)
### [2-Factor-Authorization](https://docs.jupyter-jsc.fz-juelich.de/github/FZJ-JSC/jupyter-jsc-notebooks/blob/documentation/02-Configuration/2-Factor-Authentication.ipynb)
%% Cell type:markdown id:db1e5246-f784-48fe-ab50-696f7b56ab2f tags:
## Authorization
### System access
Jupyter-JSC gets the information about the users HPC accounts and projects directly from JuDoor.
While the cloud systems are available for everyone, the resources and quotas may differ for each virtual organization. Both are also dependent on the use of the system and may change. You can see your current quota while your Service is starting.
### Virtual organizations
Jupyter-JSC supports multiple virtual organizations (VO) in Jupyter-JSC. This allows us to offer specific resources and quotas for different communities or workshops. If you have multiple VOs you can choose your active VO in the top right corner of the website.
%% Cell type:markdown id:4be179c9-0210-4788-9128-2fd2f5a2c9fc tags:
## Service Start
Jupyter-JSC uses [UNICORE](https://unicore.eu) to start jobs on the HPC-Systems. These jobs contain all the information to start your service.
On Cloud-Systems, where [UNICORE](https://unicore.eu) is not available, Jupyter-JSC uses an internal solution (UserLab Manager) to start a [Kubernetes Pod](https://kubernetes.io/de/docs/concepts/workloads/pods/) for each of the users services.
During the startup process of the users service, Jupyter-JSC creates a secure ssh-connection between JupyterHub and the users service.
%% Cell type:markdown id:eaa4d91e-6161-446f-96c7-4daebf9f796a tags:
![jupyter-jsc-start-01.png](attachment:697396c6-a739-4884-bdc7-e4877e196bee.png)
......
01-Introduction/03-Login-to-Jupyter-JSC.ipynb 0 → 100644
View file @ 227ad4ab
This diff is collapsed.
02-Configuration/03-Environment.ipynb
View file @ 227ad4ab
%% Cell type:markdown id:ca8705ef-351e-411b-b988-6658cc06a450 tags:
![header.png](attachment:f66d472c-49c1-47d2-8d28-99749aa5c770.png)
<h5 style="text-align: right">Author: <a href="mailto:j.goebbert@fz-juelich.de?subject=Jupyter-JSC%20documentation">Jens Henrik Göbbert</a></h5>
<h5><a href="../index.ipynb">Index</a></h5>
<h1 style="text-align: center">Environment</h1>
%% Cell type:markdown id:da15f1a1-b8e9-449e-a17c-13006f5492a6 tags:
# Modules
%% Cell type:markdown id:ba0376e8-5268-4c5c-98ed-62065def57df tags:
%% Cell type:markdown id:08a89c95-d669-4b51-ae2c-b085612fb73a tags:
- [List of installed python packages](https://docs.jupyter-jsc.fz-juelich.de/github/kreuzert/jupyter-jsc-notebooks/blob/documentation/02-Configuration/details/List_PythonPackages.ipynb)
- [Howto load additional software modules](https://docs.jupyter-jsc.fz-juelich.de/github/kreuzert/jupyter-jsc-notebooks/blob/documentation/03-HowTos/Howto-load-additional-software-modules.ipynb)
- [List of installed python packages](https://docs.jupyter-jsc.fz-juelich.de/github/FZJ-JSC/jupyter-jsc-notebooks/blob/documentation/02-Configuration/details/List_PythonPackages.ipynb)
- [Howto load additional software modules](https://docs.jupyter-jsc.fz-juelich.de/github/FZJ-JSC/jupyter-jsc-notebooks/blob/documentation/03-HowTos/Howto-load-additional-software-modules.ipynb)
......
02-Configuration/2-Factor-Authentication.ipynb
View file @ 227ad4ab
%% Cell type:markdown id: tags:
![header.png](attachment:2087d183-9538-4c34-a9f7-5f6a5428c6ee.png)
<h5 style="text-align: right">Author: <a href="mailto:j.goebbert@fz-juelich.de?subject=Jupyter-JSC%20documentation">Jens Henrik Göbbert</a></h5>
<h5><a href="../index.ipynb">Index</a></h5>
<h1 style="text-align: center">2-Factor Authentication (2FA)</h1>
%% Cell type:markdown id: tags:
<div>
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/default/images/2fa/jupyter-jsc_2fa_img01.png title="2-factor-authentication" width="320" style="float:left"/>
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/images/2fa/jupyter-jsc_2fa_img01.png title="2-factor-authentication" width="320" style="float:left"/>
<!-- <img src=images/jupyter-jsc_2fa_img01.png title="2-factor-authentication" width="320" style="float:left"/> -->
</div>
## Introduction
2-Factor Authentication (2FA), sometimes referred to as two-factor verification, is a security method in which you provide **two different authentication factors** to identify yourself at login.
This process is **performed to better protect** both your credentials and the resources that you can access.
In the **first login step**, you start with the usual entry of a good password. The service then confirms the correctness of the password entered.
This does not, however, lead directly to the desired entrance - but to a further barrier.
The **second login step** prevents unauthorized third parties from gaining access to your account just because they might have stolen your password.
A quite common 2nd-factor is a **One-Time Password (OTP)** generated by a so-called **OTP-App** you install and initialize once on one of your personal devices.
This *OTP-app* then provides (in our case every 30 seconds) a new *one-time password* that needs to be entered on the login page.
<div style="clear:both"></div>
%% Cell type:markdown id: tags:
<div>
<video controls src="https://multimedia.gsb.bund.de/BSI/Video/2-Faktor-Authentisierung_SD.conv.mp4" width=480 style="float:right"/>
</div>
## Basic Principle
These two factors for authentication combine the building blocks **knowledge** and **possession** in the login procedure.
- **knowledge** - the secret knowledge is the password you enter.
- **possession** - With the *one-time password* you show that you are in possession of a certain device (e.g. your smartphone), because only the *OTP-App*, installed on that device, can generate it.
<div style="clear:both"></div>
<div>
<p style="float:right">Source: Bundesamt für Sicherheit in der Informationstechnik</p>
</div>
%% Cell type:markdown id: tags:
<div>
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/default/images/2fa/jupyter-jsc_2fa_img02.png title="2-factor-authentication" width="320" style="float:left"/>
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/images/2fa/jupyter-jsc_2fa_img02.png title="2-factor-authentication" width="320" style="float:left"/>
<!-- <img src=images/jupyter-jsc_2fa_img02.png title="2-factor-authentication" width="320" style="float:left"/> -->
</div>
## Algorithm
The **OTP-App** can calculate personal one-time passwords completely autonomously from the outside world using a standardized and open algorithm for the generation of **Time-based One-Time Passwords (TOTP)**.
The *TOTP algorithm* was published in 2011 by the [Internet Engineering Task Force (IETF)](https://www.ietf.com) as [RFC 6238](https://tools.ietf.org/html/rfc6238). The *TOTP algorithm* is a hash function in which a secret code is hashed together with the current time.
Behind the hash function is the HMAC-based One-time Password Algorithm according to [RFC 4226](https://tools.ietf.org/html/rfc4226) - in simple terms nothing more than a standard that forms a hash in a certain way.
The calculation includes both a **"secret initialization code"**, that is known to both the server and the client, and the **current time**.
The final *one-time password* is generated from these two inputs and is valid for a certain period of time. (in our case for **30 seconds**).
The procedure can be implemented in such a way that slight differences in time between client and server are accepted.
Hence, any *one-time password* is time-based, calculated locally, and always unique.
<div style="clear:both"></div>
------------------
%% Cell type:markdown id: tags:
# How to get started with 2FA
<div>
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/default/images/2fa/jupyter-jsc_2fa_img03.png title="2-factor-authentication" width="320" style="float:right"/>
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/images/2fa/jupyter-jsc_2fa_img03.png title="2-factor-authentication" width="320" style="float:right"/>
<!-- <img src=images/jupyter-jsc_2fa_img03.png title="2-factor-authentication" width="320" style="float:right"/> -->
</div>
## Preparation
To get ready to use 2-Factor Authentication (2FA) for Jupyter-JSC you have to **prepare** it ONCE:
- (1) **request 2FA** for Jupyter-JSC,
- (a) login to [Jupyter-JSC](https://jupyter-jsc.fz-juelich.de)
- (b) visit https://jupyter-jsc.fz-juelich.de/2fa and request 2FA
- (c) wait for a *confirmation emails* and click the provided *activation link*
- (2) **activate 2FA** for Juypter-JSC,
- (a) install an **OTP-App**, which supports the TOTP algorithm
- (b) communicate the **secret initialization code** to this *OTP-App*
- (c) test a first **one-time password** generated.
... and then 2FA is ready to be used next time you log in.
### 1. Request 2FA
Please login to Jupyter-JSC as usual through https://jupyter-jsc.fz-juelich.de
and visit the webpage **https://jupyter-jsc.fz-juelich.de/2fa** for requesting 2FA.
Please read the notes on this webpage carefully and click the button **Request 2FA** to start.
A **confirmation email** including an **activation link** will be send to you directly.
### 2. Activate 2FA
Please follow this *activation link* to instruct Jupyter-JSC for preparation of your 2FA.
You will be asked to re-login to your account to recieve a **secret initialization code** as QR-Code (and string)
for a required *OTP-App*.
So first, you need to install an **OTP-App** on one of your personal devices (if you haven´t done so already),
which you plan to use in the future to generate the required **one-time passwords** for each time you log in:
<div style="clear:both"></div>
%% Cell type:markdown id: tags:
<div>
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/default/images/2fa/jupyter-jsc_2fa_img04.png title="2-factor-authentication" width="320" style="float:left"/>
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/images/2fa/jupyter-jsc_2fa_img04.png title="2-factor-authentication" width="320" style="float:left"/>
<!-- <img src=images/jupyter-jsc_2fa_img04.png title="2-factor-authentication" width="320" style="float:left"/> -->
</div>
<div>
<!-- <img src=https://jupyter-jsc.fz-juelich.de/hub/static/default/images/2fa/jupyter-jsc_2fa_img04-1.png title="2-factor-authentication" width="320" style="float:right"/>-->
<img src=https://raw.githubusercontent.com/FZJ-JSC/jupyter-jsc-notebooks/master/001-Jupyter/images/jupyter-jsc_2fa_img04-1.png title="2-factor-authentication" width="120" style="float:right"/>
<!-- <img src=https://jupyter-jsc.fz-juelich.de/hub/static/images/2fa/jupyter-jsc_2fa_img04-1.png title="2-factor-authentication" width="320" style="float:right"/>-->
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/images/2fa/jupyter-jsc_2fa_img04-1.png title="2-factor-authentication" width="120" style="float:right"/>
</div>
### a. OTP-App Installation
There are a large number of different *OTP-Apps* available that implemented the *TOTP algorithm*.
You have to install **one of them** - for example, take one of the following:
Recommended, free & open-source:
- [**FreeOTP**](https://freeotp.github.io) ([iOS](https://apps.apple.com/de/app/freeotp-authenticator/id872559395), [Android](https://play.google.com/store/apps/details?id=org.fedorahosted.freeotp&hl=de))
- [**KeeWeb**](https://keeweb.info) ([Windows](https://keeweb.info), [macOS](https://keeweb.info), [Linux](https://keeweb.info), [online](https://keeweb.info))
Free, but closed source:
- [**Authy**](https://authy.com/download/) ([iOS](https://apps.apple.com/de/app/authy/id494168017), [Android](https://play.google.com/store/apps/details?id=com.authy.authy), [Windows](https://authy.com/download/), [macOS](https://authy.com/download/), [Linux](https://snapcraft.io/authy))
- [**Protectimus Smart OTP**](https://www.protectimus.com/protectimus-smart) ([iOS](https://apps.apple.com/ie/app/protectimus-smart/id854508919), [Android](https://play.google.com/store/apps/details?id=com.protectimus.android))
- [**Google Authenticator**](https://de.wikipedia.org/wiki/Google_Authenticator) ([iOS](https://apps.apple.com/de/app/google-authenticator/id388497605), [Android](https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2) )
- [**Microsoft Authenticator**](https://www.microsoft.com/en-us/account/authenticator) ([iOS](https://apps.apple.com/de/app/microsoft-authenticator/id983156458), [Android](https://play.google.com/store/apps/details?id=com.azure.authenticator), [Windows 10 Mobile](https://www.microsoft.com/en-us/p/microsoft-authenticator/9nblgggzmcj6))
The *TOTP algorithm* can also be implemented in hardware as a so-called "hardware token" (e.g. [Protectimus Tokens](https://www.protectimus.com/tokens/), [Microcosm Tokens](https://www.microcosm.com/products/oath-otp-authentication-tokens))
<div style="clear:both"></div>
%% Cell type:markdown id: tags:
<div>
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/default/images/2fa/jupyter-jsc_2fa_img05.png title="2-factor-authentication" width="320" style="float:left"/>
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/images/2fa/jupyter-jsc_2fa_img05.png title="2-factor-authentication" width="320" style="float:left"/>
<!-- <img src=images/jupyter-jsc_2fa_img05.png title="2-factor-authentication" width="320" style="float:left"/> -->
</div>
### b. OTP-App Initialization & Validation
Before you can use 2FA for Jupyter-JSC a random, user-specific, unique and **secret initialization code** must be known by both Jupyter-JSC and the your *OTP-App*.
This *secret initialization code* gets generated by Jupyter-JSC and is shown as a **QR-Code** (or string) on the activation page.
The QR-Code provides the *secret initialization code* with the descriptive data (1) algorithm = TOTP, (2) period of validity = 30s.
**If you prefer to use the string** instead of the QR-Code, please ensure you set these descriptive dates manually in your *OTP-App*.
Next, the *OTP-App* provides now a **verification code** you have to enter on the activation webpage.
Jupyter-JSC compares the *verification code* you provide with the one generated by Jupyter-JSC.
If they match, **2FA is now activated**.
<div style="clear:both"></div>
----------------------
%% Cell type:markdown id: tags:
<div>
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/default/images/2fa/jupyter-jsc_2fa_img06.png title="2-factor-authentication" width="320" style="float:right"/>
<img src=https://jupyter-jsc.fz-juelich.de/hub/static/images/2fa/jupyter-jsc_2fa_img06.png title="2-factor-authentication" width="320" style="float:right"/>
<!-- <img src=images/jupyter-jsc_2fa_img06.png title="2-factor-authentication" width="320" style="float:right"/> -->
</div>
### 2FA-Login at Jupyter-JSC
Congratulation! You are now ready to use 2-Factor Authentication with Jupyter-JSC.
Login is now as simple as this
1. **Enter your JSC-account password**
Each time you log in, you enter your JSC-account password as usual.
2. **Enter the current one-time password**
You will then be asked for a *one-time password* that you can read from your installed & initialized *OTP-App* (e.g. on your smartphone).
**Remember me**
Jupyter-JSC can set a cookie to remember, that you have logged in from this device already.
Just check the "Remember me" **checkbox** where you enter *one-time password* .
Jupyter-JSC **skips the request** of a *one-time password* in this browser on that device then for **one week**.
......
03-HowTos/Create_JupyterKernel_general.ipynb
View file @ 227ad4ab
%% Cell type:markdown id: tags:
![header.png](attachment:9f53dcb1-00d6-4245-955a-b527f1540865.png)
<h5 style="text-align: right">Author: <a href="mailto:j.goebbert@fz-juelich.de?subject=Jupyter-JSC%20documentation">Jens Henrik Göbbert</a></h5>
<h5><a href="../index.ipynb">Index</a></h5>
<h1 style="text-align: center">Create your own Jupyter Kernel</h1>
%% Cell type:markdown id: tags:
Often the standard kernel do not provide all features you need for your work. This might be that certain modules are not loaded or packages are not installed.
With your own kernel you can overcome that problem easily and define your own environment, in which you work.
This notebook shows you how you can build your own kernel for a **python environment**.
<div class="alert alert-block alert-info">
<b>Attention:</b>
This notebook is meant to run out of a JupyterLab on JSC's HPC systems.</br>
</div>
-------------------------
%% Cell type:markdown id: tags:
## Building your own Jupyter kernel is a three step process
1. Create/Pimp new virtual Python environment
* venv
2. Create/Edit launch script for the Jupyter kernel
* kernel.sh
3. Create/Edit Jupyter kernel configuration
* kernel.json
%% Cell type:markdown id: tags:
### Settings
%% Cell type:markdown id: tags:
#### Set the kernel name
- must be lower case
- change if you like
%% Cell type:code id: tags:
``` bash
# INPUT NEEDED:
KERNEL_NAME=${USER}_kernel
export KERNEL_NAME=$(echo "${KERNEL_NAME}" | awk '{print tolower($0)}')
echo ${KERNEL_NAME} # double check
```
%% Cell type:markdown id: tags:
#### Set the kernel directory
- check that the kernel name is unique
- print the location of the new kernel
%% Cell type:code id: tags:
``` bash
# define KERNEL_SPECS_DIR
export KERNEL_SPECS_PREFIX=${HOME}/.local
if [ ! -d "$KERNEL_SPECS_PREFIX" ]; then
echo "ERROR: please create directory $KERNEL_SPECS_PREFIX"
fi
export KERNEL_SPECS_DIR=${KERNEL_SPECS_PREFIX}/share/jupyter/kernels
# check if kernel name is unique
if [ -d "${KERNEL_SPECS_DIR}/${KERNEL_NAME}" ]; then
echo "ERROR: Kernel already exists in ${KERNEL_SPECS_DIR}/${KERNEL_NAME}"
echo " Rename kernel name or remove directory."
fi
# print the location of the new kernel
echo ${KERNEL_SPECS_DIR}/${KERNEL_NAME}
```
%% Cell type:markdown id: tags:
#### Set the kernel's virtual environment
- by default it is located at $PROJECT
- print the location of the new kernels virtual environment
%% Cell type:code id: tags:
``` bash
# define KERNEL_VENVS_DIR
export KERNEL_VENVS_DIR=${PROJECT}/${USER}/jupyter/kernels
mkdir -p ${KERNEL_VENVS_DIR}
# print the location of the new kernels virtual environment
echo ${KERNEL_VENVS_DIR}
```
%% Cell type:markdown id: tags:
---
%% Cell type:markdown id: tags:
## 1. Create/Pimp new virtual Python environment
%% Cell type:markdown id: tags:
#### 1.1 - Load basic Python module
%% Cell type:code id: tags:
``` bash
module -q purge
module -q load Stages/2023 # any stage can be used
module -q load GCC
module -q load Python # only Python is required
module purge
module load Stages/2025 # any stage can be used
module load GCC
module load Python # only Python is mandatory
module load jupyter-server # provides ipykernel
```
%% Cell type:code id: tags:
``` bash
# get Python version
export PYV=$(python -c 'import sys; print(".".join(map(str, sys.version_info[:2])))')
echo $PYV
```
%% Cell type:markdown id: tags:
#### 1.2 - Load extra modules you need for your kernel
%% Cell type:code id: tags:
``` bash
# module load <module you need>
```
%% Cell type:markdown id: tags:
#### 1.3 - Create and activate a virtual environment for the kernel
and ensure python packages installed in the virtual environment are always prefered
%% Cell type:code id: tags:
``` bash
export VIRTUAL_ENV=${KERNEL_VENVS_DIR}/${KERNEL_NAME}
if [ -d "${VIRTUAL_ENV}" ]; then
echo "ERROR: Directory for virtual environment already ${VIRTUAL_ENV}"
echo " Rename kernel name or remove directory."
else
python -m venv --system-site-packages ${VIRTUAL_ENV}
source ${VIRTUAL_ENV}/bin/activate
export PYTHONPATH=${VIRTUAL_ENV}/lib/python${PYV}/site-packages:${PYTHONPATH}
echo ${VIRTUAL_ENV} # double check
fi
```
%% Cell type:markdown id: tags:
#### 1.4 - Install Python libraries required for communication with Jupyter
#### 1.4 - Install whatever else you need in your Python virtual environment (using pip)
%% Cell type:code id: tags:
``` bash
which pip
if [ -z "${VIRTUAL_ENV}" ]; then
echo "ERROR: Virtual environment not successfully initialized."
else
pip install ipykernel
echo "Installing custom Python packages using pip from the virtual environment:"
which pip
# pip install <python-package you need>
fi
```
%% Cell type:markdown id: tags:
#### 1.5 - Install whatever else you need in your Python virtual environment (using pip)
%% Cell type:code id: tags:
``` bash
#pip install <python-package you need>
```
%% Cell type:markdown id: tags:
---
%% Cell type:markdown id: tags:
## 2. Create/Edit launch script for the Jupyter kernel
%% Cell type:markdown id: tags:
#### 2.1 - Create launch script, which loads your Python virtual environment and starts the ipykernel process inside:
<div class="alert alert-block alert-info">
<b>Attention:</b>
You MUST load the exactly the same modules as you did above for your virtual Python environment.
</div>
%% Cell type:code id: tags:
``` bash
echo '#!/bin/bash'"
# Load basic Python module
module purge
module load Stages/2023
module load Stages/2025
module load GCC
module load Python
module load jupyter-server # provides ipykernel
# Load extra modules you need for your kernel (as you did in step 1.2)
#module load <module you need>
# module load <module you need>
# Activate your Python virtual environment
source ${VIRTUAL_ENV}/bin/activate
# Ensure python packages installed in the virtual environment are always prefered
export PYTHONPATH=${VIRTUAL_ENV}/lib/python${PYV}/site-packages:"'${PYTHONPATH}'"
exec python -m ipykernel "'$@' > ${VIRTUAL_ENV}/kernel.sh
exec python -Xfrozen_modules=off -m ipykernel "'$@' > ${VIRTUAL_ENV}/kernel.sh
chmod +x ${VIRTUAL_ENV}/kernel.sh
cat ${VIRTUAL_ENV}/kernel.sh # double check
```
%% Cell type:markdown id: tags:
---
%% Cell type:markdown id: tags:
## 3. Create/Edit Jupyter kernel configuration
%% Cell type:markdown id: tags:
#### 3.1 - Create Jupyter kernel configuration directory and files
%% Cell type:code id: tags:
``` bash
python -m ipykernel install --name=${KERNEL_NAME} --prefix ${VIRTUAL_ENV}
export VIRTUAL_ENV_KERNELS=${VIRTUAL_ENV}/share/jupyter/kernels
```
%% Cell type:markdown id: tags:
#### 3.2 - Adjust kernel.json file
%% Cell type:code id: tags:
``` bash
mv ${VIRTUAL_ENV_KERNELS}/${KERNEL_NAME}/kernel.json ${VIRTUAL_ENV_KERNELS}/${KERNEL_NAME}/kernel.json.orig
echo '{
"argv": [
"'${KERNEL_VENVS_DIR}/${KERNEL_NAME}/kernel.sh'",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"display_name": "'${KERNEL_NAME}'",
"language": "python",
"metadata": {
"debugger": true
}
}' > ${VIRTUAL_ENV_KERNELS}/${KERNEL_NAME}/kernel.json
cat ${VIRTUAL_ENV_KERNELS}/${KERNEL_NAME}/kernel.json # double check
```
%% Cell type:markdown id: tags:
#### 3.3 - Create link to kernel specs
%% Cell type:code id: tags:
``` bash
mkdir -p ${KERNEL_SPECS_DIR}
cd ${KERNEL_SPECS_DIR}
ln -s ${VIRTUAL_ENV_KERNELS}/${KERNEL_NAME} .
echo -e "\n\nThe new kernel '${KERNEL_NAME}' was added to your kernels in '${KERNEL_SPECS_DIR}/'\n"
ls ${KERNEL_SPECS_DIR} # double check
```
%% Cell type:markdown id: tags:
#### 3.4 - Use the kernel
- You can select the new kernel in the top right corner of your notebook or from JupyterLab's Launchpad
- The kernel icon will be added to your launcher, after a while by JupyterLab automatically or once you've restarted the JupyterLab
%% Cell type:markdown id: tags:
---
%% Cell type:markdown id: tags:
## 4. Cleanup
%% Cell type:code id: tags:
``` bash
deactivate
```
......
03-HowTos/Create_JupyterKernel_conda.ipynb → 03-HowTos/Create_JupyterKernel_miniforge.ipynb
View file @ 227ad4ab
%% Cell type:markdown id: tags:
%% Cell type:markdown id:ccca6570-1a7d-44ea-a7b5-892d99f7e9c5 tags:
![header.png](attachment:364f4d26-8fd6-45ed-a6c3-1ab27dad25c4.png)
<h5 style="text-align: right">Author: <a href="mailto:s.luehrs@fz-juelich.de?subject=Jupyter-JSC%20documentation">Sebastian Lührs</a></h5>
<h5 style="text-align: right">Author: <a href="mailto:s.luehrs@fz-juelich.de?subject=Jupyter-JSC%20documentation">Sebastian Lührs, Jens Henrik Göbbert</a></h5>
<h5><a href="../index.ipynb">Index</a></h5>
<h1 style="text-align: center">Create your own Jupyter CONDA-Kernel</h1>
%% Cell type:markdown id: tags:
%% Cell type:markdown id:d347ba8a-5d72-4fb9-a24c-0d119299e0c2 tags:
Often the standard kernel do not provide all features you need for your work. This might be that certain modules are not loaded or packages are not installed.
With your own kernel you can overcome that problem easily and define your own environment, in which you work.
This notebook shows you how you can build your own kernel for a **conda environment**.
--------------------------------------
%% Cell type:markdown id: tags:
%% Cell type:markdown id:5e365cf9-53a7-42c3-ac03-710131b25e13 tags:
## Building your own Jupyter CONDA-kernel is a three step process
Download Minconda installer
1. Download/Install Miniconda
* Miniconda3.sh
1. Download/Install Miniforge
* Miniforge3.sh
2. Create Conda Environment
* conda create
2. Create/Edit launch script for the Jupyter kernel
* kernel.sh
3. Create/Edit Jupyter kernel configuration
* kernel.json
%% Cell type:markdown id: tags:
%% Cell type:markdown id:999e5a83-6c38-4b0f-90a0-9556c003b084 tags:
### Settings
%% Cell type:markdown id: tags:
%% Cell type:markdown id:1b97ea53-0394-42b3-8c6e-528e8cdc3bdf tags:
Selectable **CONDA_ENV** name, will be used to specify the environment name
- must be lowercase
%% Cell type:code id: tags:
%% Cell type:code id:b67a421a-90ae-4eb0-b2ea-6d81c22858f7 tags:
``` bash
CONDA_ENV=my_condaenv
export CONDA_ENV=$(echo "${CONDA_ENV}" | awk '{print tolower($0)}')
echo ${CONDA_ENV} # double check
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id:4e3c03ed-958e-45ce-81c3-2bb286a23f00 tags:
Selectable **CONDA_TARGET_DIR** path for the central conda installation, should be in the project filesystem
%% Cell type:code id: tags:
%% Cell type:code id:c7c016fe-3073-4513-a74d-2a0c9bb75962 tags:
``` bash
export CONDA_TARGET_DIR=${PROJECT}/${USER}/miniconda3/${CONDA_ENV}
export CONDA_TARGET_DIR=${PROJECT}/${USER}/miniforge3/${CONDA_ENV}
echo ${CONDA_TARGET_DIR} # double check
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id:849cc1bc-fd95-4829-80c5-aebaf2ce1635 tags:
---
## 1. Download/Install Miniconda
## 1. Download/Install Miniforge
%% Cell type:markdown id: tags:
%% Cell type:markdown id:e22949b1-e4f9-40de-b58d-b6a5165d5225 tags:
Start here if you want to run the full installation.
If you want to create another environment in an existing conda setup go to **create environment**. If you want to attach yourself to an existing environment go to **create user kernel**.
%% Cell type:markdown id: tags:
%% Cell type:markdown id:599cf30a-9f94-4d0e-a1f0-a4a8e9e3053a tags:
* 1.1 - Download Minconda installer
* 1.1 - Download Miniforge installer
%% Cell type:code id: tags:
%% Cell type:code id:c83c19db-eece-45ab-8a89-943941c6123f tags:
``` bash
wget --output-document=$HOME/Miniconda3.sh https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
wget --output-document=$HOME/Miniforge3.sh https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id:b16ff496-9214-4089-8cb1-b9f3d5bb24dd tags:
* 1.2 - Create target directory
%% Cell type:code id: tags:
%% Cell type:code id:c4d47aa3-3894-4110-b18a-538125f45008 tags:
``` bash
mkdir -p ${CONDA_TARGET_DIR}
echo ${CONDA_TARGET_DIR}
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id:18ad5818-93ca-4e1c-b7ab-5146b906ceb1 tags:
* 1.3 - Install Miniconda
* 1.3 - Install Miniforge
%% Cell type:code id: tags:
%% Cell type:code id:1dbfb9bc-82cf-44e6-9727-7ee4c82e3302 tags:
``` bash
bash $HOME/Miniconda3.sh -b -u -p ${CONDA_TARGET_DIR}
unset PYTHONPATH
bash $HOME/Miniforge3.sh -b -s -u -p ${CONDA_TARGET_DIR}
```
%% Cell type:code id: tags:
``` bash
${CONDA_TARGET_DIR}/bin/conda init --no-rc bash
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id:c37ee302-e75c-4cff-9694-ee4921870f7d tags:
* 1.4 - Disable automatic activation
Create `~/.condarc` and add the configuration settings.
%% Cell type:code id: tags:
%% Cell type:code id:fa14ffbe-e04b-4bea-a3d5-b78cc415f059 tags:
``` bash
${CONDA_TARGET_DIR}/bin/conda config --set auto_activate_base false
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id:ae8161d0-ddde-4cd8-b0de-f53acb839caf tags:
---
## 2. Create conda environment
%% Cell type:markdown id: tags:
%% Cell type:markdown id:b1da0745-b29b-4543-a2f6-d6d1a0bd875c tags:
Create new conda environment. The following steps can be repeated if multiple environments should be created. If the Python version differ towards the external Python version, a mix of Conda modules and external modules will not be possible
%% Cell type:code id: tags:
%% Cell type:code id:cdd6f8cb-c9bc-4392-a8f0-3f142a5e954b tags:
``` bash
${CONDA_TARGET_DIR}/bin/conda create -n ${CONDA_ENV} -y python=3.10.4 ipykernel
${CONDA_TARGET_DIR}/bin/conda create -n ${CONDA_ENV} -y python=3.12.10 ipykernel
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id:efbc13b9-90c6-4839-b91d-f3eaa31f1878 tags:
---
## 3. Create/Edit launch script for the Jupyter kernel
%% Cell type:markdown id: tags:
%% Cell type:markdown id:f9d44329-627f-4ce8-82e4-3b81c0c0deec tags:
* 3.1 - Create kernel to allow access to the conda environment. Adapte `module purge` and `PYTHONPATH` according to the comments.
%% Cell type:code id: tags:
%% Cell type:code id:97bccebb-6628-4cd8-918b-dba896024e1f tags:
``` bash
echo '#!/bin/bash
# module purge # optional to disable the external environment, necessary, if python version is different
# Activate your Python virtual environment
source '"${CONDA_TARGET_DIR}"'/bin/activate '"${CONDA_ENV}"'
# Ensure python packages installed in conda are always prefered, not necessary if module purge is used
export PYTHONPATH=${CONDA_PREFIX}/lib/python3.10/site-packages:${PYTHONPATH}
export PYTHONPATH=${CONDA_PREFIX}/lib/python3.12/site-packages:${PYTHONPATH}
exec python -m ipykernel $@' > ${CONDA_TARGET_DIR}/envs/${CONDA_ENV}/kernel.sh
```
%% Cell type:code id: tags:
%% Cell type:code id:9cf78d43-7164-48bd-860b-18609c070faa tags:
``` bash
chmod +x ${CONDA_TARGET_DIR}/envs/${CONDA_ENV}/kernel.sh
echo ${CONDA_TARGET_DIR}/envs/${CONDA_ENV}/kernel.sh
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id:d80e821f-f967-4740-b458-724321b9d7c6 tags:
---
## 4. Create/Edit Jupyter kernel configuration
%% Cell type:markdown id: tags:
%% Cell type:markdown id:2377f50c-47de-4027-833e-7371e0c13a89 tags:
* 4.1 - Create user kernel, if you want to access the conda environment of a colleague, only these steps are necessary
%% Cell type:code id: tags:
%% Cell type:code id:5bd4cd14-2a3e-4518-8766-6d8f87643a38 tags:
``` bash
mkdir -p $HOME/.local/share/jupyter/kernels/conda_${CONDA_ENV}
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id:05dc2d2f-3666-46a0-9cf8-4dddd7117906 tags:
* 4.2 - Adjust kernel.json file
%% Cell type:code id: tags:
%% Cell type:code id:cd4b11ea-3d80-413e-bf1c-ab11fadae811 tags:
``` bash
echo '{
"argv": [
"'"${CONDA_TARGET_DIR}"'/envs/'"${CONDA_ENV}"'/kernel.sh",
"-f",
"{connection_file}"
],
"display_name": "conda_'"${CONDA_ENV}"'",
"language": "python",
"metadata": {
"debugger": true
}
}' > $HOME/.local/share/jupyter/kernels/conda_${CONDA_ENV}/kernel.json
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id:28be0835-bfd9-4ad4-bb86-47717e04725d tags:
Restart of JupyterLab might be necessary to see the kernel in the kernel selection overview.
......
03-HowTos/Create_JupyterKernel_pyenv.ipynb 0 → 100644
View file @ 227ad4ab
This diff is collapsed.
03-HowTos/install-singularity-jupyter-kernel.ipynb
View file @ 227ad4ab
%% Cell type:markdown id: tags:
![header.png](attachment:dee407d8-ed50-42d4-8200-c39761fee461.png)
<!--<h5 style="text-align: right">Author: <a href="mailto:@fz-juelich.de?subject=Jupyter-JSC%20documentation"></a></h5>--><h5 style="text-align: right">Author: Katharina Höflich</h5>
<h5><a href="../index.ipynb">Index</a></h5>
<h1 style="text-align: center">Install containerized Jupyter kernel at Jupyter-JSC</h1>
%% Cell type:markdown id: tags:
This Jupyter notebook will walk you through the installation of a containerized Jupyter kernel (for use at Jupyter-JSC, but it should actually work with any Jupyter server on a system where Singularity is installed). Considerable performance improvements (especially with respect to kernel start-up times) over e.g. conda-based Jupyter kernels on distributed filesystems, as are typically installed on HPC systems, might be experienced. In the example below, the `base-notebook` from the [Jupyter docker stacks](https://jupyter-docker-stacks.readthedocs.io/en/latest/) is used as an IPython kernel (already having the required `ipykernel` package installed), the approach presented here might be extended to any other [Jupyter kernel compatible programming language](https://github.com/jupyter/jupyter/wiki/Jupyter-kernels), though.
Requirements:
* Python environment with an installed `ipykernel` package in a Docker (or Singularity) container
* `container` group access for the JSC systems as described [here](https://apps.fz-juelich.de/jsc/hps/juwels/container-runtime.html#getting-access) in the docs
%% Cell type:markdown id: tags:
Check that the Singularity container runtime is available via the JupyterLab environment,
%% Cell type:code id: tags:
``` bash
singularity --version
```
%% Output
singularity version 3.6.4-1.el8
%% Cell type:markdown id: tags:
Specify the filesystem location that stores the Singularity container image,
%% Cell type:code id: tags:
``` bash
IMAGE_TARGET_DIR=/p/project/cesmtst/hoeflich1/jupyter-base-notebook
```
%% Cell type:markdown id: tags:
Optional, if you already have a Singularity container image available at the above location: Convert a containerized Python environment (e.g. the Jupyter `base-notebook` that is [available via Dockerhub](https://hub.docker.com/r/jupyter/base-notebook)) into a Singularity container image to be used as an example here,
%% Cell type:code id: tags:
``` bash
mkdir -p ${IMAGE_TARGET_DIR}
```
%% Cell type:markdown id: tags:
Note that pulling and converting the Dockerhub image will take a bit of time,
%% Cell type:code id: tags:
``` bash
singularity pull ${IMAGE_TARGET_DIR}/jupyter-base-notebook.sif docker://jupyter/base-notebook &> singularity.log
```
%% Cell type:code id: tags:
``` bash
cat singularity.log | grep -v warn
```
%% Output
INFO: Converting OCI blobs to SIF format
INFO: Starting build...
Getting image source signatures
Copying blob sha256:da7391352a9bb76b292a568c066aa4c3cbae8d494e6a3c68e3c596d34f7c75f8
Copying blob sha256:14428a6d4bcdba49a64127900a0691fb00a3f329aced25eb77e3b65646638f8d
Copying blob sha256:2c2d948710f21ad82dce71743b1654b45acb5c059cf5c19da491582cef6f2601
Copying blob sha256:e3cbfeece0aec396b6793a798ed1b2aed3ef8f8693cc9b3036df537c1f8e34a1
Copying blob sha256:48bd2a353bd8ed1ad4b841de108ae42bccecc44b3f05c3fcada8a2a6f5fa09cf
Copying blob sha256:235d93b8ccf12e8378784dc15c5bd0cb08ff128d61b856d32026c5a533ac3c89
Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Copying blob sha256:b6c06056c45bc1da74604fcf368b02794fe4e36dcae881f4c6b4fa32b37a1385
Copying blob sha256:60918bcbe6d44988e4e48db436996106cc7569a4b880488be9cac90ea6883ae0
Copying blob sha256:762f9ebe4ddc05e56e33f7aba2cdd1be62f747ecd9c8f9eadcb379debf3ebe06
Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Copying blob sha256:1df9d491a0390ecc3f9fac4484c92b2a5f71a79450017f2fca1849f2d6e7f949
Copying blob sha256:be84c8c720e3c53037ac2c5cbc53cf9a2a674503b2c995da1351e5560f60cc12
Copying blob sha256:28807e96859dc8c00c96255dfa51a0822380638a092803e7143473d1870970fb
Copying blob sha256:bcdaf848f29a8bf0efc18a5883dc65a4a7a6b2c6cf4094e5115188ed22165a00
Copying blob sha256:49777cff52f155a9ba35e58102ecec7029dddf52aa4947f2cffbd1af12848e81
Copying blob sha256:7fb3bffa2e730b052c0c7aabd715303cc5830a05b992f2d3d70afeffa0a9ed4f
Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Copying config sha256:79f074439b14ae0634f2f217e5debc159c4e8c3a9ff2e0119e4dc88f9c7e21a5
Writing manifest to image destination
Storing signatures
2021/01/19 11:59:33 info unpack layer: sha256:da7391352a9bb76b292a568c066aa4c3cbae8d494e6a3c68e3c596d34f7c75f8
2021/01/19 11:59:34 info unpack layer: sha256:14428a6d4bcdba49a64127900a0691fb00a3f329aced25eb77e3b65646638f8d
2021/01/19 11:59:34 info unpack layer: sha256:2c2d948710f21ad82dce71743b1654b45acb5c059cf5c19da491582cef6f2601
2021/01/19 11:59:34 info unpack layer: sha256:e3cbfeece0aec396b6793a798ed1b2aed3ef8f8693cc9b3036df537c1f8e34a1
2021/01/19 11:59:34 info unpack layer: sha256:48bd2a353bd8ed1ad4b841de108ae42bccecc44b3f05c3fcada8a2a6f5fa09cf
2021/01/19 11:59:34 info unpack layer: sha256:235d93b8ccf12e8378784dc15c5bd0cb08ff128d61b856d32026c5a533ac3c89
2021/01/19 11:59:34 info unpack layer: sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
2021/01/19 11:59:34 info unpack layer: sha256:b6c06056c45bc1da74604fcf368b02794fe4e36dcae881f4c6b4fa32b37a1385
2021/01/19 11:59:34 info unpack layer: sha256:60918bcbe6d44988e4e48db436996106cc7569a4b880488be9cac90ea6883ae0
2021/01/19 11:59:34 info unpack layer: sha256:762f9ebe4ddc05e56e33f7aba2cdd1be62f747ecd9c8f9eadcb379debf3ebe06
2021/01/19 11:59:34 info unpack layer: sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
2021/01/19 11:59:34 info unpack layer: sha256:1df9d491a0390ecc3f9fac4484c92b2a5f71a79450017f2fca1849f2d6e7f949
2021/01/19 11:59:36 info unpack layer: sha256:be84c8c720e3c53037ac2c5cbc53cf9a2a674503b2c995da1351e5560f60cc12
2021/01/19 11:59:40 info unpack layer: sha256:28807e96859dc8c00c96255dfa51a0822380638a092803e7143473d1870970fb
2021/01/19 11:59:40 info unpack layer: sha256:bcdaf848f29a8bf0efc18a5883dc65a4a7a6b2c6cf4094e5115188ed22165a00
2021/01/19 11:59:40 info unpack layer: sha256:49777cff52f155a9ba35e58102ecec7029dddf52aa4947f2cffbd1af12848e81
2021/01/19 11:59:40 info unpack layer: sha256:7fb3bffa2e730b052c0c7aabd715303cc5830a05b992f2d3d70afeffa0a9ed4f
2021/01/19 11:59:40 info unpack layer: sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
INFO: Creating SIF file...
%% Cell type:markdown id: tags:
Check that the Singularity image is available,
%% Cell type:code id: tags:
``` bash
ls -lah ${IMAGE_TARGET_DIR}
```
%% Output
total 177M
drwxr-sr-x 2 hoeflich1 cesmtst 4.0K Jan 19 11:59 .
drwxr-sr-x 5 hoeflich1 cesmtst 4.0K Jan 19 11:59 ..
-rwxr-xr-x 1 hoeflich1 cesmtst 183M Jan 19 11:59 jupyter-base-notebook.sif
%% Cell type:markdown id: tags:
Now, setup a Jupyter kernel specification with the `install-jupyter-kernel.sh` script from this repository (which basically writes a `kernel.json` file to the home directory location that Jupyter expects for user-specific kernels),
%% Cell type:code id: tags:
``` bash
KERNEL_DISPLAY_NAME=Singularity-Python # don't use whitespaces here!
SINGULARITY_IMAGE=${IMAGE_TARGET_DIR}/jupyter-base-notebook.sif
```
%% Cell type:markdown id: tags:
Link to [install-singularity-jupyter-kernel.sh](https://docs.jupyter-jsc.fz-juelich.de/github/kreuzert/jupyter-jsc-notebooks/blob/documentation/03-HowTos/details/install-singularity-jupyter-kernel.sh)
Link to [install-singularity-jupyter-kernel.sh](https://raw.githubusercontent.com/FZJ-JSC/jupyter-jsc-notebooks/documentation/03-HowTos/details/install-singularity-jupyter-kernel.sh)
%% Cell type:code id: tags:
``` bash
./install-singularity-jupyter-kernel.sh ${KERNEL_DISPLAY_NAME} ${SINGULARITY_IMAGE}
```
%% Cell type:markdown id: tags:
Check that the Jupyter kernel specification was written,
%% Cell type:code id: tags:
``` bash
cat ${HOME}/.local/share/jupyter/kernels/${KERNEL_DISPLAY_NAME}/kernel.json
```
%% Output
{
"argv": [
"singularity",
"exec",
"--cleanenv",
"/p/project/cesmtst/hoeflich1/jupyter-base-notebook/jupyter-base-notebook.sif",
"python",
"-m",
"ipykernel",
"-f",
"{connection_file}"
],
"language": "python",
"display_name": "Singularity-Python"
}
%% Cell type:markdown id: tags:
And that the above Singularity-Python kernel is visible by the Jupyter server,
%% Cell type:code id: tags:
``` bash
jupyter kernelspec list
```
%% Output
Available kernels:
singularity-python /p/home/jusers/hoeflich1/juwels/.local/share/jupyter/kernels/Singularity-Python
ruby /p/software/juwels/stages/Devel-2019a/software/JupyterKernel-Ruby/2.6.3-gcccoremkl-8.3.0-2019.3.199-2019a.2.4/share/jupyter/kernels/ruby
ir35 /p/software/juwels/stages/Devel-2019a/software/JupyterKernel-R/3.5.3-gcccoremkl-8.3.0-2019.3.199-2019a.2.4/share/jupyter/kernels/ir35
pyquantum-1.0 /p/software/juwels/stages/Devel-2019a/software/JupyterKernel-PyQuantum/1.0-gcccoremkl-8.3.0-2019.3.199-2019a.2.4/share/jupyter/kernels/pyquantum-1.0
pyparaview-5.8 /p/software/juwels/stages/Devel-2019a/software/JupyterKernel-PyParaView/5.8.0-gcccoremkl-8.3.0-2019.3.199-2019a.2.4/share/jupyter/kernels/pyparaview-5.8
octave /p/software/juwels/stages/Devel-2019a/software/JupyterKernel-Octave/5.1.0-gcccoremkl-8.3.0-2019.3.199-2019a.2.4/share/jupyter/kernels/octave
julia-1.4 /p/software/juwels/stages/Devel-2019a/software/JupyterKernel-Julia/1.4.2-gcccoremkl-8.3.0-2019.3.199-2019a.2.4/share/jupyter/kernels/julia-1.4
javascript /p/software/juwels/stages/Devel-2019a/software/JupyterKernel-JavaScript/5.2.0-gcccoremkl-8.3.0-2019.3.199-2019a.2.4/share/jupyter/kernels/javascript
cling-cpp17 /p/software/juwels/stages/Devel-2019a/software/JupyterKernel-Cling/0.6-gcccoremkl-8.3.0-2019.3.199-2019a.2.4/share/jupyter/kernels/cling-cpp17
bash /p/software/juwels/stages/Devel-2019a/software/JupyterKernel-Bash/0.7.1-gcccoremkl-8.3.0-2019.3.199-2019a.2.4/share/jupyter/kernels/bash
python3 /p/software/juwels/stages/Devel-2019a/software/Jupyter/2019a.2.4-gcccoremkl-8.3.0-2019.3.199-Python-3.6.8/share/jupyter/kernels/python3
%% Cell type:markdown id: tags:
If so, you should be able to choose and connect to the containerized Python kernel from the drop down menu and/or the kernel launcher tab (a reload of the JupyterLab web page might be necessary).
......
03-HowTos/setup-singularity-jupyter-server.ipynb
View file @ 227ad4ab
%% Cell type:markdown id: tags:
![header.png](attachment:09224a99-4f86-40e9-8761-b299d24050c2.png)
<!--<h5 style="text-align: right">Author: <a href="mailto:?subject=Jupyter-JSC%20documentation">Katharina Höflich</a></h5>--><h5 style="text-align: right">Author: Katharina Höflich</h5>
<h5><a href="../index.ipynb">Index</a></h5>
<h1 style="text-align: center">Setup containerized Jupyter server for Jupyter-JSC</h1>
%% Cell type:markdown id: tags:
This Jupyter notebook will explain how to setup a containerized Jupyter server at Jupyter-JSC. It makes use of the expert features described on page 22 (as of November 25th, 2020) of the training material available [here](https://jupyter-jsc.fz-juelich.de/nbviewer/github/FZJ-JSC/jupyter-jsc-notebooks/blob/master/Jupyter-JSC_supercomputing-in-the-browser.pdf). Please note, that setting up a containerized Jupyter server for the JupyterHub at JSC might introduce certain drawbacks to your Jupyter-JSC experience. Specifically, you will be restricted to the software environment that is installed in your container environment only, which might introduce unwanted side-effects to your JupyterLab-based workflows on the JSC HPC systems. For example, usage of the SLURM batch scheduler commands is not possible, because the SLURM libraries are not visible from the container environment per default. Also, you won't be able to use the Lmod software environment modules provided by JSC. Please note, that if these kind of side-effects are not acceptable, you might rather use a containerized Jupyter kernel as [described here](install-singularity-jupyter-kernel.ipynb). You could also setup your own non-containerized JupyterLab server.
Please note, you can switch back to the default Jupyter-JSC server environment anytime by deleting `$HOME/.jupyter/start_jupyter-jsc.sh` after [login to the JSC systems](https://apps.fz-juelich.de/jsc/hps/juwels/access.html) via SSH.
Requirements:
* Jupyter server environment in a Docker (or Singularity) container
* `container` group access for the JSC systems as described [here](https://apps.fz-juelich.de/jsc/hps/juwels/container-runtime.html#getting-access) in the docs
* Jupyter server environment in a Docker (or Singularity) container
* install at least the python packages **jupyterhub** and **jupyterlab**.
%% Cell type:markdown id: tags:
Specify the filesystem location that stores the Singularity container image,
%% Cell type:code id: tags:
``` bash
IMAGE_TARGET_DIR=/p/project/cesmtst/hoeflich1/jupyter-base-notebook
```
%% Cell type:markdown id: tags:
Convert the example Jupyter base-notebook (that is [available via Dockerhub](https://hub.docker.com/r/jupyter/base-notebook)) into a Singularity container image,
%% Cell type:code id: tags:
``` bash
mkdir -p ${IMAGE_TARGET_DIR}
```
%% Cell type:code id: tags:
``` bash
singularity pull --force ${IMAGE_TARGET_DIR}/jupyter-base-notebook.sif docker://jupyter/base-notebook &> singularity.log
```
%% Cell type:code id: tags:
``` bash
cat singularity.log | grep -v warn
```
%% Output
INFO: Using cached SIF image
%% Cell type:markdown id: tags:
Check that the Singularity image is available,
%% Cell type:code id: tags:
``` bash
ls -lah ${IMAGE_TARGET_DIR}
```
%% Output
total 177M
drwxr-sr-x 2 hoeflich1 cesmtst 4.0K Jan 19 18:50 .
drwxr-sr-x 5 hoeflich1 cesmtst 4.0K Jan 19 18:05 ..
-rwxr-xr-x 1 hoeflich1 cesmtst 183M Jan 19 18:50 jupyter-base-notebook.sif
%% Cell type:markdown id: tags:
Now, manually (!) specify the Singularity image filesystem location in the `start_jupyter-jsc.sh` script and check that the specified path is correct,
%% Cell type:code id: tags:
``` bash
cat start_jupyter-jsc.sh
```
%% Output
#!/bin/bash
# Author: Katharina Höflich
# Repository: https://github.com/FZJ-JSC/jupyter-jsc-notebooks
SINGULARITY_IMAGE=/p/project/cesmtst/hoeflich1/jupyter-base-notebook/jupyter-base-notebook.sif
JUPYTERJSC_USER_CMD="singularity exec ${SINGULARITY_IMAGE} jupyterhub-singleuser --config ${JUPYTER_LOG_DIR}/.config.py"
JUPYTERJSC_USER_CMD="singularity exec ${SINGULARITY_IMAGE} jupyterhub-singleuser --config ${JUPYTER_LOG_DIR}/config.py"
%% Cell type:markdown id: tags:
And copy the `start_jupyter-jsc_singularity.sh` script to the filesystem location expected by Jupyter-JSC (Link to [start_jupyter-jsc_singularity.sh](https://docs.jupyter-jsc.fz-juelich.de/github/kreuzert/jupyter-jsc-notebooks/blob/documentation/03-HowTos/details/start_jupyter-jsc_singularity.sh))
And copy the `start_jupyter-jsc_singularity.sh` script to the filesystem location expected by Jupyter-JSC (Link to [start_jupyter-jsc_singularity.sh](https://docs.jupyter-jsc.fz-juelich.de/github/FZJ-JSC/jupyter-jsc-notebooks/blob/documentation/03-HowTos/details/start_jupyter-jsc_singularity.sh))
%% Cell type:code id: tags:
``` bash
cp start_jupyter-jsc_singulartiy.sh $HOME/.jupyter/start_jupyter-jsc.sh
```
%% Cell type:markdown id: tags:
Finally, opening a new Jupyter session via the Jupyter-JSC control panel should now load the containerized Jupyter server that was setup here.
......
05-News&Updates/Announcement-2022-11_Kernels_and_Extensions.ipynb 0 → 100644
View file @ 227ad4ab
Source diff could not be displayed: it is too large. Options to address this: view the blob.
05-News&Updates/Announcement-2023-07_JupyterLab3.6-Upgrade.ipynb
View file @ 227ad4ab
%% Cell type:markdown id:0be93b1b-f7bb-494f-a0df-9db58fad291c tags:
![header.png](attachment:305e82df-ddd6-439d-bc3d-a036f3e1dc79.png)
<h5 style="text-align: right">Author: <a href="mailto:j.goebbert@fz-juelich.de?subject=Jupyter-JSC%20documentation">Jens Henrik Göbbert</a></h5>
<h5><a href="../index.ipynb">Index</a></h5>
<h1 style="text-align: center">JupyterLab 3.6 on Jupyter-JSC</h1>
%% Cell type:markdown id:293939ae-a70c-4f37-9d01-7e06f21d0a6a tags:
**JupyterLab** is continuously evolving and our most recent version for Jupyter-JSC on https://jupyter-jsc.fz-juelich.de updates now to 3.6.
All the more we are happy to announce this upgrade is based on our recent **Software Stage 2023** and therefore provides much easier access to all the updates on the ~800 modules provided there.
**Jupyter-JSC** saw the light of day six years ago and continues to grow at JSC with new functions and tasks. Its main purpose is to enable interactive supercomputing via JupyterLab at Jülich Supercomputing Centre.
It is part of the Helmholtz Cloud (https://cloud.helmholtz.de) and makes it possible to use the HPC systems of the JSC and also the HDF Cloud in everyday work completely from within the web browser.
Today, about 500 JupyterLab sessions are started by about 100 different users via Jupyter-JSC per week.
To help you get the most out of the JupyterLab 3 setup, here is the official link to [JupyterLab's changelog](https://jupyterlab.readthedocs.io/en/3.6.x/getting_started/changelog.html). And in the following we describe in detail the packages with come in addition to the default Python packages with our JupyterLab environment.
-------------------------
%% Cell type:markdown id:ed0cfca7-a93e-486b-a6d2-63ef0ae8b15f tags:
## JupyterLab-specific Packages
%% Cell type:markdown id:34193c96-70a8-47f4-8c3a-0d9ef3d6dc08 tags:
In the following table you find an overview of the key packages. More details can be found [HERE](https://github.com/easybuilders/JSC/blob/2023/Golden_Repo/j/JupyterLab/JupyterLab-2023.3.6-GCCcore-11.3.0.eb)
| Core packages | Version | Link | Description |
|:-------------:|:-------:|:--------------------------------------|:------------|
| jupyterlab | 3.6.5 | https://jupyterlab.readthedocs.io/en/3.6.x/ | |
| jupyterlab | 3.6.5 | https://jupyterlab.readthedocs.io/en/3.6.x/ | - |
| nbclassic | 1.0.0 | https://nbclassic.readthedocs.io | Jupyter Notebook as a Jupyter Server extension |
| jupyterlab_server | 2.23.0 | https://jupyterlab-server.readthedocs.io | Server components for JupyterLab applications |
| jupyterhub |3.1.1 | https://jupyterhub.readthedocs.io | Multi-user server for Jupyter notebooks |
| jupyterhub | 3.1.1 | https://jupyterhub.readthedocs.io | Multi-user server for Jupyter notebooks |
| Core Extensions | Version | Link | Description |
|:---------------:|:-------:|:--------------------------------------|:------------|
| jupyter-server-proxy | 4.0.0 | https://jupyter-server-proxy.readthedocs.io | Jupyter notebook server extension to proxy web services. |
| jupyterlab-lsp | 4.2.0 | https://github.com/jupyter-lsp/jupyterlab-lsp | Coding assistance for JupyterLab using Language Server Protocol |
| ipympl | 0.9.3 | https://matplotlib.org/ipympl/ | Interactive features of matplotlib in Jupyter |
| ipyleaflet | 0.17.3 | https://ipyleaflet.readthedocs.io | Interactive maps in the Jupyter notebook |
| bqplot | 0.12.39 | https://bqplot.github.io/bqplot/ | Plotting library for IPython/Jupyter notebooks |
| ipyvolume | 0.6.3 | https://github.com/widgetti/ipyvolume | 3d plotting for Python in the Jupyter notebook based on IPython widgets using WebGL |
| jupyterlab_gitlab | 3.0.0 | https://gitlab.com/beenje/jupyterlab-gitlab | A JupyterLab extension for browsing GitLab repositories |
| jupyterlab_git | 0.41.0 | https://github.com/jupyterlab/jupyterlab-git | A Git extension for JupyterLab |
| nbdime | 3.2.1 | https://nbdime.readthedocs.io/ | Tools for diffing and merging of Jupyter notebooks. |
| jupyterlab_latex | 3.2.0 | https://github.com/jupyterlab/jupyterlab-latex | JupyterLab extension for live editing of LaTeX documents |
| jupyterlab_s3_browser | 0.13.0 | https://github.com/IBM/jupyterlab-s3-browser | A JupyterLab extension for browsing S3-compatible object storage |
| plotly | 5.15.0 | https://plotly.com/python/ | Python graphing library for interactive, publication-quality graphs. |
| jupyter_bokeh | 3.0.4 | https://github.com/bokeh/jupyter_bokeh | An extension for rendering Bokeh content in JupyterLab notebooks |
| panel | 0.14.4 | https://panel.holoviz.org/ | The powerful data exploration & web app framework for Python |
| holoviews | 1.16.0 | https://holoviews.org/ | With Holoviews, your data visualizes itself. |
| jupyterlab_hdf | 1.3.0 | https://github.com/jupyterlab/jupyterlab-hdf5 | Open and explore HDF5 files in JupyterLab. Can handle very large (TB) sized files, and datasets of any dimensionality |
| ipyparallel | 8.6.1 | https://ipyparallel.readthedocs.io | IPython Parallel: Interactive Parallel Computing in Python |
| dask_labextension | 6.1.0 | https://github.com/dask/dask-labextension | JupyterLab extension for Dask |
| voila | 0.5.0a4 | https://voila.readthedocs.io | Voilà turns Jupyter notebooks into standalone web applications |
| nbdev | 2.3.12 | https://nbdev.fast.ai/ | Create delightful software with Jupyter Notebooks |
| sidecar | 0.5.2 | https://github.com/jupyter-widgets/jupyterlab-sidecar | A sidecar output widget for JupyterLab |
| dash | 2.11.1 | https://plotly.com/dash | Data Apps & Dashboards for Python. No JavaScript Required. |
| ipyvue | 1.9.2 | https://github.com/widgetti/ipyvue | Jupyter widgets base for Vue libraries |
| ipywebrtc | 0.6.0 | https://github.com/maartenbreddels/ipywebrtc | WebRTC for Jupyter notebook/lab |
| jupyterlab-spellchecker | 0.7.3 | https://github.com/jupyterlab-contrib/spellchecker | Spellchecker for JupyterLab notebook markdown cells and file editor. |
| jupyterlab_code_formatter | 1.6.1 | https://github.com/ryantam626/jupyterlab_code_formatter | A JupyterLab plugin to facilitate invocation of code formatters. |
| jupyterlab_recents | 3.2.0 | https://github.com/NERSC/jupyterlab-recents | A JupyterLab extension that tracks recent files and directories. |
| jupyterlab-favorites | 3.1.1 | https://github.com/NERSC/jupyterlab-favorites | Add the ability to save favorite folders to JupyterLab for quicker browsing |
| jupyterlab-system-monitor | 0.8.0 | https://github.com/jtpio/jupyterlab-system-monitor | JupyterLab extension to display system metrics |
| jupyterlab_iframe | 0.4.4 | https://github.com/timkpaine/jupyterlab_iframe | View html as an embedded iframe in JupyterLab |
| jupyterlab-tour | 3.1.4 | https://github.com/jupyterlab-contrib/jupyterlab-tour | A JupyterLab UI tour built on jupyterlab-tutorial and react-joyride. |
| papermill | 2.4.0 | https://papermill.readthedocs.io | Parameterize, execute, and analyze notebooks |
| pyunicore | 0.15.0 | https://github.com/HumanBrainProject/pyunicore | UNICORE REST bindings for python |
| Optional extension | Version | Link | Description |
|:------------------:|:-------:|:--------------------------------------|:------------|
| jupyterlab-nvdashboard | 0.8.0 | https://github.com/rapidsai/jupyterlab-nvdashboard , [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterExtension-nvdashboard/JupyterExtension-nvdashboard-0.8.0-GCCcore-11.3.0-2023.3.6.eb) | A JupyterLab extension for displaying dashboards of GPU usage. |
| jupyter-slurm-provisioner | 0.6.0 | https://github.com/FZJ-JSC/jupyter-slurm-provisioner , [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterExtension-slurmprovisioner/JupyterExtension-slurmprovisioner-0.6.0-GCCcore-11.3.0-2023.3.6.eb) | Allows to start Jupyter kernels as a SLURM job remote from the Jupyter server |
| nglview | 3.0.6 | http://nglviewer.org/nglview/latest/ , [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterExtension-nglview/JupyterExtension-nglview-3.0.6-GCCcore-11.3.0-2023.3.6.eb)| Jupyter widget to interactively view molecular structures and trajectories |
| jupyter-ai | 0.9.0 | https://jupyter-ai.readthedocs.io/ , [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterExtension-jupyterai/JupyterExtension-jupyterai-0.9.0-GCCcore-11.3.0-2023.3.6.eb) | A generative AI extension for JupyterLab |
| Core Kernels | Version | Link | Description |
|:------------:|:-------:|:--------------------------------------|:------------|
| Bash | 0.9.0 | https://github.com/takluyver/bash_kernel , [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterKernel-Bash/JupyterKernel-Bash-0.9.0-GCCcore-11.3.0-2023.3.6.eb) | A bash kernel for IPython |
| Cling (C++) | 20230205 | https://github.com/root-project/cling/ , [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterKernel-Cling/JupyterKernel-Cling-20230205-GCCcore-11.3.0-2023.3.6.eb) | Jupyter kernel for the C++ programming language |
| Julia | 1.8.5 | https://github.com/JuliaPy/pyjulia , [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterKernel-Julia/JupyterKernel-Julia-1.8.5-GCCcore-11.3.0-2023.3.6.eb) | python interface to julia |
| LFortran | 0.19.0 | https://lfortran.org/ , [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterKernel-LFortran/JupyterKernel-LFortran-0.19.0-GCCcore-11.3.0-2023.3.6.eb) | Modern interactive LLVM-based Fortran compiler |
| Octave | 8.2.0 | https://www.octave.org/ , [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/tree/2023/Golden_Repo/j/JupyterKernel-Octave) | Scientific Programming Language - Powerful mathematics-oriented syntax with built-in 2D/3D plotting and visualization tools |
| R | 4.2.1 | https://irkernel.github.io , [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterKernel-R/JupyterKernel-R-4.2.1-GCCcore-11.3.0-2023.3.6.eb) | R kernel for Jupyter
| Ruby | 3.0.5 | https://github.com/SciRuby/iruby , [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterKernel-Ruby/JupyterKernel-Ruby-3.0.5-GCCcore-11.3.0-2023.3.6.eb) | Ruby kernel for Jupyter
| Community Kernels | Version | Link | Description |
|:------------:|:-------:|:--------------------------------------|:------------|
| DeepLearning | 2023.5 | [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterKernel-PyDeepLearning/JupyterKernel-PyDeepLearning-2023.5-GCCcore-11.3.0-2023.3.6.eb#L108) | Python kernel incl. a collection of extra modules/packages for Deep Learning |
| PyEarthSystem | 2023.5 | [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterKernel-PyEarthSystem/JupyterKernel-PyEarthSystem-2023.5-GCCcore-11.3.0-2023.3.6.eb#L105) | Python kernel incl. a collection of extra modules/packages for the Earth System community |
| QuantumComputing | 2023.5 | [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterKernel-PyQuantum/JupyterKernel-PyQuantum-2023.5-GCCcore-11.3.0-2023.3.6.eb#L101) | Python kernel incl. a collection of extra modules/packages for the quantum computing community |
| Visualization | 2023.5 | [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/tree/2023/Golden_Repo/j/JupyterKernel-PyVisualization) | Python kernel incl. a collection of extra modules/packages for visualization |
| JupyterLab Applications | Version | Link | Description |
|:------------:|:-------:|:--------------------------------------|:------------|
| Xpra | 4.4.6 | https://xpra.org [eb-file](https://gitlab.jsc.fz-juelich.de/software-team/easybuild/-/blob/2023/Golden_Repo/j/JupyterProxy-XpraHTML5/JupyterProxy-XpraHTML5-0.3.5-GCCcore-11.3.0-2023.3.6.eb) | Remote desktop for X11 applications in the browser |
......
07-Communities/Quantum-Computing/D-Wave/.tsp/GraphToHMatrix.py 0 → 100644
View file @ 227ad4ab
from tkinter import *
from tkinter import ttk
from tkinter import filedialog
from os import path, getcwd, remove
from base64 import b64decode
from datetime import datetime
from secrets import token_urlsafe
class Empty(Exception):
pass
class VertexStack:
class _Element:
def __init__(self, data):
if len(data) != 6:
raise ValueError("Invalid data entry. Expected structure: [line1, line2, text, x, y]")
self._data = data
def __str__(self):
return "(L1: {0} L2: {1}, T: {2}, X: {3}, Y: {4}, Q: {5})".format(self._data[0], self._data[1], self._data[2], self._data[3], self._data[4], self._data[5])
def get(self):
return self._data
def __init__(self):
self._data = []
def __len__(self):
return len(self._data)
def is_empty(self):
return len(self._data) == 0
def push(self, d):
self._data.append(self._Element(d))
def top(self):
if self.is_empty():
raise Empty('Stack is empty')
return self._data[-1]
def pop(self):
if self.is_empty():
return None
d = self._data[-1].get()
self._data.pop()
return d
def stringify(self):
result = ""
for d in self._data:
c = d.get()
result += str(c[3]) + " " + str(c[4]) + "\n"
return result
def get(self, e):
if self.is_empty() or e >= len(self._data):
raise Empty('Something is wrong')
return self._data[e].get()
class Event:
def __init__(self, x, y):
self.x = x
self.y = y
class GraphCanvas(Canvas):
_CROSS_SIZE = 7
_LABELS = ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z']
id = ""
def __init__(self, parent, log, **kwargs):
super().__init__(parent, **kwargs)
self.grid(column=1, rowspan=5, sticky=(N, S, E, W))
self.create_rectangle(4, 4, 500, 350, outline="dim gray", width=2, fill="white")
self.bind("<Button-1>", self.addVertex)
self.bind("<Control-z>", self.deleteLast)
self._log = log
self._vertex_count = 0
self._vertices = VertexStack()
self._last_action = []
self._lines = []
def addVertex(self, event):
x, y = event.x, event.y
if self._vertex_count >= len(self._LABELS):
self._log.logMsg("ERROR: Maximum number of vertices (%d) reached! Last entry will not be saved." % len(self._LABELS), "error")
return
l1 = self.create_line((x-self._CROSS_SIZE, y, x+self._CROSS_SIZE, y), fill="red", width=3)
l2 = self.create_line((x, y-self._CROSS_SIZE, x, y+self._CROSS_SIZE), fill="red", width=3)
t = self.create_text((x+self._CROSS_SIZE, y-self._CROSS_SIZE), text=self._LABELS[self._vertex_count], anchor="sw", font="TkMenuFont", fill="dim gray")
self._log.logMsg("Added vertex at: x = {0}, y = {1}".format(x, y))
if self._vertex_count == 8:
self._log.logMsg("Computational limit of D-Wave 2000Q (8) exceeded. Are you sure you want to add more vertices?", "warning")
self._vertices.push([l1, l2, t, x, y, "q"+str(self._vertex_count)])
self.id = ''.join([c for c in token_urlsafe(10) if c not in '-_OI0l'])[:5]
self._vertex_count += 1
def deleteLast(self):
d = self._vertices.pop()
if not d:
self._log.logMsg("Cannot perform undo action. Canvas already empty", "warning")
return
self._last_action = d[:]
self.delete(d[0])
self.delete(d[1])
self.delete(d[2])
self._vertex_count -= 1
def restoreLast(self):
if len(self._last_action) == 0:
return
x, y = self._last_action[3], self._last_action[4]
l1 = self.create_line((x-self._CROSS_SIZE, y, x+self._CROSS_SIZE, y), fill="red", width=3)
l2 = self.create_line((x, y-self._CROSS_SIZE, x, y+self._CROSS_SIZE), fill="red", width=3)
t = self.create_text((x+self._CROSS_SIZE, y-self._CROSS_SIZE), text=self._LABELS[self._vertex_count], anchor="sw", font="TkMenuFont", fill="dim gray")
self._vertices.push([l1, l2, t, x, y, "q"+str(self._vertex_count)])
self._vertex_count += 1
self._last_action = []
def getStackSnapshot(self):
return self._vertices
def __len__(self):
return len(self._vertices)
def deleteLines(self):
for l in self._lines:
self.delete(l)
def generateHamiltonian(self):
self.deleteLines()
distances = {}
for i in range(len(self._vertices)):
for j in range(i+1, len(self._vertices)):
point1 = self._vertices.get(i)
point2 = self._vertices.get(j)
d = float("{:.2f}".format(((point2[3]-point1[3])**2 + (point2[4]-point1[4])**2)**0.5))
distances[str(point1[5]+point2[5])] = d
distances[str(point2[5]+point1[5])] = d
self._lines.append(self.create_line((point1[3], point1[4], point2[3], point2[4]), fill="light gray", width=2))
self._lines.append(self.create_text(((point1[3]+point2[3])/2, (point1[4]+point2[4])/2 - 5), text=str(d), anchor="sw", font="TkMenuFont", fill="light gray"))
tour = TourMatrix(len(self._vertices), distances)
return tour.getHamiltonian()
class TourMatrix():
SCALE = 0.01
def __init__(self, dim, dist):
counter = 0
self._field = []
for _ in range(dim):
f = []
for _ in range(dim):
f.append("q"+str(counter))
counter += 1
self._field.append(f)
self._dim = dim
self._distances = dist
def getHamiltonian(self):
h = [[0.0 for x in range(self._dim**2)] for y in range(self._dim**2)]
# Diagonal
for i in range(len(h)):
h[i][i] = -8.0
# "1 entry/row and 1 entry/column is allowed" rule
for i in range(len(self._field)):
for j in range(len(self._field[i])):
# Iterate row
for k in self._field[i][j+1:]:
index = str(self._field[i][j]+k).split("q")[1:]
h[int(index[0])][int(index[1])] = 8.0
# Iterate column
for k in range(len(self._field)):
if k > i:
index = str(self._field[i][j]+self._field[k][j]).split("q")[1:]
h[int(index[0])][int(index[1])] = 8.0
# "distances" rule
for i in range(len(self._field)):
for j in range(len(self._field[i])):
for k in range(len(self._field)):
if k == i:
continue
field1 = int(self._field[i][j][1:])
field2 = int(self._field[k][(j+1) % len(self._field[i])][1:])
if field1 < field2:
h[field1][field2] = float("{:.1f}".format(self._distances["q"+str(i)+"q"+str(k)]*self.SCALE))
else:
h[field2][field1] = float("{:.1f}".format(self._distances["q"+str(i)+"q"+str(k)]*self.SCALE))
return h
class LogWindow(Text):
def __init__(self, parent, **kwargs):
super().__init__(parent, **kwargs)
# Configure tags
self.tag_configure("warning", foreground="#e0ba1f")
self.tag_configure("error", foreground="#e0291f")
self.tag_configure("notification", foreground="black")
self.insert('end', "--- GRAPH TO MATRIX - TSPQ (v.1.1.0) ---\n")
self.insert('end', "[" + datetime.now().strftime("%Y/%m/%d %H:%M:%S") + "] START LOG")
self['state'] = "disabled"
def insertToGrid(self):
self.grid(columnspan=3, row=7, sticky=(N, S, E, W), padx=10, pady=10)
def logMsg(self, msg, msg_type="notification"):
msg_types = ["notification", "warning", "error"]
if msg_type not in msg_types:
raise ValueError("Invalid message type. Expected one of: %s" % msg_types)
numlines = int(self.index('end - 1 line').split('.')[0])
self['state'] = 'normal'
if numlines == 24:
self.delete(1.0, 2.0)
if self.index('end-1c') != '1.0':
self.insert('end', '\n')
self.insert('end', "[" + datetime.now().strftime("%Y/%m/%d %H:%M:%S") + "] " + msg, (msg_type))
self.see('end')
self['state'] = 'disabled'
class GraphToHMatrix:
global DESC_PADDING
DESC_PADDING = 30
h_matrix = []
h_matrix_name = "- n.a. -"
tex_equation = {}
def __init__(self, root):
# Basic options
self._root = root
self._root.title("Graph to Matrix - TSPQ")
self._generated = StringVar(value="Generated: "+self.h_matrix_name)
# Create menu bar
menubar = Menu(self._root)
root['menu'] = menubar
menu_file = Menu(menubar)
menu_edit = Menu(menubar)
menubar.add_cascade(menu=menu_file, label='File')
menubar.add_cascade(menu=menu_edit, label='Edit')
menu_file.add_command(label='Clear', command=self._clearCanvas)
menu_file.add_command(label='Open...', command=self._openFile)
menu_file.add_command(label='Exit', command=self._exitProgram)
menu_edit.add_command(label='Undo', command=self._undo)
menu_edit.add_command(label='Redo', command=self._redo)
# Title
ttk.Label(self._root, text="Draw on the canvas to set vertices.").grid(column=1, row=1, sticky=W)
# Logging window
self._log = LogWindow(self._root, width=40, height=5, background="white", wrap="none", state="normal")
# Canvas
self._c = GraphCanvas(self._root, self._log, width=500, height=350, background="white")
# Labels and buttons
ttk.Label(self._root, text="Click to generate the distances\nand the Hamilton matrix.\nThis can take a few moments,\ndepending on the number of\nvertices drawn.", justify="center").grid(column=2, row=2, padx=DESC_PADDING)
ttk.Button(self._root, text="Generate", command=self._generateHamiltonian).grid(column=2, row=3)
ttk.Label(self._root, textvariable=self._generated).grid(column=2, row=4)
ttk.Button(self._root, text="Save Hamiltonian...", command=self._saveHamil).grid(column=2, row=5)
ttk.Button(self._root, text="Save Vertices...", command=self._saveConfig).grid(column=2, row=6)
self._log.insertToGrid()
# Key bindings and shortcuts
self._root.bind_all("<Control-z>", self._undo)
self._root.bind_all("<Control-y>", self._redo)
def _generateHamiltonian(self):
self._log.logMsg("Generating Hamiltonian, please stand by...", "warning")
self.h_matrix = self._c.generateHamiltonian()
self.h_matrix_name = "hmatrix-" + self._c.id + "_x" + str(len(self.h_matrix))
self._log.logMsg("Generating complete! Result: " + self.h_matrix_name)
self._generated.set("Generated: "+self.h_matrix_name)
def _saveHamil(self):
if len(self.h_matrix) == 0:
self._log.logMsg("Unable to save Hamilton matrix. No matrix generated ({0})".format(self.h_matrix), "error")
return
filename = filedialog.asksaveasfilename(initialfile=self.h_matrix_name+".txt", defaultextension=".txt", filetypes=[("Text Document", "*.txt"),("All Files", "*.*")])
if not filename:
self._log.logMsg("Unable to save Hamilton matrix. No save location specified", "error")
return
self._log.logMsg("Saving Hamilton matrix in: {0}".format(filename))
content = ""
for i in range(len(self.h_matrix)):
for j in range(len(self.h_matrix[i])):
content += str(self.h_matrix[i][j]) + " "
content = content[:-1]
if i != len(self.h_matrix) - 1:
content += "\n"
f = open(filename, "w")
f.write(content)
f.close()
self._log.logMsg("Saving complete!")
def _saveConfig(self):
if len(self._c) == 0:
self._log.logMsg("Unable to save configuration file. No vertices set (%d)" % len(self._c), "error")
return
filename = filedialog.asksaveasfilename(initialfile=self._c.id+".vertices", defaultextension=".vertices", filetypes=(("Vertices File", "*.vertices"), ("All Files", "*.*")))
if not filename:
self._log.logMsg("Unable to save configuration file. No save location specified", "error")
return
self._log.logMsg("Saving configuration file in: {0}".format(filename))
vertices = self._c.getStackSnapshot()
content = vertices.stringify()
f = open(filename, "w")
f.write(content)
f.close()
self._log.logMsg("Saving complete!")
# Menubar functions
def _clearCanvas(self):
for _ in range(len(self._c)):
self._c.deleteLast()
self._c.deleteLines()
def _openFile(self):
filename = filedialog.askopenfilename(initialdir="./", filetypes=(("Vertices File", "*.vertices"),("All Files", "*.*")))
if not filename:
self._log.logMsg("Unable to load configuration file. No file specified", "error")
return
self._log.logMsg("Importing vertices configuration from {0}".format(filename))
self._clearCanvas()
f = open(filename, "r")
c = 0
while True:
c += 1
line = f.readline()
if not line or line == "\n":
break
pos = line[:-1].split(" ")
try:
for i in range(len(pos)):
pos[i] = int(pos[i])
except:
self._log.logMsg("Unable to parse configuration file. Corrupted data format (line %d)" % c, "error")
return
if len(pos) != 2 or not isinstance(pos[0], int) or not isinstance(pos[1], int):
self._log.logMsg("Unable to verify configuration data. Corrupted data (line %d)" % c, "error")
return
self._c.addVertex(Event(pos[0], pos[1]))
f.close()
self._log.logMsg("Import complete!")
def _exitProgram(self):
self._root.destroy()
def _undo(self, e=None):
self._c.deleteLast()
def _redo(self, e=None):
self._c.restoreLast()
if __name__ == "__main__":
root = Tk()
window = GraphToHMatrix(root)
root.mainloop()
\ No newline at end of file
07-Communities/Quantum-Computing/D-Wave/.tsp/PathVisualizer.py 0 → 100644
View file @ 227ad4ab
from tkinter import *
from tkinter import ttk
from tkinter import filedialog
from os import path, getcwd, remove
from base64 import b64decode
from datetime import datetime
class Empty(Exception):
pass
class VertexStack:
class _Element:
def __init__(self, data):
if len(data) != 6:
raise ValueError(
"Invalid data entry. Expected structure: [line1, line2, text, x, y]")
self._data = data
def __str__(self):
return "(L1: {0} L2: {1}, T: {2}, X: {3}, Y: {4}, Q: {5})".format(self._data[0], self._data[1], self._data[2], self._data[3], self._data[4], self._data[5])
def get(self):
return self._data
def __init__(self):
self._data = []
def __len__(self):
return len(self._data)
def is_empty(self):
return len(self._data) == 0
def push(self, d):
self._data.append(self._Element(d))
def top(self):
if self.is_empty():
raise Empty('Stack is empty')
return self._data[-1]
def pop(self):
if self.is_empty():
return None
d = self._data[-1].get()
self._data.pop()
return d
def stringify(self):
result = ""
for d in self._data:
c = d.get()
result += str(c[3]) + " " + str(c[4]) + "\n"
return result
def get(self, e):
if self.is_empty() or e >= len(self._data):
raise Empty('Something is wrong')
return self._data[e].get()
class Event:
def __init__(self, x, y):
self.x = x
self.y = y
class GraphCanvas(Canvas):
_CROSS_SIZE = 7
_LABELS = ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z']
def __init__(self, parent, log, **kwargs):
super().__init__(parent, **kwargs)
self.grid(columnspan=4, row=2, sticky=(N, S, E, W))
self.create_rectangle(4, 4, 500, 350, outline="dim gray", width=2, fill="white")
self._log = log
self._vertex_count = 0
self._vertices = VertexStack()
self._lines = []
def __len__(self):
return len(self._vertices)
def addVertex(self, event):
x, y = event.x, event.y
if self._vertex_count >= len(self._LABELS):
self._log.logMsg("ERROR: Maximum number of vertices (%d) reached! Last entry will not be saved." % len(self._LABELS), "error")
return
l1 = self.create_line((x-self._CROSS_SIZE, y, x+self._CROSS_SIZE, y), fill="red", width=3)
l2 = self.create_line((x, y-self._CROSS_SIZE, x, y+self._CROSS_SIZE), fill="red", width=3)
t = self.create_text((x+self._CROSS_SIZE, y-self._CROSS_SIZE), text=self._LABELS[self._vertex_count], anchor="sw", font="TkMenuFont", fill="dim gray")
self._log.logMsg("Added vertex at: x = {0}, y = {1}".format(x, y))
if self._vertex_count == 8:
self._log.logMsg("Computational limit of D-Wave 2000Q (8) exceeded. Are you sure you want to add more vertices?", "warning")
self._vertices.push([l1, l2, t, x, y, "q"+str(self._vertex_count)])
self._vertex_count += 1
def drawTour(self, tour, num):
tour_matrix = []
points = []
for i in range(0, len(tour)-1, len(self._vertices)):
tour_matrix.append(tour[i:i+int(len(tour)**0.5)])
for i in range(len(tour_matrix[0])):
for j in range(len(tour_matrix)):
if tour_matrix[j][i] == 1:
points.append(self._vertices.get(j))
if len(points) > 1:
d = float("{:.2f}".format(((points[-2][3]-points[-1][3])**2 + (points[-2][4]-points[-1][4])**2)**0.5))
self._lines.append(self.create_line((points[-1][3], points[-1][4], points[-2][3], points[-2][4]), fill="lime green", width=2))
self._lines.append(self.create_text(((points[-1][3]+points[-2][3])/2, (points[-1][4]+points[-2][4])/2 - 5), text=str(d), anchor="sw", font="TkMenuFont", fill="lime green"))
d = float("{:.2f}".format(((points[-1][3]-points[0][3])**2 + (points[-1][4]-points[0][4])**2)**0.5))
self._lines.append(self.create_line((points[-1][3], points[-1][4], points[0][3], points[0][4]), fill="lime green", width=2))
self._lines.append(self.create_text(((points[-1][3]+points[0][3])/2, (points[-1][4]+points[0][4])/2 - 5), text=str(d), anchor="sw", font="TkMenuFont", fill="lime green"))
self._log.logMsg("Drawn tour {0} | Energy: {1} {2}".format(num, tour[-1], ("(best)" if num == 0 else "")))
def deleteLast(self):
d = self._vertices.pop()
if not d:
self._log.logMsg("Cannot perform undo action. Canvas already empty", "warning")
return
self._last_action = d[:]
self.delete(d[0])
self.delete(d[1])
self.delete(d[2])
self._vertex_count -= 1
def deleteLines(self):
for l in self._lines:
self.delete(l)
class LogWindow(Text):
def __init__(self, parent, **kwargs):
super().__init__(parent, **kwargs)
# Configure tags
self.tag_configure("warning", foreground="#e0ba1f")
self.tag_configure("error", foreground="#e0291f")
self.tag_configure("notification", foreground="black")
self.insert('end', "--- PATH VISUALIZER - TSPQ (v.1.0.1) ---\n")
self.insert('end', "[" + datetime.now().strftime("%Y/%m/%d %H:%M:%S") + "] START LOG")
self['state'] = "disabled"
def insertToGrid(self):
self.grid(columnspan=4, row=5, sticky=(N, S, E, W), padx=10, pady=10)
def logMsg(self, msg, msg_type="notification"):
msg_types = ["notification", "warning", "error"]
if msg_type not in msg_types:
raise ValueError("Invalid message type. Expected one of: %s" % msg_types)
numlines = int(self.index('end - 1 line').split('.')[0])
self['state'] = 'normal'
if numlines == 24:
self.delete(1.0, 2.0)
if self.index('end-1c') != '1.0':
self.insert('end', '\n')
self.insert('end', "[" + datetime.now().strftime("%Y/%m/%d %H:%M:%S") + "] " + msg, (msg_type))
self.see('end')
self['state'] = 'disabled'
class PathVisualizer():
solution_name = "- n. a. -"
config_name = "- n. a. -"
def __init__(self, root):
# Basic options
self._root = root
self._root.title("Path Visualizer - TSPQ")
self._opened_solution = StringVar(value="Opened solution(s): "+self.solution_name)
self._opened_config = StringVar(value="Opened graph: "+self.config_name)
# Create menu bar
menubar = Menu(self._root)
root['menu'] = menubar
menu_file = Menu(menubar)
menu_edit = Menu(menubar)
menubar.add_cascade(menu=menu_file, label='File')
menu_file.add_command(label='Clear', command=self._clearCanvas)
menu_file.add_command(label='Open solution...', command=self._openSolution)
menu_file.add_command(label='Open graph...', command=self._openConfig)
menu_file.add_command(label='Exit', command=self._exitProgram)
# Title
ttk.Label(self._root, text="View the trips calculated by the quantum annealer on the canvas.", justify="center").grid(columnspan=4, row=1)
# Logging window
self._log = LogWindow(self._root, width=40, height=5, background="white", wrap="none", state="normal")
# Canvas
self._c = GraphCanvas(self._root, self._log, width=500, height=350, background="white")
# Labels and buttons
ttk.Label(self._root, textvariable=self._opened_config, justify="center").grid(columnspan=2, row=3, column=0)
ttk.Label(self._root, textvariable=self._opened_solution, justify="center").grid(columnspan=2, row=3, column=2)
ttk.Button(self._root, text="<<<", command=self._prevSolution).grid(column=0, row=4, pady=10)
ttk.Button(self._root, text=">>>", command=self._nextSolution).grid(column=3, row=4, pady=10)
ttk.Button(self._root, text="Open graph...", command=self._openConfig).grid(column=1, row=4, pady=10)
ttk.Button(self._root, text="Open solution(s)...", command=self._openSolution).grid(column=2, row=4, pady=10)
self._log.insertToGrid()
self._loadedSolutions = []
self._display = 0
def _openSolution(self):
if len(self._c) == 0:
self._log.logMsg("Importing aborted. Open a graph first before viewing the solutions", "error")
return
filename = filedialog.askopenfilename(initialdir="./", filetypes=(("Text Document", "*.txt"),("All Files", "*.*")))
if not filename:
self._log.logMsg("Unable to load solutions. No file specified", "error")
return
self._log.logMsg("Importing solutions from {0}".format(filename))
self._loadedSolutions = []
self._c.deleteLines()
f = open(filename, "r")
c = 0
while True:
c += 1
line = f.readline()
if not line or line == "\n":
break
try:
array = line[line.find("[")+1:line.find("]")].split(" ")
if len(array) != len(self._c)**2:
self._log.logMsg("Unable to parse solutions. Array size ({0}) is not the quadratic of the number of vertices ({1})".format(len(array), len(self._c)), "error")
return
for i in range(len(array)):
array[i] = int(array[i])
if array[i] != 1 and array[i] != 0:
self._log.logMsg("Unable to parse solutions. Non-binary entry found (line %d)" % c, "error")
return
array.append(float("{:.2f}".format(float(line[:line.find("\t")]))))
self._loadedSolutions.append(array)
except:
self._log.logMsg("Unable to parse solutions. Corrupted data (line %d)" % c, "error")
return
if c == 10:
break
f.close()
self.solution_name = filename[filename.rfind("/")+1:filename.rfind(".")]
self._opened_solution.set("Opened graph: "+self.solution_name)
self._display = 0
self._c.drawTour(self._loadedSolutions[self._display], self._display)
self._log.logMsg("Import complete!")
def _openConfig(self):
filename = filedialog.askopenfilename(initialdir="./", filetypes=(("Vertices File", "*.vertices"),("All Files", "*.*")))
if not filename:
self._log.logMsg("Unable to load configuration file. No file specified", "error")
return
self._log.logMsg("Importing vertices configuration from {0}".format(filename))
self._clearCanvas()
self._loadedSolutions = []
f = open(filename, "r")
c = 0
while True:
c += 1
line = f.readline()
if not line or line == "\n":
break
pos = line[:-1].split(" ")
try:
for i in range(len(pos)):
pos[i] = int(pos[i])
except:
self._log.logMsg("Unable to parse configuration file. Corrupted data format (line %d)" % c, "error")
return
if len(pos) != 2 or not isinstance(pos[0], int) or not isinstance(pos[1], int):
self._log.logMsg("Unable to verify configuration data. Corrupted data (line %d)" % c, "error")
return
self._c.addVertex(Event(pos[0], pos[1]))
f.close()
self.config_name = filename[filename.rfind("/")+1:filename.rfind(".")]
self._opened_config.set("Opened graph: "+self.config_name)
self._log.logMsg("Import complete!")
def _prevSolution(self):
if self._display == 0:
return
self._c.deleteLines()
self._display -= 1
self._c.drawTour(self._loadedSolutions[self._display], self._display)
def _nextSolution(self):
if self._display == len(self._loadedSolutions)-1:
return
self._c.deleteLines()
self._display += 1
self._c.drawTour(self._loadedSolutions[self._display], self._display)
def _clearCanvas(self):
for _ in range(len(self._c)):
self._c.deleteLast()
self._c.deleteLines()
self._loadedSolutions = []
self.config_name = "- n. a. -"
self.solution_name = "- n. a. -"
self._opened_config.set("Opened graph: "+self.config_name)
self._opened_solution.set("Opened solution(s): "+self.solution_name)
def _exitProgram(self):
self._root.destroy()
if __name__ == "__main__":
root = Tk()
window = PathVisualizer(root)
root.mainloop()
\ No newline at end of file
07-Communities/Quantum-Computing/D-Wave/Getting_started.ipynb 0 → 100644
View file @ 227ad4ab
Source diff could not be displayed: it is too large. Options to address this: view the blob.