Skip to content
Snippets Groups Projects
Commit e13174c1 authored by Tim Kreuzer's avatar Tim Kreuzer
Browse files

update faq hdfcloud notebook

parent c6c321e8
No related branches found
No related tags found
No related merge requests found
%% Cell type:markdown id: tags:
# Welcome to the Jupyter@JSC Virtual Machine JupyterLab
# Welcome to the Jupyter-JSC Virtual Machine JupyterLab
%% Cell type:markdown id: tags:
---
## Where is this JupyterLab running?
%% Cell type:markdown id: tags:
      This JupyterLab instance is running on a [virtual machine](https://www.techopedia.com/definition/4805/virtual-machine-vm) on the [HDF-Cloud](https://www.fz-juelich.de/ias/jsc/EN/Expertise/SciCloudServices/HDFCloud/_node.html). It is started as a [Docker Container](https://www.docker.com/resources/what-container).
%% Cell type:markdown id: tags:
## Is there any limitation?
%% Cell type:markdown id: tags:
      You can use a maximum of 4 GB of RAM.
      If you need more memory please start your JupyterLab on a HPC system.
%% Cell type:markdown id: tags:
## How do I stop this JupyterLab?
%% Cell type:markdown id: tags:
      You can stop this JupyterLab in the Control Panel of Jupyter@JSC.
      You can reach the Control Panel [here](https://zam10116.zam.kfa-juelich.de/hub/start) or in the menu File -> Hub Control Panel.
%% Cell type:markdown id: tags:
## Why is the memory usage so high?
%% Cell type:markdown id: tags:
      If you have the feeling that your used Memory in the top right corner is too high, ensure that your kernels were shutdown.
      Whenever a Tab of your Notebook is closed, the kernel is still running in the background.
      You can see all running kernels by clicking on this symbol ![Kernels](https://jupyter-jsc.fz-juelich.de/hub/static/images/kernels.png "kernels") on the left sidebar.
      You can stop this JupyterLab in the Control Panel of Jupyter-JSC.
      You can reach the Control Panel [here](https://jupyter-jsc.fz-juelich.de/hub/start) or in the menu File -> Hub Control Panel.
%% Cell type:markdown id: tags:
## How can I upload files to JupyterLab?
%% Cell type:markdown id: tags:
      You can use git or you upload files with this button in the top left corner:
      ![Upload files](https://zam10183.zam.kfa-juelich.de/hub/static/images/upload.png "Upload Files")
      ![Upload files](https://jupyter-jsc.fz-juelich.de/hub/static/images/upload.png "Upload Files")
%% Cell type:markdown id: tags:
## How can I download a file from JupyterLab?
%% Cell type:markdown id: tags:
      Just right click on a file and click "Download".
%% Cell type:markdown id: tags:
## What kind of Docker Images do we support?
%% Cell type:markdown id: tags:
      You can select between [these eight](https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-base-notebook) Images.
      We installed additionally a few JupyterLab [extensions](https://nbviewer.jupyter.org/github/kreuzert/Jupyter-JSC/blob/master/Extensions.ipynb).
      We installed additionally a few JupyterLab extensions. You can find a List of all installed extensions [here.](https://nbviewer.jupyter.org/github/kreuzert/Jupyter-JSC/blob/master/Extensions.ipynb)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you want to add your own Docker Image to Jupyter@JSC please contact the <a href="mailto:ds-support@fz-juelich.de?subject=Jupyter@JSC Support&amp;body=Please describe your problem here. (english or german)">Jupyter@JSC support</a>.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;We installed additionally a few JupyterLab extensions.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you want to add your own Docker Image to Jupyter-JSC please contact the <a href="mailto:ds-support@fz-juelich.de?subject=Jupyter-JSC Support&amp;body=Please describe your problem here. (english or german)">Jupyter-JSC support</a>.
%% Cell type:markdown id: tags:
## How much disk space do I have?
%% Cell type:markdown id: tags:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You can store 10 GB of data in /home/jovyan/work.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<font color='red'>All files in the ~/work directory will be stored persistently.</font>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<font color='red'>All files in the ~/work directory will be accessible in all of your HDF-Cloud JupyterLabs.</font>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<font color='red'>Any other files or directories will be deleted if you stop this JupyterLab.</font>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Under no circumstances are we liable for any lost data.
%% Cell type:markdown id: tags:
## How much memory do I have?
%% Cell type:markdown id: tags:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;These JupyterLabs are limited to 4GB Ram.
%% Cell type:markdown id: tags:
## Collaborative work
# Collaborative work
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;We offer four different solutions to share your work with your colleagues.
%% Cell type:markdown id: tags:
### GIT
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You can use the git command in a terminal to work on any git repositories.
%% Cell type:markdown id: tags:
### Projects
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Projects are the best way, if you just want to share a few files (or notebooks) with your colleague, who is also using Jupyter@JSC.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Projects are the best way, if you just want to share a few files (or notebooks) with your colleague, that is also using Jupyter-JSC.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Projects are directories, that are generated on the [virtual machine](https://www.techopedia.com/definition/4805/virtual-machine-vm) where your JupyterLab is running.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you add another user to your project this directory will be mounted into their [Docker Container](https://www.docker.com/resources/what-container) .
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You can manage your projects with the following command:
```
$ bash /home/jovyan/manage_projects.sh
```
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You can store up to 10 GB of data on all your projects combined.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you want to add users to your project you need their email address. It must be the same email address, that is connected to their [JSC webservice credentials](https://judoor.fz-juelich.de/login) .
%% Cell type:markdown id: tags:
### B2DROP
#### B2DROP
%% Cell type:markdown id: tags:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You can easily mount your [B2DROP](https://b2drop.eudat.eu) files into this JupyterLab.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Just run the command "$ mount B2DROP" in a terminal and insert your [application credentials](https://eudat.eu/services/userdoc/b2drop#UserDocumentation-B2DROPUsage-WebDavclient)\.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Just run the command `mount B2DROP` in a terminal and insert your [application credentials](https://eudat.eu/services/userdoc/b2drop#UserDocumentation-B2DROPUsage-WebDavclient)\.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you want to store your [application credentials](https://eudat.eu/services/userdoc/b2drop#UserDocumentation-B2DROPUsage-WebDavclient) add the following line to /home/jovyan/work/.davfs2/secrets:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;https://b2drop.eudat.eu/remote.php/webdav \<USERNAME> \<PASSWORD>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;To unmount it run `umount B2DROP`
%% Cell type:markdown id: tags:
### HPCMOUNT (experimental)
%% Cell type:markdown id: tags:
<span style="color:red">Your groups are not mounted. So you have only access to files that are yours.</span>
### HPCMOUNT
%% Cell type:markdown id: tags:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You can mount your files on the HPC system with [sshfs](https://en.wikipedia.org/wiki/SSHFS) .
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;For this we created the script at /home/jovyan/mount_hpc.sh .
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;To mount your HPC files just execute
```
$ bash /home/jovyan/mount_hpc.sh
```
### &nbsp;&nbsp;&nbsp; What will the script do?
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you run it for the first time, it will create a private/public key pair for you. For your safety, a passphrase is required for this key.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Afterward, it will upload the public key to your [JUDAC](https://www.fz-juelich.de/ias/jsc/EN/Expertise/Datamanagement/JUDAC/JUDAC_node.html) ~/.ssh/authorized_keys file. This will be done with [UNICORE](https://www.unicore.eu) and the [credentials](https://judoor.fz-juelich.de/login) you used to log in to Jupyter@JSC.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Afterward, it will upload the public key to your [JUDAC](https://www.fz-juelich.de/ias/jsc/EN/Expertise/Datamanagement/JUDAC/JUDAC_node.html) ~/.ssh/authorized_keys file. This will be done with [UNICORE](https://www.unicore.eu) and the [credentials](https://judoor.fz-juelich.de/login) you used to log in to Jupyter-JSC.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If your key is uploaded (or you have already used this script before) it will run the sshfs command to mount your HPC files.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You have to enter your passphrase, that you chose in the first run, to use your generated private key.
### &nbsp;&nbsp;&nbsp; How can I unmount it?
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Just run the command
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;```$ fusermount -u /home/jovyan/HPCMOUNT```
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The folder will be automatically unmounted if you stop this JupyterLab.
### &nbsp;&nbsp;&nbsp; Where is my private key stored?
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The private key will be stored, like your other files, in the /home/jovyan/work directory, on a [virtual machine](https://www.techopedia.com/definition/4805/virtual-machine-vm) on the [HDF-Cloud](https://www.fz-juelich.de/ias/jsc/EN/Expertise/SciCloudServices/HDFCloud/_node.html) .
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The access to this virtual machine is limited to the administrators of Jupyter@JSC.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The access to this virtual machine is limited to the administrators of Jupyter-JSC.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Since your private key is encrypted with a passphrase, even an administrator is not able to use your private key to connect to any HPC system.
### &nbsp;&nbsp;&nbsp; I want to mount another account
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you have multiple accounts (connected to your [JSC webservice credentials](https://judoor.fz-juelich.de/login)) you can tell the script which one it should use.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;```$ bash /home/jovyan/mount_hpc.sh <SYSTEM> <ACCOUNT>```
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\<SYSTEM> is the HPC system that is used to upload your public key. Please use one of these: JURECA, JUWELS, JURON. Your account must have an active project on the chosen system.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\<ACCOUNT> is your HPC account on this system ( and on [JUDAC](https://www.fz-juelich.de/ias/jsc/EN/Expertise/Datamanagement/JUDAC/JUDAC_node.html)) .
### &nbsp;&nbsp;&nbsp; I want to delete the public key from my HPC account
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Jupyter@JSC does not delete anything from your authorized_keys file. So if you want to remove the public key you have to log in to JUDAC and remove it manually.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Jupyter-JSC does not delete anything from your authorized_keys file. So if you want to remove the public key you have to log in to JUDAC and remove it manually.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment