Commit e13174c1 authored by Tim Kreuzer's avatar Tim Kreuzer
Browse files

update faq hdfcloud notebook

parent c6c321e8
%% Cell type:markdown id: tags:
# Welcome to the Jupyter@JSC Virtual Machine JupyterLab
# Welcome to the Jupyter-JSC Virtual Machine JupyterLab
%% Cell type:markdown id: tags:
---
## Where is this JupyterLab running?
%% Cell type:markdown id: tags:
      This JupyterLab instance is running on a [virtual machine](https://www.techopedia.com/definition/4805/virtual-machine-vm) on the [HDF-Cloud](https://www.fz-juelich.de/ias/jsc/EN/Expertise/SciCloudServices/HDFCloud/_node.html). It is started as a [Docker Container](https://www.docker.com/resources/what-container).
%% Cell type:markdown id: tags:
## Is there any limitation?
%% Cell type:markdown id: tags:
      You can use a maximum of 4 GB of RAM.
      If you need more memory please start your JupyterLab on a HPC system.
%% Cell type:markdown id: tags:
## How do I stop this JupyterLab?
%% Cell type:markdown id: tags:
      You can stop this JupyterLab in the Control Panel of Jupyter@JSC.
      You can reach the Control Panel [here](https://zam10116.zam.kfa-juelich.de/hub/start) or in the menu File -> Hub Control Panel.
%% Cell type:markdown id: tags:
## Why is the memory usage so high?
%% Cell type:markdown id: tags:
      If you have the feeling that your used Memory in the top right corner is too high, ensure that your kernels were shutdown.
      Whenever a Tab of your Notebook is closed, the kernel is still running in the background.
      You can see all running kernels by clicking on this symbol ![Kernels](https://jupyter-jsc.fz-juelich.de/hub/static/images/kernels.png "kernels") on the left sidebar.
      You can stop this JupyterLab in the Control Panel of Jupyter-JSC.
      You can reach the Control Panel [here](https://jupyter-jsc.fz-juelich.de/hub/start) or in the menu File -> Hub Control Panel.
%% Cell type:markdown id: tags:
## How can I upload files to JupyterLab?
%% Cell type:markdown id: tags:
      You can use git or you upload files with this button in the top left corner:
      ![Upload files](https://zam10183.zam.kfa-juelich.de/hub/static/images/upload.png "Upload Files")
      ![Upload files](https://jupyter-jsc.fz-juelich.de/hub/static/images/upload.png "Upload Files")
%% Cell type:markdown id: tags:
## How can I download a file from JupyterLab?
%% Cell type:markdown id: tags:
      Just right click on a file and click "Download".
%% Cell type:markdown id: tags:
## What kind of Docker Images do we support?
%% Cell type:markdown id: tags:
      You can select between [these eight](https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-base-notebook) Images.
      We installed additionally a few JupyterLab [extensions](https://nbviewer.jupyter.org/github/kreuzert/Jupyter-JSC/blob/master/Extensions.ipynb).
      We installed additionally a few JupyterLab extensions. You can find a List of all installed extensions [here.](https://nbviewer.jupyter.org/github/kreuzert/Jupyter-JSC/blob/master/Extensions.ipynb)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you want to add your own Docker Image to Jupyter@JSC please contact the <a href="mailto:ds-support@fz-juelich.de?subject=Jupyter@JSC Support&amp;body=Please describe your problem here. (english or german)">Jupyter@JSC support</a>.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;We installed additionally a few JupyterLab extensions.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you want to add your own Docker Image to Jupyter-JSC please contact the <a href="mailto:ds-support@fz-juelich.de?subject=Jupyter-JSC Support&amp;body=Please describe your problem here. (english or german)">Jupyter-JSC support</a>.
%% Cell type:markdown id: tags:
## How much disk space do I have?
%% Cell type:markdown id: tags:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You can store 10 GB of data in /home/jovyan/work.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<font color='red'>All files in the ~/work directory will be stored persistently.</font>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<font color='red'>All files in the ~/work directory will be accessible in all of your HDF-Cloud JupyterLabs.</font>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<font color='red'>Any other files or directories will be deleted if you stop this JupyterLab.</font>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Under no circumstances are we liable for any lost data.
%% Cell type:markdown id: tags:
## How much memory do I have?
%% Cell type:markdown id: tags:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;These JupyterLabs are limited to 4GB Ram.
%% Cell type:markdown id: tags:
## Collaborative work
# Collaborative work
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;We offer four different solutions to share your work with your colleagues.
%% Cell type:markdown id: tags:
### GIT
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You can use the git command in a terminal to work on any git repositories.
%% Cell type:markdown id: tags:
### Projects
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Projects are the best way, if you just want to share a few files (or notebooks) with your colleague, who is also using Jupyter@JSC.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Projects are the best way, if you just want to share a few files (or notebooks) with your colleague, that is also using Jupyter-JSC.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Projects are directories, that are generated on the [virtual machine](https://www.techopedia.com/definition/4805/virtual-machine-vm) where your JupyterLab is running.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you add another user to your project this directory will be mounted into their [Docker Container](https://www.docker.com/resources/what-container) .
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You can manage your projects with the following command:
```
$ bash /home/jovyan/manage_projects.sh
```
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You can store up to 10 GB of data on all your projects combined.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you want to add users to your project you need their email address. It must be the same email address, that is connected to their [JSC webservice credentials](https://judoor.fz-juelich.de/login) .
%% Cell type:markdown id: tags:
### B2DROP
#### B2DROP
%% Cell type:markdown id: tags:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You can easily mount your [B2DROP](https://b2drop.eudat.eu) files into this JupyterLab.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Just run the command "$ mount B2DROP" in a terminal and insert your [application credentials](https://eudat.eu/services/userdoc/b2drop#UserDocumentation-B2DROPUsage-WebDavclient)\.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Just run the command `mount B2DROP` in a terminal and insert your [application credentials](https://eudat.eu/services/userdoc/b2drop#UserDocumentation-B2DROPUsage-WebDavclient)\.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you want to store your [application credentials](https://eudat.eu/services/userdoc/b2drop#UserDocumentation-B2DROPUsage-WebDavclient) add the following line to /home/jovyan/work/.davfs2/secrets:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;https://b2drop.eudat.eu/remote.php/webdav \<USERNAME> \<PASSWORD>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;To unmount it run `umount B2DROP`
%% Cell type:markdown id: tags:
### HPCMOUNT (experimental)
%% Cell type:markdown id: tags:
<span style="color:red">Your groups are not mounted. So you have only access to files that are yours.</span>
### HPCMOUNT
%% Cell type:markdown id: tags:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You can mount your files on the HPC system with [sshfs](https://en.wikipedia.org/wiki/SSHFS) .
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;For this we created the script at /home/jovyan/mount_hpc.sh .
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;To mount your HPC files just execute
```
$ bash /home/jovyan/mount_hpc.sh
```
### &nbsp;&nbsp;&nbsp; What will the script do?
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you run it for the first time, it will create a private/public key pair for you. For your safety, a passphrase is required for this key.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Afterward, it will upload the public key to your [JUDAC](https://www.fz-juelich.de/ias/jsc/EN/Expertise/Datamanagement/JUDAC/JUDAC_node.html) ~/.ssh/authorized_keys file. This will be done with [UNICORE](https://www.unicore.eu) and the [credentials](https://judoor.fz-juelich.de/login) you used to log in to Jupyter@JSC.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Afterward, it will upload the public key to your [JUDAC](https://www.fz-juelich.de/ias/jsc/EN/Expertise/Datamanagement/JUDAC/JUDAC_node.html) ~/.ssh/authorized_keys file. This will be done with [UNICORE](https://www.unicore.eu) and the [credentials](https://judoor.fz-juelich.de/login) you used to log in to Jupyter-JSC.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If your key is uploaded (or you have already used this script before) it will run the sshfs command to mount your HPC files.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;You have to enter your passphrase, that you chose in the first run, to use your generated private key.
### &nbsp;&nbsp;&nbsp; How can I unmount it?
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Just run the command
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;```$ fusermount -u /home/jovyan/HPCMOUNT```
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The folder will be automatically unmounted if you stop this JupyterLab.
### &nbsp;&nbsp;&nbsp; Where is my private key stored?
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The private key will be stored, like your other files, in the /home/jovyan/work directory, on a [virtual machine](https://www.techopedia.com/definition/4805/virtual-machine-vm) on the [HDF-Cloud](https://www.fz-juelich.de/ias/jsc/EN/Expertise/SciCloudServices/HDFCloud/_node.html) .
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The access to this virtual machine is limited to the administrators of Jupyter@JSC.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The access to this virtual machine is limited to the administrators of Jupyter-JSC.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Since your private key is encrypted with a passphrase, even an administrator is not able to use your private key to connect to any HPC system.
### &nbsp;&nbsp;&nbsp; I want to mount another account
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If you have multiple accounts (connected to your [JSC webservice credentials](https://judoor.fz-juelich.de/login)) you can tell the script which one it should use.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;```$ bash /home/jovyan/mount_hpc.sh <SYSTEM> <ACCOUNT>```
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\<SYSTEM> is the HPC system that is used to upload your public key. Please use one of these: JURECA, JUWELS, JURON. Your account must have an active project on the chosen system.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\<ACCOUNT> is your HPC account on this system ( and on [JUDAC](https://www.fz-juelich.de/ias/jsc/EN/Expertise/Datamanagement/JUDAC/JUDAC_node.html)) .
### &nbsp;&nbsp;&nbsp; I want to delete the public key from my HPC account
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Jupyter@JSC does not delete anything from your authorized_keys file. So if you want to remove the public key you have to log in to JUDAC and remove it manually.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Jupyter-JSC does not delete anything from your authorized_keys file. So if you want to remove the public key you have to log in to JUDAC and remove it manually.
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment