Welcome to the IBG-3 specific introduction for getting started with the JSC supercomputers.
This document is intended to complement the general documentation of JURECA and JUWELS by providing specific guidance tailored for IBG-3 users, helping you navigate common pitfalls and optimize your computing experience.
Feel free to correct/adapt this document!
Content:
-[Joining a compute time project](#Joining-a-compute-time-project) for first-time-users
-[Tips for first-time users](#Tips-for-first-time-users)
-[Data handling](#Data-handling): best practices for members of computer projects `cjicg41` (JURECA), `cjibg36` (JUWELS)
Useful links:
- Introduction to Supercomputing at JSC: https://www.fz-juelich.de/en/ias/jsc/education/training-courses/training-materials/course-material-sc-introduction-may-23
- Follow the instructions in the JURECA manual (https://apps.fz-juelich.de/jsc/hps/jureca/access.html)to generate an SSH
key.
4. Add SSH Key to JuDoor:
- Specify your SSH key on JuDoor as per the manual’s instructions.
- Use IP-Adresses of the form `x.y.0.0/16` in the "from" clause, where the first
two numbers "x" and "y" should be copied from the ip-adress displayed in the top
of the JuDoor SSH-key-upload-page. Background: The last two numbers of the
ip-adress may vary between logins and the suffix `0.0/16` ensures that the from-clause stays valid for these variations.
5. Contact for Assistance:
- If you encounter login issues or have questions about using the
system, feel free to contact for assistance.
- Johannes: jo.keller@fz-juelich.de
- JSC general support contacts: https://www.fz-juelich.de/en/ias/jsc/services/user-support
## Tips for first time users
As a first-time user, familiarize yourself with the following helpful commands and configurations to enhance your experience:
-`TAB` for command completion (and f.e. to quickly check if a command or file exists on the system).
-`Ctrl-r` to search through command history efficiently, facilitating the repetition of commands.
-`Ctrl-c` to cancel the execution of a command if needed.
If you find yourself stuck in a shell-opened program, try the following steps to exit the program:
1. Press `Ctrl-c`.
2. Press `Esc`.
3. Type `q`.
4. Type `:q`.
5. Type `:!q`.
6. Press `Ctrl-x` followed by `Ctrl-c`.
---
Every login starts with a clear environment. This means for every new connection:
- The `jutil` command - that sets various useful variables like `$SCRATCH` - is reset and has to be re-activated `jutil env activate -p <project>`.
- The default modules are loaded. For example, any `source loadenvs.Intel` commands or similar have to be repeated to load a specific module list.
- Any previously set variables e.g. `export VARIABLE=VALUE` are no longer set.
On the other hand, changes to `$HOME/.bashrc` will load automatically at login.
---
Add the following lines to `$HOME/.bashrc` to enhance your command line environment. They define a longer history and some nice alias commands (feel free to add to the list):
```shell=
export HISTFILE=~/.bash_history
export SAVEHIST=2000
export HISTSIZE=1000
# Aliases
alias ls='ls --color -o -h '
alias l='ls'
alias less='less -N'
alias dirs='dirs -v'
alias ..='cd ..'
alias ...='cd ../..'
alias ....='cd ../../..'
alias open='chmod go+r'
alias shut='chmod go-r'
alias myq="watch squeue -u $USER"
```
---
Add the following line to `$HOME/.bash_profile` to display more information in the command line prompt.
Create a file named `$HOME/.gitconfig` with the following lines and adapt your name and email (ensure this information is suitable for public use). These configurations provide shortcuts for common Git commands:
- JSC, available file systems: https://apps.fz-juelich.de/jsc/hps/juwels/environment.html#available-file-systems
Next to your `$HOME` directory, you can add a personal directory on each file system (example for JUWELS, replace project code for other projects):
- compute project
-`mkdir $PROJECT_cjibg36/$USER`
-`mkdir $SCRATCH_cjibg36/$USER`
- data project
-`mkdir $DATA_jibg36/$USER`
-`mkdir $ARCHIVE_jibg36/$USER`
### File Systems and How to Use Them
For detailed information about the JSC file systems, refer to https://apps.fz-juelich.de/jsc/hps/juwels/environment.html#available-file-systems.
We add some more specific information for the file systems used in our projects.
#### $HOME
- Minimal usage with very few files.
- Reserved for user program configuration files.
#### $PROJECT
- Regular backups available, but limited memory and file count.
- Store source codes, binaries, and important files.
- Avoid storing full simulation directories; include only necessary files for reproducing runs (e.g., scripts for data download, excluding actual data).
#### $SCRATCH
- No backups provided, but larger space compared to $PROJECT.
- Files are automatically removed after 3 months.
- Utilize for active simulations; move relevant outputs and reproduction-related files to `$PROJECT/$DATA/$ARCHIVE` after completion.
#### $DATA
- Few backups, offering ample storage space.
- Suitable for storing input files, large output files, and forcing data.
#### $ARCHIVE
- No quick file access, large storage capacity, limited file count.
- Archive files using "tar-ball" compression before moving here.
- Ideal for archiving significant simulations or those related to publications.
### Moving data to archive
**Remark:** The same procedure applies with `jibg36` for JUWELS users instead of `jicg41`.
1.**Apply for Access**: Request access to the data project `jicg41` on JuDoor by clicking the `Join a project` button and entering `jicg41` (without the `c`).
2.**Create Directory**: Once approved, create a directory named after your username under `$ARCHIVE_jicg41` using the following commands:
```shell
cd$ARCHIVE_jicg41
mkdir$USER
```
3.**Archive Files**: Before transfering files from `$PROJECT_cjicg41` to `$ARCHIVE_jicg41`, archive these files using `tar`:
```shell
cd$PROJECT_cjicg41/$USER
```
- Create tar-balls for the files (no compression):
```
tar -cvf NAME.tar PATH/TO/DIR/FILES
```
- Optionally, compress tar-balls (e.g., gzip):
```
tar -czvf NAME.tar.gz PATH/TO/DIR/FILES
```
4.**Move Files**: After archiving, move the tar-balls:
