Skip to content
Snippets Groups Projects
Commit e393b634 authored by Mehdi Cherti's avatar Mehdi Cherti :speech_balloon:
Browse files

template challenge first commit

parents
No related branches found
No related tags found
No related merge requests found
README.md 0 → 100644
# Template Challenge
This is a template challenge. If you would like to create a new challenge
at Jülich Callenges, please follow the steps. All the steps are necessary.
## 1) Clone the repository
- `git clone https://gitlab.version.fz-juelich.de/MLDL_FZJ/juhaicu/jsc_internal/superhaicu/vouchers/j-lich-challenges/challenges/template_challenge my_challenge`
- `cd my_challenge`
## 2) Download Raw data
You first need to download the raw data if needed, it does not matter where you put as long as it is acccessible, for instance
yon can put it in the folder `raw/` in `my_challenge`.
## 3) Build annotation files
You need to create annotation splits.
Please check `annotations/generate.py` for more details.
Basically, the script `annotations/generate.py` should be edited
so that it creates annotation files (train.csv, valid.csv, test.csv, submission_valid.csv, submission_test.csv)
based on raw data. The splits can be generated randomly or follow the splits you already
have in your raw data.
Note that The CSVs should not be heavy, it should not contain images themselves
if your challenge have some. If your challenge have images, you can have a column
pointing to filenames of the images.
Once you finish editing `annotations/generate.py`, please run it:
`cd annotations;python generate.py`
you will have now the following files in the folder `annotations/`:
- `annotations/train.csv`
- `annotations/valid.csv`
- `annotations/test.csv`
- `annotations/submission_valid.csv`
- `annotations/submission_test.csv`
## 4) Edit the evaluation script
This is one of the most important steps because in this step
you define the metrics that will be used to rank the participants
in the leaderboard.
Please edit `evaluation_script/main.py` for your challenge.
It comes with an example where the accuracy metric is used
for a classification problem. But you can use any set of metrics.
It is a regular python file, so any metric which can be written
in Python could be used. You can use `Scikit-Learn` which already
offers most of the usual metrics.
Basically, the script `evaluation_script/main.py` returns a Python dictionary
containing the metric names as keys and the metric values as values.
You can use as many metrics as you want, the metric that will be used
to rank the participants in the leaderboard will be chosen later in `challenge_config.yaml`.
Here is just one note about how `evaluation_script/main.py` works.
In order to compute the metrics, the participant submission and the annotation files that you generated
previously are opened (e.g., valid.csv or test.csv depending on the challenge phase)
in `evaluation_script/main.py`. The two (participant submission and annotation file) are then compared using the column
that corresponds to the label (e.g., the column `label` in this template challenge).
## 5) Edit challenge_config.yaml
Please dit `challenge_config.yaml` and change everything which is specific to your challenge.
More documentation about the fields is given at <https://evalai.readthedocs.io/en/latest/configuration.html>
The most important parts to modify are the fields in the top of the file, they contain basic information about the challenge.
Plase also chalenge the leaderboard section. In the leaderboard section, please put all the metric names you want to display in the leaderboard.
The metric names correspond to the metric names you defined in `evaluation_script/main.py`.
`default_order_by` defines the metric you use to rank the participants.
Please check carefully all the fields so that it correspond to your needs for the challenge,
such as the starting date and ending date of the different phases and the number of submissions.
## 6) Edit the template files and the logo
Please edit the HTML files in the folder `templates/`. These HTML files
are displayed in the frontend in your challenge page.
Please also modify the logo file `logo.jpg` to have a custom logo for your challenge,
this will displayed in the frontend as well.
## 7) Edit notebook.ipynb
Please edit `notebook.ipynb`. This part is important for the participants. This is given to the participants so that they can
have the challlenge introduced to them and be able to easily understand the problem and start to submit easily.
Steps such as downloading the data, explanation of the problem, exploratory data analysis, baseline solutions, and example submissions
should be included.
## 8) Edit run.sh
Please edit `run.sh` if needed. This script will make all the archives needed for the challenge. More details are given in `run.sh`.
Here is a short description of the files that `run.sh` need to create:
- `challenge_config.zip`. This archive contains the challenge configuration, this file will be uploaded at the frontend by yourself
to create the challenge
- `data_public_leaderboard_phase.zip`. This archive is given to the users for the public leaderboard phase, so you need to upload
it somehwere in a public link.
- `data_private_leaderboard_phase.zip`. This archive is given to the users for the private leaderboard phase, so you need to upload
it somehwere in a public link.
`run.sh` should work as is, but if you have any a additional files to included in the data archives (such as images), you need to change it.
When `run.sh` is ready, please run it:
`./run.sh`
After running the script, you should have these files:
- `challenge_config.zip`
- `data_public_leaderboard_phase.zip`
- `data_private_leaderboard_phase.zip`
Please then upload the data archives in a public link, and reference the links
in the HTML files in `templates/` as well as in the `notebook.ipynb`
## 9) Upload `challenge_config.zip` to the frontend
- Go to the frontend
- Create a new challenge, then upload `challenge_config.zip`.
This is the last step. One of the administrators should then accept your challenge so that
it becomes public.
Thanks for following this guide, your challenge should be ready now!
import uuid
import random
import pandas as pd
# Please modify this script
# This script should take raw data you have
# and generate the data splits in a CSV file:
# train.csv, valid.csv, test.csv
# - train.csv would be given to the participants
# - valid.csv is the public leaderboard data with the **labels**, so it will not be given to the participants.
# - submission_valid.csv is the same as valid.csv but the labels column contains some dummy values,
# this will be given to the participants as a dummy submission for the public leaderboard phase
# - test.csv is the private leaderboard data with the **labels"", so it will not be given to the participants.
# - submission_test.csv is the same as test.csv, the difference is that the labels column contains dummy values,
# this will be given to the participants as a dummy subbmission for the private leaderboard phase
# WARNING: in the following script we just generate the CSVs randomly, please edit this script
def generate_dummy(nb):
names = [uuid.uuid4() for _ in range(nb)]
labels = [random.choice(("label1", "label2", "label3")) for _ in range(nb)]
df = pd.DataFrame({"name": names, "label": labels})
return df
nb_train = 10000
nb_valid = 1000
nb_test = 1000
train = generate_dummy(nb_train)
valid = generate_dummy(nb_valid)
test = generate_dummy(nb_test)
train.to_csv("train.csv", index=False)
valid.to_csv("valid.csv", index=False)
test.to_csv("test.csv", index=False)
valid["label"] = ""
valid.to_csv("submission_valid.csv", index=False)
test["label"] = ""
test.to_csv("submission_test.csv", index=False)
# If you are not sure what all these fields mean, please refer our documentation here:
# https://evalai.readthedocs.io/en/latest/configuration.html
title: Template Challenge
short_description: Predict XXX from XXX
description: templates/description.html
evaluation_details: templates/evaluation_details.html
terms_and_conditions: templates/terms_and_conditions.html
image: logo.jpg
submission_guidelines: templates/submission_guidelines.html
leaderboard_description: We use the XXX metric for evaluation
evaluation_script: evaluation_script.zip
remote_evaluation: False
is_docker_based: False
start_date: 2019-01-01 00:00:00
end_date: 2099-05-31 23:59:59
published: True
leaderboard:
- id: 1
schema:
{
"labels": ["accuracy"],
"default_order_by": "accuracy",
}
challenge_phases:
- id: 1
name: Dev Phase
description: templates/challenge_phase_1_description.html
leaderboard_public: True
is_public: True
is_submission_public: True
start_date: 2019-01-19 00:00:00
end_date: 2099-04-25 23:59:59
test_annotation_file: annotations/valid.csv
codename: dev
max_submissions_per_day: 100
max_submissions_per_month: 50
max_submissions: 50
submission_meta_attributes: null
is_restricted_to_select_one_submission: False
is_partial_submission_evaluation_enabled: False
- id: 2
name: Test Phase
description: templates/challenge_phase_2_description.html
leaderboard_public: False
is_public: False
is_submission_public: False
start_date: 2019-01-19 00:00:00
end_date: 2099-04-25 23:59:59
test_annotation_file: annotations/test.csv
codename: test
max_submissions_per_day: 100
max_submissions_per_month: 50
max_submissions: 50
submission_meta_attributes: null
is_restricted_to_select_one_submission: True
is_partial_submission_evaluation_enabled: False
dataset_splits:
- id: 1
name: Valid Split
codename: dev
- id: 2
name: Test Split
codename: test
challenge_phase_splits:
- challenge_phase_id: 1
leaderboard_id: 1
dataset_split_id: 1
visibility: 1
leaderboard_decimal_precision: 3
- challenge_phase_id: 2
leaderboard_id: 1
dataset_split_id: 2
visibility: 1
leaderboard_decimal_precision: 3
is_leaderboard_order_descending: true
logo.jpg 0 → 100644
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd" >
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en" >
<head>
<base href="https://www.fz-juelich.de/"></base>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" /> <title>Forschungszentrum Jülich - Hidden - FZJ Logo</title>
<meta http-equiv="Content-Style-Type" content="text/css" />
<meta http-equiv="Content-Script-Type" content="text/javascript" />
<meta http-equiv="imagetoolbar" content="false" />
<meta name="MSSmartTagsPreventParsing" content="true" />
<meta name="genTime" content="Fri Oct 02 10:50:09 CEST 2020"/>
<meta name="generator" content="Government Site Builder 4.0"/>
<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:site" content="@fz_juelich" />
<link rel="bookmark" href="SharedDocs/Bilder/INM/INM-1/EN/FZj_Logo.html;jsessionid=4B925518FD147504BBE18536E5D01851?nn=534374#Start" type="text/html" title="Zum Inhalt" /><!-- NOTE: points to the beginning of the document -->
<link rel="copyright" href="portal/DE/Service/Impressum/impressum_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" type="text/html" title="Imprint" />
<link rel="glossary" href="" type="text/html" title="Glossary" />
<link rel="help" href="" type="text/html" title="Help" />
<link rel="start" href="inm/inm-1/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" type="text/html" title="Homepage" />
<link rel="contents" href="portal/DE/Service/Sitemap/sitemap_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" type="text/html" title="Sitemap" />
<link rel="search" href="portal/DE/Service/Suche/suche_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" type="text/html" title="Search" />
<link rel="up" href="inm/inm-1/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" type="text/html" title="Strukturelle und funktionelle Organisation des Gehirns (INM-1)" />
<link rel="chapter" href="inm/inm-1/DE/Aktuelles/Meldungen/weitereMeldungen_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" type="text/html" title="Meldungen" />
<link rel="chapter" href="inm/inm-1/DE/Home/_service/service_hidden_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" type="text/html" title="" />
<link rel="chapter" href="inm/inm-1/DE/Home/bottomservice_hidden_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" type="text/html" title="Homepage" />
<link rel="chapter" href="inm/inm-1/DE/Home/topservice_hidden_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" type="text/html" title="Homepage" />
<link rel="shortcut icon" href="/SiteGlobals/StyleBundles/Bilder/favicon.jpg;jsessionid=4B925518FD147504BBE18536E5D01851?__blob=normal" type="image/ico" />
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/visual/visual.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=4" type="text/css" media="print, projection, screen" />
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/screen/screen-a.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=7" type="text/css" media="projection, screen" />
<!-- Additional IE/Win specific style sheet (Conditional Comments) --><!--[if lte IE 7]><link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/screen/screen_iew.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=2" type="text/css" media="projection, screen" /><![endif]-->
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/screen/startseite2.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=8" type="text/css" media ="projection, screen" />
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/print/print.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=2" type="text/css" media="print" />
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/screen/kalender.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=7" type="text/css" media ="projection, screen" />
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/screen/HintergrundInstitute/hintergrundbild-allgemein.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=5" type="text/css" media ="projection, screen" />
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/screen/mobile.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=8" type="text/css" media="screen" />
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/visual/visual.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=4" type="text/css" media="print, projection, screen" />
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/screen/screen-a.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=7" type="text/css" media="projection, screen" />
<!-- Additional IE/Win specific style sheet (Conditional Comments) --><!--[if lte IE 7]><link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/screen/screen_iew.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=2" type="text/css" media="projection, screen" /><![endif]-->
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/screen/startseite2.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=8" type="text/css" media ="projection, screen" />
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/print/print.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=2" type="text/css" media="print" />
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/screen/kalender.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=7" type="text/css" media ="projection, screen" />
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/screen/HintergrundInstitute/hintergrundbild-allgemein.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=5" type="text/css" media ="projection, screen" />
<link rel="stylesheet" href="SiteGlobals/StyleBundles/CSS/screen/mobile.css;jsessionid=4B925518FD147504BBE18536E5D01851?v=8" type="text/css" media="screen" />
<script type="text/javascript" src="https://apps.fz-juelich.de/common/MathJax/current/MathJax.js?config=TeX-MML-AM_CHTML&noContrib" async>
MathJax.Hub.Config({
TeX: {
extensions: ["mhchem.js"]
}
});
</script>
</head>
<body class="gsb mitHeader">
<div id="wrapperBranding">
<div id="branding_1" class="outer">
<div id="brandingInnen">
<div id="searchTop">
<h2 class="aural"><a id="Suche" name="Suche">Search</a></h2>
<form name="ServicebereichSuche" action="SiteGlobals/Forms/Suche/Servicesuche_Formular.html;jsessionid=4B925518FD147504BBE18536E5D01851" method="get" enctype="application/x-www-form-urlencoded">
<input type="hidden" name="nn" value="534374"/>
<input type="hidden" name="resourceId" value="12694" />
<input type="hidden" name="input_" value="2415824" />
<input type="hidden" name="pageLocale" value="en" />
<p>
<label class="aural" for="f12694d11524">Search term</label>
<input id="f12694d11524" name="templateQueryString" value="Suchbegriff" alt="Search term" type="text" size="26" maxlength="100"/>
<input class="image" type="image" src="/SiteGlobals/Forms/_components/Buttons/Servicesuche_Submit.gif;jsessionid=4B925518FD147504BBE18536E5D01851?__blob=image" id="f12694d12724" name="submit" alt="Search" title="Search" />
</p>
</form>
</div><!-- #search -->
<p><a href="portal/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="link to homepage"><img src="/SiteGlobals/StyleBundles/Bilder/NeuesLayout/logo.jpg;jsessionid=4B925518FD147504BBE18536E5D01851?__blob=normal" alt="link to homepage" /></a></p>
<div id="navServiceMeta">
<h2>Servicemeu</h2>
<ul><li ><a title="Zum deutschen Auftritt" class="languageLink lang_de" href="portal/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" xml:lang="de" hreflang="de" lang="de">Deutsch</a>&nbsp;|&nbsp;</li><li ><span title="Switch to English (not active)" class="languageLink lang_en" xml:lang="en" lang="en">English</span></li><li class="servicesearch">
<a data-toggle="servicesearch-desktop" href="SiteGlobals/Forms/Suche/Servicesuche_Formular.html;jsessionid=4B925518FD147504BBE18536E5D01851?nn=534374" class="servicesearch__trigger"
title="open search" aria-label="open search"
data-label-inactive="open search" data-label-active="close search">
</a>
<div class="servicesearch__formcontainer" id="servicesearch-desktop" data-toggler="servicesearch__formcontainer--opened">
<div class="servicesearch__form">
<h2 id="Suche" class="aural">Search</h2>
<form name="ServicebereichSuche" action="SiteGlobals/Forms/Suche/Servicesuche_Formular.html;jsessionid=4B925518FD147504BBE18536E5D01851" method="get" enctype="application/x-www-form-urlencoded">
<input type="hidden" name="nn" value="534374"/>
<input type="hidden" name="resourceId" value="12694" />
<input type="hidden" name="input_" value="2415824" />
<input type="hidden" name="pageLocale" value="en" />
<p>
<label class="aural" for="f12694d11524">Search term</label>
<input id="f12694d11524" name="templateQueryString" value="Suchbegriff" alt="Search term" type="text" size="26" maxlength="100"/>
<input class="image" type="image" src="/SiteGlobals/Forms/_components/Buttons/Servicesuche_Submit.gif;jsessionid=4B925518FD147504BBE18536E5D01851?__blob=image" id="f12694d12724" name="submit" alt="Search" title="Search" />
</p>
</form></div>
</div>
</li>
</ul>
</div><!-- #navServiceMeta -->
<div id="instNaviLinkWrapper"><a href="inm/inm-1/DE/Home/hidden_node.html;jsessionid=4B925518FD147504BBE18536E5D01851#instNaviContentLink" id="instNaviLink">Institutes</a></div>
<div id="instHeaders">
<div class="bereich1">Institut für Neurowissenschaften und Medizin</div>
<div class="bereich2"><a href="inm/inm-1/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="ZumInstitutsbereich"><strong> Strukturelle und funktionelle Organisation des Gehirns (INM-1) </strong></a></div></div>
<div id="navPrimary">
<h2> Main Menu</h2>
<ul>
<li><a href="inm/inm-1/DE/Aktuelles/aktuelles_node.html;jsessionid=4B925518FD147504BBE18536E5D01851">Aktuelles</a></li>
<li><a href="inm/inm-1/DE/Forschung/forschung_node.html;jsessionid=4B925518FD147504BBE18536E5D01851">Forschung</a></li>
<li><a href="inm/inm-1/DE/Leistungen/leistungen_node.html;jsessionid=4B925518FD147504BBE18536E5D01851">Leistungen</a></li>
<li><a href="inm/inm-1/DE/Karriere/karriere_node.html;jsessionid=4B925518FD147504BBE18536E5D01851">Karriere</a></li>
<li><a href="inm/inm-1/DE/UeberUns/ueberUns_node.html;jsessionid=4B925518FD147504BBE18536E5D01851">Über uns</a></li>
</ul>
</div><!-- #navPrimary -->
</div><!-- #brandingInnen -->
</div><!-- #branding_1 -->
</div><!-- #wrapperBranding -->
<div id="wrapperOuter" >
<img src="/SiteGlobals/StyleBundles/Bilder/KeyVisual/KV_INM.png;jsessionid=4B925518FD147504BBE18536E5D01851?__blob=normal" title="(leer)" alt="(leer)" class="headerIdentImg" />
<div id="wrapperInner">
<a id="Start" name="Start"></a>
<h1 class="navSkip">Navigation and service</h1>
<p class="navSkip"><em>Go to:</em></p>
<ul class="navSkip">
<li><a href="SharedDocs/Bilder/INM/INM-1/EN/FZj_Logo.html;jsessionid=4B925518FD147504BBE18536E5D01851?nn=534374#Inhalt">Content</a></li>
<li><a href="SharedDocs/Bilder/INM/INM-1/EN/FZj_Logo.html;jsessionid=4B925518FD147504BBE18536E5D01851?nn=534374#navPrimary">Main Menu</a></li>
<li><a href="SharedDocs/Bilder/INM/INM-1/EN/FZj_Logo.html;jsessionid=4B925518FD147504BBE18536E5D01851?nn=534374#Suche">Search</a></li>
</ul><!-- .navSkip -->
<div id="navBreadcrumbs">
<h2 class="aural">You are here:</h2>
<ol>
<li><a class="home" href="inm/inm-1/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851"><span><span>INM-1</span></span></a></li>
<li><strong><span><span>FZJ Logo</span></span></strong></li></ol>
<div class="clear"></div>
</div><!-- #navBreadcrumbs -->
<hr />
<div id="wrapperDivisions" class="modgrid">
<div id="navSecondary">
<h2 class="aural"><a id="Bereichsmenu" name="Bereichsmenu">Area Menu</a></h2>
<div class="navMain">
<h3> </h3></div>
<!-- #search -->
<div id="navService">
<h2>Service</h2>
<ul>
<li id="navServiceAnsprechpartner"><a href="inm/inm-1/DE/UeberUns/Ansprechpartner/_node.html;jsessionid=4B925518FD147504BBE18536E5D01851">Ansprechpartner</a></li>
<li id="navServiceMitarbeiter"><a href="inm/inm-1/DE/UeberUns/Mitarbeiter/mitarbeiter_node.html;jsessionid=4B925518FD147504BBE18536E5D01851">Mitarbeiter</a></li>
<li id="navServicePublikationen"><a href="inm/inm-1/DE/Service/Publikationslisten/publikationslisten_node.html;jsessionid=4B925518FD147504BBE18536E5D01851">Publikationen</a></li>
<li id="navServiceAnfahrt"><a href="inm/inm-1/DE/UeberUns/Anfahrt/anfahrt_node.html;jsessionid=4B925518FD147504BBE18536E5D01851">Anfahrt</a></li>
<li id="navServiceDownloads"><a href="inm/inm-1/DE/Service/Download/download_node.html;jsessionid=4B925518FD147504BBE18536E5D01851">Downloads</a></li>
</ul>
</div><!-- #navService -->
</div><!-- #navSecondary -->
<div id="wrapperContent">
<div id="content">
<a id="Inhalt" name="Inhalt"></a>
<div >
<h1 class="isFirstInSlot">FZJ Logo</h1>
<p class="illustration"><a href="http://www.fz-juelich.de/portal/DE/Home/home_node.html" target="_blank" title="Opens new window"><span class="wrapper"><img src="/SharedDocs/Bilder/INM/INM-1/EN/FZj_Logo.jpg;jsessionid=4B925518FD147504BBE18536E5D01851?__blob=normal" title="refer to: Forschungszentrum Juelich (Opens new window)" alt="FZJ Logo (refer to: Forschungszentrum Juelich (Opens new window))" style="width:340px;;" /></span></a><span class="caption"></span><br />
<span class="source">Copyright:&nbsp;<cite>Forschungszentrum Juelich GmbH</cite></span><br />
</p>
<p><a href="/SharedDocs/Bilder/INM/INM-1/EN/FZj_Logo.jpg;jsessionid=4B925518FD147504BBE18536E5D01851?__blob=poster" target="_blank" class="imagedownload" title="Opens in new window">Image (jpeg, 112&nbsp;kB)</a></p>
</div>
</div><!-- #content -->
</div><!-- #wrapperContent -->
<div id="supplement" >
<h2 class="aural"><a id="Zusatzinformationen" name="Zusatzinformationen">Additional Information</a></h2>
</div><!-- #supplement -->
<div class="clear"></div>
</div><!-- #wrapperDivisions -->
</div><!-- #wrapperInner -->
<hr />
<div id="siteInfo">
<h2>Servicemeu</h2>
<div id="navServiceFooter">
<ul><li><a href="portal/DE/Service/Impressum/impressum_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="Opens new window" target="_blank">Impressum</a>&nbsp;&nbsp;</li><li><a href="portal/DE/Service/Kontakt/kontakt_node.html;jsessionid=4B925518FD147504BBE18536E5D01851?cms_docId=2415824" target="_blank" title="Opens new window">Kontakt</a>&nbsp;&nbsp;</li><li><a title="Zum deutschen Auftritt" class="languageLink lang_de" href="portal/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" xml:lang="de" hreflang="de" lang="de">Deutsch</a>&nbsp;&nbsp;</li><li><span title="Switch to English (not active)" class="languageLink lang_en" xml:lang="en" lang="en">English</span></li></ul>
<ul id="navFunctions">
<li id="navFunctionsRecommend"></li>
<li class="social"><a href="http://de.facebook.com/sharer.php?u=https%3A%2F%2Fwww.fz-juelich.de%2FSharedDocs%2FBilder%2FINM%2FINM-1%2FEN%2FFZj_Logo.html%3Fnn%3D534374" title="Facebook" target="_blank"><img src="/SiteGlobals/Functions/SocialBookmarks/facebook.png;jsessionid=4B925518FD147504BBE18536E5D01851?__blob=normal" alt="Bookmarken Sie diese Seite via facebook - öffnet ein neues Browserfenster"/></a></li>
<li class="social"><a href="http://twitter.com/home?status=https%3A%2F%2Fwww.fz-juelich.de%2FSharedDocs%2FBilder%2FINM%2FINM-1%2FEN%2FFZj_Logo.html%3Fnn%3D534374" title="twitter" target="_blank"><img src="/SiteGlobals/Functions/SocialBookmarks/twitter.png;jsessionid=4B925518FD147504BBE18536E5D01851?__blob=normal" alt="Bookmarken Sie diese Seite via twitter - öffnet ein neues Browserfenster"/></a></li>
<li class="social"><a href="https://www.researchgate.net/go.Share.html?url=https%3A%2F%2Fwww.fz-juelich.de%2FSharedDocs%2FBilder%2FINM%2FINM-1%2FEN%2FFZj_Logo.html%3Fnn%3D534374" title="ResearchGate" target="_blank"><img src="/SiteGlobals/Functions/SocialBookmarks/researchgate.png;jsessionid=4B925518FD147504BBE18536E5D01851?__blob=normal" alt="Bookmarken Sie diese Seite via Research Gate - öffnet ein neues Browserfenster"/></a></li>
</ul><!-- #navFunctions -->
<div class="clear"></div>
</div><!-- #navServiceFooter -->
<p class="logo">
<a href="http://www.helmholtz.de/en" target="_blank">
<img src="/SiteGlobals/StyleBundles/Bilder/NeuesLayout/logo_helmholtz.png;jsessionid=4B925518FD147504BBE18536E5D01851?__blob=normal" title="Helmholtz-Gemeinschaft" alt="Helmholtz-Gemeinschaft" />
</a>
</p>
</div><!-- #SiteInfo -->
<div class="clear"></div>
<div id="instNavi">
<a id="instNaviContentLink"></a>
<div class="start">
<h1>Homepage</h1>
<ul>
<li class="mod">
<a href="portal/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Portal</a>
</li>
<li class="mod"><a class="external" href="http://intranet.fz-juelich.de/portal/DE/Home/home_node.html" title="External link Intranet für Mitarbeiter des Forschungszentrums (Opens new window)" target="_blank">Intranet</a></li>
</ul>
</div>
<div class="institute">
<h1>institutes</h1>
<ul>
<li class="mod">
<a href="er-c/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Ernst Ruska-Centrum für Mikroskopie und Spektroskopie mit Elektronen (ER‑C)</a>
</li>
<li class="mod">
<a href="portal/DE/Institute/InstituteAdvancedSimulation/_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Institute for Advanced Simulation (IAS)</a>
</li>
<li class="mod">
<a href="portal/DE/Institute/InstitutBioGeowissenschaften/_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Institut für Bio- und Geowissenschaften (IBG)</a>
</li>
<li class="mod">
<a href="ibi/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Institut für Biologische Informationsverarbeitung (IBI)</a>
</li>
<li class="mod">
<a href="portal/DE/Institute/InstitutEnergieundKlima/_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Institut für Energie- und Klimaforschung (IEK)</a>
</li>
</ul>
<ul>
<li class="mod">
<a href="portal/DE/Institute/InstitutKernphysik/_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Institut für Kernphysik (IKP)</a>
</li>
<li class="mod">
<a href="inm/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="Institut für Neurowissenschaften und Medizin (INM)" class="themenLink">Institut für Neurowissenschaften und Medizin (INM)</a>
</li>
<li class="mod">
<a href="jcns/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Jülich Centre for Neutron Science (JCNS)</a>
</li>
<li class="mod">
<a href="pgi/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Peter Grünberg Institut (PGI)</a>
</li>
<li class="mod">
<a href="portal/DE/Institute/ZentralinstitutEngineering/_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Zentralinstitut für Engineering, Elektronik und Analytik (ZEA)</a>
</li>
</ul>
</div>
<div class="einrichtungen">
<h1>Administration</h1>
<ul>
<li class="mod">
<a href="gd/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="Drittmittelmanagement " class="themenLink">Drittmittelmanagement</a>
</li>
<li class="mod">
<a href="portal/DE/zentrum/ptj/_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Projektträger</a>
</li>
<li class="mod">
<a href="julab/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">JuLab</a>
</li>
<li class="mod">
<a href="zb/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Zentralbibliothek</a>
</li>
<li class="mod">
<a href="bfc/DE/Home/home_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Büro für Chancengleichheit</a>
</li>
<li class="mod">
<a href="portal/DE/zentrum/Organisation/_doc/zc_node.html;jsessionid=4B925518FD147504BBE18536E5D01851" title="" class="themenLink">Zukunftscampus (ZC)</a>
</li>
</ul>
</div>
</div>
</div> <!-- #wrapperOuter -->
<script type="text/javascript">//<![CDATA[
// i18n
var PRINT_PAGE_TEXT = 'Print';
var PRINT_TOOLTIP = 'Print (opens dialog)';
//]]></script>
<script src="SiteGlobals/Functions/JavaScript/libs/jquery.js;jsessionid=4B925518FD147504BBE18536E5D01851?view=renderPlain&amp;v=2&amp;nn=534374" type="text/javascript" ></script>
<script src="SiteGlobals/Functions/JavaScript/libs/jquery.ui.core.js;jsessionid=4B925518FD147504BBE18536E5D01851?view=renderPlain&amp;v=2&amp;nn=534374" type="text/javascript" ></script>
<script src="SiteGlobals/Functions/JavaScript/libs/jquery.ui.widget.js;jsessionid=4B925518FD147504BBE18536E5D01851?view=renderPlain&amp;v=2&amp;nn=534374" type="text/javascript" ></script>
<script src="SiteGlobals/Functions/JavaScript/rotator.js;jsessionid=4B925518FD147504BBE18536E5D01851?v=5" type="text/javascript" ></script>
<script src="SiteGlobals/Functions/JavaScript/instnav.js;jsessionid=4B925518FD147504BBE18536E5D01851?view=renderPlain&amp;v=3&amp;nn=534374" type="text/javascript" ></script>
<script src="SiteGlobals/Functions/JavaScript/img_switcher.js;jsessionid=4B925518FD147504BBE18536E5D01851?view=renderPlain&amp;v=7&amp;nn=534374" type="text/javascript" ></script>
<script src="SiteGlobals/Functions/JavaScript/calendar.js;jsessionid=4B925518FD147504BBE18536E5D01851?view=renderPlain&amp;v=6&amp;nn=534374" type="text/javascript" ></script>
<script src="SiteGlobals/Functions/JavaScript/blind.js;jsessionid=4B925518FD147504BBE18536E5D01851?view=renderPlain&amp;v=6&amp;nn=534374" type="text/javascript" ></script>
<script src="SiteGlobals/Functions/JavaScript/initjs.js;jsessionid=4B925518FD147504BBE18536E5D01851?view=renderPlain&amp;v=3&amp;nn=534374" type="text/javascript" ></script>
<script src="SiteGlobals/Functions/JavaScript/lib.js;jsessionid=4B925518FD147504BBE18536E5D01851?view=renderPlain&amp;v=3&amp;nn=534374" type="text/javascript" ></script>
<script src="SiteGlobals/Functions/JavaScript/print.js;jsessionid=4B925518FD147504BBE18536E5D01851?view=renderPlain&amp;v=3&amp;nn=534374" type="text/javascript" ></script>
<script src="SiteGlobals/Functions/JavaScript/behavior.js;jsessionid=4B925518FD147504BBE18536E5D01851?view=render&amp;v=8&amp;nn=534374" type="text/javascript" ></script>
<script src="SiteGlobals/Functions/JavaScript/mobile.js;jsessionid=4B925518FD147504BBE18536E5D01851?view=render&amp;v=6&amp;nn=534374" type="text/javascript" ></script>
<!--Realisiert mit dem Government Site Builder, der Content Management Lösung der Bundesverwaltung, www.government-site-builder.de -->
<!-- Piwik -->
<script type="text/javascript">
var _paq = _paq || [];
_paq.push(["trackPageView"]);
_paq.push(["enableLinkTracking"]);
(function() {
var u=(("https:" == document.location.protocol) ? "https" : "http") + "://apps.fz-juelich.de/piwik/";
_paq.push(["setTrackerUrl", u+"piwik.php"]);
_paq.push(["setSiteId", "5"]);
var d=document, g=d.createElement("script"), s=d.getElementsByTagName("script")[0]; g.type="text/javascript";
g.defer=true; g.async=true; g.src=u+"piwik.js"; s.parentNode.insertBefore(g,s);
})();
</script>
<noscript><img src="https://apps.fz-juelich.de/piwik/piwik.php?idsite=5&amp;rec=1" style="border:0" alt="" /></noscript>
<!-- End Piwik Code -->
</body>
</html>
\ No newline at end of file
%% Cell type:markdown id: tags:
# The XXX challenge
%% Cell type:markdown id: tags:
Please provide a logo
%% Cell type:markdown id: tags:
<img src="" alt="logo" width="400"/>
%% Cell type:markdown id: tags:
Please give a short description of your challenge
%% Cell type:markdown id: tags:
# Requirements
%% Cell type:code id: tags:
``` python
!pip install pandas torch torchvision scikit-learn # please put requirements here
```
%% Cell type:markdown id: tags:
## Donwloading the data
Please describe the steps to download the data:
%% Cell type:markdown id: tags:
The full data is around 5GB. In order to start quickly we also provide a subset of the training data here:
%% Cell type:code id: tags:
``` python
!wget XXXX
```
%% Output
--2020-10-02 08:42:27-- http://xxxx/
Resolving xxxx (xxxx)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘xxxx’
%% Cell type:code id: tags:
``` python
!ls
```
%% Output
README.md challenge_config.yaml logo.jpg run.sh templates
annotations evaluation_script notebook.ipynb tags
%% Cell type:markdown id: tags:
# Exploratory Data Analysis
Please provide different cells that explore a little bit
the data with some nice visualizations.
Possibilities:
- Class Frequencies
- Plots
- Show different examples
- Summary Statistics
- etc.
%% Cell type:code id: tags:
``` python
```
%% Cell type:code id: tags:
``` python
```
%% Cell type:code id: tags:
``` python
```
%% Cell type:code id: tags:
``` python
```
%% Cell type:code id: tags:
``` python
```
%% Cell type:code id: tags:
``` python
```
%% Cell type:markdown id: tags:
# Dummy Submission
%% Cell type:markdown id: tags:
Please provide a dummy submission.
Here a CSV submission example should be generated, no training would be involved,
it can be constant classifier model for instance (sklearn's DummyClassifier for instance).
It's the simplest example for the participants to start with.
%% Cell type:code id: tags:
``` python
# the following cells should generate submission.csv
```
%% Cell type:code id: tags:
``` python
```
%% Cell type:code id: tags:
``` python
```
%% Cell type:markdown id: tags:
%% Cell type:code id: tags:
``` python
!head submission.csv
```
%% Output
head: cannot open 'submission.csv' for reading: No such file or directory
%% Cell type:markdown id: tags:
Now, you can open the submision.csv file (File -> Open) file and download it!
After you download it, you can upload it to the frontend, here: XXX (direct link to the challenge)
%% Cell type:markdown id: tags:
# Baseline simple solution
%% Cell type:markdown id: tags:
Please provide a simple baselne submission.
Here a CSV submission example should be generated, training
will be involved but it should be very fast (matter of seconds),
and the solution should be a good starting point
%% Cell type:code id: tags:
``` python
# the following cells should generate submission.csv
```
%% Cell type:code id: tags:
``` python
```
%% Cell type:code id: tags:
``` python
```
%% Cell type:code id: tags:
``` python
```
%% Cell type:code id: tags:
``` python
```
%% Cell type:markdown id: tags:
Now, you can open the submision.csv file (File -> Open) file and download it!
After you download it, you can upload it to the frontend, here: XXX (direct link to the challenge)
run.sh 0 → 100755
#!/bin/bash
# Please edit this script
# it should create the following archives:
# 1) evaluation_script.zip (already done, no need to modify)
# 2) challenge_config.zip (arleady done, no need to modify), this is the file you upload to the challenge frontend to create a new challenge
# 3) data_public_leaderboard_phase.zip (archive to download for the users for the public leaderboard phase)
# 4) data_private_leaderboard_phase.zip (archive to download for the users for the private leaderboard phase)
# Remove already existing zip files
rm -f *.zip *.tar.gz
# Create new zip configuration according the updated code
zip -r -j evaluation_script.zip evaluation_script/* -x "*.DS_Store"
zip -r challenge_config.zip * -x "*.DS_Store" -x "evaluation_script/*" -x "*.git" -x "run.sh" -x "raw/*" -x "annotations/train/*" -x "annotations/valid/*" -x "annotations/test/*"
cd annotations
zip -r -j ../data_public_leaderboard_phase.zip train.csv train/ valid/ submission_valid.csv README.md
zip -r -j ../data_private_leaderboard_phase.zip test test.csv submission_test.csv README.md
cd ..
Please provide a description of the public leaderboard phase
Please provide a description of the private leaderboard phase
Pleaes provide:
- a description of the challenge
- link to the notebook
Please provide the details of the evaluation (metrics, etc)
Please provide guidelines for how to submit (e.g., how the submission file should be formatted)
Please provide terms and conditions to partcipate in the challenge
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment