Skip to content
Snippets Groups Projects
Commit 0d02af08 authored by Alexandre Strube's avatar Alexandre Strube
Browse files

Added Peter's slide

parent 4155e75f
No related branches found
No related tags found
No related merge requests found
Pipeline #177671 passed
Stage: test
Stage: deploy
Show whitespace changes
Inline Side-by-side
Makefile
+ 2
− 2
View file @ 0d02af08
......@@ -15,7 +15,7 @@ PANDOC_OPTIONS=-t revealjs -s \
-V hlss=zenburn \
-V theme=sky \
-V transition=fade \
-i
-i \
# --embed-resources # This make a single file, good for distribution
# -A footer.html # The footer is just too big
......
index.md
+ 17
− 6
View file @ 0d02af08
......@@ -51,11 +51,13 @@ date: February 22, 2024
- We have no models of our own (yet) deployed
- Models based on Llama2-70 🦙 take 7 gpus (or 8 with vLLM)
- VLLM: PagedAttention, batching etc. Speeds up inference at cost of gpus (no quantization)
- SGLang: RadixAttention - Even faster inference. Single-GPU for now
- Mixtral-8x7b: takes 7 gpus, but is faster and better than Llama2-70
- No data collection at all. I don't keep ANY data whatsoever
- ***No data collection at all. I don't keep ANY data whatsoever***
- You can use it AND keep your data private
- We could, there's code for ranking answers, or running models in parallel and voting
- No records? Privacy! GDPR is happy
- ![](images/datenschutz.jpg){width=100px}(I just wrote datenschutz on bing image creator)
<!-- - ![](images/datenschutz.jpg){width=100px}(I just wrote datenschutz on bing image creator) -->
---
......@@ -68,7 +70,7 @@ date: February 22, 2024
- Firewall
- Availability
- Etc
- If you have a model and want to deploy it, contact me!
- ***If you have a model and want to deploy it, contact me!***
---
......@@ -198,7 +200,7 @@ curl --header "Authorization: Bearer MY_TOKEN_GOES_HERE" https://helmholtz-bla
## Demo: VScode + Continue.dev
- Yes. It DOES run with Emacs too. Ask your favorite Emacs expert (aka Jan)
- Yes. It DOES run with Emacs too. Ask your favorite Emacs expert.
- And vim as well. I guess.
- But this demo is for VSCode. Sorry.
- Add [continue.dev](https://marketplace.visualstudio.com/items?itemName=Continue.continue) extension to VSCode
......@@ -255,6 +257,12 @@ curl --header "Authorization: Bearer MY_TOKEN_GOES_HERE" https://helmholtz-bla
## It's being used in the wild!
![https://indico.desy.de/event/38849/contributions/162118/](images/peter-steinbach-blablador-talk-lips-2024.png)
---
## It's being used in the wild!
- Someone reverse-engineered the API and created a python package
---
......@@ -318,6 +326,9 @@ curl --header "Authorization: Bearer MY_TOKEN_GOES_HERE" https://helmholtz-bla
- Run gigantic models multi-node (eg. Falcon-180B)
- Open for Helmholtz to have their models pointing to Blablador (no one asked yet)
- Multi-GPU for SGLang (For Mixtral, and other models which are too big for my GPUs)
- Multi-modal models (text+image, text+audio, etc)
- Auto-RAG with privacy: don't upload pdf, vector database in browser's ram
### Maybe
......@@ -335,9 +346,9 @@ curl --header "Authorization: Bearer MY_TOKEN_GOES_HERE" https://helmholtz-bla
## Link to these Slides
[https://strube1.pages.jsc.fz-juelich.de/2024-talk-hai-retreat-blablador](https://strube1.pages.jsc.fz-juelich.de/2024-talk-hai-retreat-blablador)
[https://strube1.pages.jsc.fz-juelich.de/2024-02-talk-lips-blablador/](https://strube1.pages.jsc.fz-juelich.de/2024-02-talk-lips-blablador/)
[Gitlab link to source code of the slides (need JUDOOR account)](https://gitlab.jsc.fz-juelich.de/strube1/2024-talk-hai-retreat-blablador/pages)
[Gitlab link to source code of the slides (need JUDOOR account)](https://gitlab.jsc.fz-juelich.de/strube1/2024-02-talk-lips-blablador/)
---
......
public/images/peter-steinbach-blablador-talk-lips-2024.png 0 → 100644
+ 0
− 0
View file @ 0d02af08
public/images/peter-steinbach-blablador-talk-lips-2024.png

1.14 MiB

public/index.html
+ 36
− 18
View file @ 0d02af08
......@@ -297,16 +297,18 @@ comes</li>
vLLM)</li>
<li class="fragment">VLLM: PagedAttention, batching etc. Speeds up
inference at cost of gpus (no quantization)</li>
<li class="fragment">SGLang: RadixAttention - Even faster inference.
Single-GPU for now</li>
<li class="fragment">Mixtral-8x7b: takes 7 gpus, but is faster and
better than Llama2-70</li>
<li class="fragment">No data collection at all. I don’t keep ANY data
whatsoever
<li class="fragment"><strong><em>No data collection at all. I don’t keep
ANY data whatsoever</em></strong>
<ul>
<li class="fragment">You can use it AND keep your data private</li>
<li class="fragment">We could, there’s code for ranking answers, or
running models in parallel and voting</li>
<li class="fragment">No records? Privacy! GDPR is happy</li>
<li class="fragment"><img data-src="images/datenschutz.jpg"
width="100" />(I just wrote datenschutz on bing image creator)</li>
<li class="fragment">No records? Privacy! GDPR is happy
<!-- - ![](images/datenschutz.jpg){width=100px}(I just wrote datenschutz on bing image creator) --></li>
</ul></li>
</ul>
</section>
......@@ -324,8 +326,8 @@ their models on their hardware and point to blablador</li>
<li class="fragment">Availability</li>
<li class="fragment">Etc</li>
</ul></li>
<li class="fragment">If you have a model and want to deploy it, contact
me!</li>
<li class="fragment"><strong><em>If you have a model and want to deploy
it, contact me!</em></strong></li>
</ul>
</section>
<section class="slide level1">
......@@ -491,7 +493,7 @@ class="sourceCode bash"><code class="sourceCode bash"><span id="cb1-1"><a href="
<h2 id="demo-vscode-continue.dev">Demo: VScode + Continue.dev</h2>
<ul>
<li class="fragment">Yes. It DOES run with Emacs too. Ask your favorite
Emacs expert (aka Jan)</li>
Emacs expert.</li>
<li class="fragment">And vim as well. I guess.</li>
<li class="fragment">But this demo is for VSCode. Sorry.</li>
<li class="fragment">Add <a
......@@ -549,6 +551,16 @@ Code to Context</li>
<section class="slide level1">
<h2 id="its-being-used-in-the-wild">It’s being used in the wild!</h2>
<figure>
<img data-src="images/peter-steinbach-blablador-talk-lips-2024.png"
alt="https://indico.desy.de/event/38849/contributions/162118/" />
<figcaption
aria-hidden="true">https://indico.desy.de/event/38849/contributions/162118/</figcaption>
</figure>
</section>
<section class="slide level1">
<h2 id="its-being-used-in-the-wild-1">It’s being used in the wild!</h2>
<ul>
<li class="fragment">Someone reverse-engineered the API and created a
python package</li>
......@@ -556,7 +568,7 @@ python package</li>
</section>
<section class="slide level1">
<h2 id="its-being-used-in-the-wild-1">It’s being used in the wild!</h2>
<h2 id="its-being-used-in-the-wild-2">It’s being used in the wild!</h2>
<figure>
<img data-src="images/python-bindings.png"
alt="https://git.geomar.de/everardo-gonzalez/blablador-python-bindings" />
......@@ -566,7 +578,7 @@ href="https://git.geomar.de/everardo-gonzalez/blablador-python-bindings">https:/
</section>
<section class="slide level1">
<h2 id="its-being-used-in-the-wild-2">It’s being used in the wild!</h2>
<h2 id="its-being-used-in-the-wild-3">It’s being used in the wild!</h2>
<ul>
<li class="fragment"><a href="https://geomar.de">GEOMAR</a> created a
chatbot for their website</li>
......@@ -582,7 +594,7 @@ href="https://git.geomar.de/marcus-krueger/tldr">Source code</a></li>
</section>
<section class="slide level1">
<h2 id="its-being-used-in-the-wild-3">It’s being used in the wild!</h2>
<h2 id="its-being-used-in-the-wild-4">It’s being used in the wild!</h2>
<figure>
<img data-src="images/geomar-blablador-slide.png"
alt="https://zenodo.org/records/10376144" />
......@@ -592,12 +604,12 @@ href="https://zenodo.org/records/10376144">https://zenodo.org/records/10376144</
</section>
<section class="slide level1">
<h2 id="its-being-used-in-the-wild-4">It’s being used in the wild!</h2>
<h2 id="its-being-used-in-the-wild-5">It’s being used in the wild!</h2>
<p><img data-src="images/conmar.png" /></p>
</section>
<section class="slide level1">
<h2 id="its-being-used-in-the-wild-5">It’s being used in the wild!</h2>
<h2 id="its-being-used-in-the-wild-6">It’s being used in the wild!</h2>
<ul>
<li class="fragment"><a href="https://eudat.eu">EUDAT</a> is a
collection of data management services</li>
......@@ -611,7 +623,7 @@ width="400" /></li>
</section>
<section class="slide level1">
<h2 id="its-being-used-in-the-wild-6">It’s being used in the wild!</h2>
<h2 id="its-being-used-in-the-wild-7">It’s being used in the wild!</h2>
<ul>
<li class="fragment">NextCloud-Blablador-Theme: <a
href="https://github.com/mwinkens/assistant">https://github.com/mwinkens/assistant</a></li>
......@@ -619,12 +631,12 @@ href="https://github.com/mwinkens/assistant">https://github.com/mwinkens/assista
</section>
<section class="slide level1">
<h2 id="its-being-used-in-the-wild-7">It’s being used in the wild!</h2>
<h2 id="its-being-used-in-the-wild-8">It’s being used in the wild!</h2>
<p><img data-src="images/nextcloud-blablador-theme.png" /></p>
</section>
<section class="slide level1">
<h2 id="its-being-used-in-the-wild-8">It’s being used in the wild!</h2>
<h2 id="its-being-used-in-the-wild-9">It’s being used in the wild!</h2>
<ul>
<li class="fragment">FZJ’s IEK7 (Stratosphere) is also using Blablador
on their IEK7Cloud</li>
......@@ -638,6 +650,12 @@ on their IEK7Cloud</li>
Falcon-180B)</li>
<li class="fragment">Open for Helmholtz to have their models pointing to
Blablador (no one asked yet)</li>
<li class="fragment">Multi-GPU for SGLang (For Mixtral, and other models
which are too big for my GPUs)</li>
<li class="fragment">Multi-modal models (text+image, text+audio,
etc)</li>
<li class="fragment">Auto-RAG with privacy: don’t upload pdf, vector
database in browser’s ram</li>
</ul>
<h3 id="maybe">Maybe</h3>
<ul>
......@@ -660,9 +678,9 @@ just one more</li>
<h2 id="link-to-these-slides">Link to these Slides</h2>
<p><a
href="https://strube1.pages.jsc.fz-juelich.de/2024-talk-hai-retreat-blablador">https://strube1.pages.jsc.fz-juelich.de/2024-talk-hai-retreat-blablador</a></p>
href="https://strube1.pages.jsc.fz-juelich.de/2024-02-talk-lips-blablador/">https://strube1.pages.jsc.fz-juelich.de/2024-02-talk-lips-blablador/</a></p>
<p><a
href="https://gitlab.jsc.fz-juelich.de/strube1/2024-talk-hai-retreat-blablador/pages">Gitlab
href="https://gitlab.jsc.fz-juelich.de/strube1/2024-02-talk-lips-blablador/">Gitlab
link to source code of the slides (need JUDOOR account)</a></p>
</section>
<section class="slide level1">
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment