Merge pull request #9 from sfeir-open-source/docs/migration-slides

Docs/migration slides
sfeir-open-source · Nov 6, 2024 · 17e6131 · 17e6131
2 parents 3ba5f1f + c5788f7
commit 17e6131
Show file tree

Hide file tree

Showing 339 changed files with 27,645 additions and 12,769 deletions.
diff --git a/.gitignore b/.gitignore
@@ -73,3 +73,7 @@ docs/test.html
 venv
 
 /dbt_local/
+/dbt/sfeir_institute/
+.user.yml
+/dbt-env/
+logs/
diff --git a/docker/.env-dist b/docker/.env-dist
diff --git a/docker/README.md b/docker/README.md
diff --git a/docker/docker-compose.yaml b/docker/docker-compose.yaml
diff --git a/...s/images/docs/markdown/00-intro/agenda-marissa-grootes-Fj1aWk4LcNg-unsplash.jpg b/...s/images/docs/markdown/00-intro/agenda-marissa-grootes-Fj1aWk4LcNg-unsplash.jpg
diff --git a/...ets/images/docs/markdown/00-intro/setup-alexandru-acea-GhwCef9VRr4-unsplash.jpg b/...ets/images/docs/markdown/00-intro/setup-alexandru-acea-GhwCef9VRr4-unsplash.jpg
diff --git a/docs/assets/images/docs/markdown/100-documentation/doc-generated.png b/docs/assets/images/docs/markdown/100-documentation/doc-generated.png
diff --git a/docs/assets/images/docs/markdown/100-documentation/doc-generation.png b/docs/assets/images/docs/markdown/100-documentation/doc-generation.png
diff --git a/docs/assets/images/docs/markdown/100-documentation/graphical-lineage.png b/docs/assets/images/docs/markdown/100-documentation/graphical-lineage.png
diff --git a/...assets/images/docs/markdown/50-historical-data/snapshots-timestamp-strategy.svg b/...assets/images/docs/markdown/50-historical-data/snapshots-timestamp-strategy.svg
diff --git a/.../assets/images/docs/markdown/70-package-dependancies/dbt-packages-exploring.png b/.../assets/images/docs/markdown/70-package-dependancies/dbt-packages-exploring.png
diff --git a/docs/assets/images/docs/markdown/70-package-dependancies/dbt-packages.png b/docs/assets/images/docs/markdown/70-package-dependancies/dbt-packages.png
diff --git a/...ocs/markdown/60-quality/tester_douter.jpg → ...ocs/markdown/80-testing/tester_douter.jpg b/...ocs/markdown/60-quality/tester_douter.jpg → ...ocs/markdown/80-testing/tester_douter.jpg
diff --git a/docs/docker-compose.yml b/docs/docker-compose.yml
@@ -1,11 +1,11 @@
 version: '3'
 services:
   npx:
-    image: node:16.18
+    image: node:20.11
     user: node
-    command: npx serve
+    command: bash -c "npm install --ignore-scripts && npm start"
     volumes:
       - .:/sources
     working_dir: /sources
     ports:
-      - 3000:3000
+      - 4242:4242
diff --git a/docs/markdown/00-intro/10-intro.md b/docs/markdown/00-intro/10-intro.md
diff --git a/docs/markdown/00-intro/00-title.md → docs/markdown/000-intro/00-title.md b/docs/markdown/00-intro/00-title.md → docs/markdown/000-intro/00-title.md
@@ -1,3 +1,5 @@
 <!-- .slide: class="first-slide" sfeir-level="1" sfeir-techno="dbt" -->
-# **Welcome to Sfeir School**
-## **dbt 100**
+
+# **DBT**
+
+## **Introduction**
diff --git a/docs/markdown/000-intro/01-logistic.md b/docs/markdown/000-intro/01-logistic.md
@@ -0,0 +1,60 @@
+<!-- .slide: class="two-column" -->
+
+# Logistique
+
+![sfeir-icons big](truck) Parking
+
+![sfeir-icons big](home) Facilities
+
+![sfeir-icons big](coffee) Food
+
+![sfeir-icons big](pause-circle) Pause
+<br><span style="font-size: 1.5rem">10h30 / 12h30 / 16h00 </span>
+
+![sfeir-icons big](clock) Horaires
+<br><span style="font-size: 1.5rem"> 9h00 à 17h00 (environ)</span>
+
+##--##
+
+<!-- .slide: data-background="./assets/images/docs/markdown/00-intro/setup-alexandru-acea-GhwCef9VRr4-unsplash.jpg" class="mask" -->
+
+##==##
+
+<!-- .slide: class="two-column" -->
+
+# DO / DON'T
+
+![sfeir-icons big](mic) **Open Cam & Mic**
+
+Merci d’ouvrir caméras et micros lors
+des ateliers et pour poser vos questions.
+
+![sfeir-icons big](moon) **Night mode**
+
+Merci de couper vos notifications pendant
+la formation, des pauses seront aménagées.
+
+![sfeir-icons big](edit) **Informations**
+
+Merci de remplir la feuille de présence à la fin
+(et de nous laisser un commentaire 😊).
+
+Remplir les enquêtes de satisfaction en fin de formation.
+
+##--##
+
+![sfeir-icons big](pause-circle) **Pauses**
+
+10h30 / 12h30 / 16h00
+
+![sfeir-icons big](volume-x) **Smartphone**
+
+Veuillez couper vos téléphone et prendre les appels à l’extérieur
+
+![sfeir-icons big](video-off) **Caméra**
+
+L’enregistrement de ce cours est interdit
+
+![sfeir-icons big](help-circle) **Questions**
+
+Posez vos questions de manière interactive ou via le chat (en ligne)
diff --git a/docs/markdown/000-intro/10-intro.md b/docs/markdown/000-intro/10-intro.md
@@ -0,0 +1,24 @@
+<!-- .slide -->
+
+# Objectifs de la formation
+
+**Acquérir les compétences suivantes :**
+
+- Comprendres les concepts clés, les avantages et l’architecture de dbt
+- Créer des modèles de données structurés et effectuer des transformations pour les préparer
+- Maîtriser les fonctionnalités avancées telles que les macros, les modèles Jinja, les variables
+- Utiliser les snapshots dbt pour suivre les changements au fil du temps et gérer les données historiques
+- Mettre en œuvre des tests pour garantir la qualité et l'intégrité des données
+
+**Et vos objectifs personnels ?**
+
+Notes:
+Comprendre les concepts clés, les avantages et l'architecture de dbt en tant qu'outil de transformation et de modélisation des données.
+
+Créer des modèles de données structurés avec dbt, et effectuer des transformations pour traiter et préparer les données en vue de l'analyse.
+
+Maîtriser les fonctionnalités avancées telles que les macros, les modèles Jinja, les variables et le contrôle de flux
+
+Utiliser les instantanés (snapshots) dbt pour suivre les changements au fil du temps et gérer les données historiques, facilitant l'analyse des tendances historiques et les dimensions à changement lent.
+
+Mettre en œuvre des tests pour garantir la qualité et l'intégrité des données, permettant de valider les résultats des transformations et de détecter les anomalies
diff --git a/docs/markdown/000-intro/11-agenda-day1.md b/docs/markdown/000-intro/11-agenda-day1.md
@@ -0,0 +1,16 @@
+<!-- .slide: class="two-column" -->
+
+# Agenda
+
+**Day 1**
+
+- Evolution of Data Stack
+- Introduction to dbt
+- Working with dbt models
+- dbt sources and references
+- Seeding your data models
+- Snapshots to manage historical data
+
+##--##
+
+<!-- .slide: data-background="./assets/images/docs/markdown/00-intro/agenda-marissa-grootes-Fj1aWk4LcNg-unsplash.jpg" class="mask" -->
diff --git a/docs/markdown/000-intro/12-agenda-day2.md b/docs/markdown/000-intro/12-agenda-day2.md
@@ -0,0 +1,17 @@
+<!-- .slide: class="two-column" -->
+
+# Agenda
+
+**Day 2**
+
+- Advanced data transformation and control
+- Dependencies
+- Testing
+- Contracts and versions
+- Documentation
+- Analyses, hooks and exposure
+- Leveraging generated artifacts
+
+##--##
+
+<!-- .slide: data-background="./assets/images/docs/markdown/00-intro/agenda-marissa-grootes-Fj1aWk4LcNg-unsplash.jpg" class="mask" -->
diff --git a/docs/markdown/00-intro/21-speaker-aal.md → docs/markdown/000-intro/21-speaker-aal.md b/docs/markdown/00-intro/21-speaker-aal.md → docs/markdown/000-intro/21-speaker-aal.md
diff --git a/docs/markdown/00-intro/22-speaker-hmd.md → docs/markdown/000-intro/22-speaker-hmd.md b/docs/markdown/00-intro/22-speaker-hmd.md → docs/markdown/000-intro/22-speaker-hmd.md
@@ -10,7 +10,6 @@
 
 <h2>Henri-Maxime <span style="padding-right:0">Ducoulombier</span></h2>
 
-
 ### Senior Data Architect
 
 <!-- .element: class="icon-rule icon-first" -->

diff --git a/docs/markdown/00-intro/30-who-are-you.md → docs/markdown/000-intro/30-who-are-you.md b/docs/markdown/00-intro/30-who-are-you.md → docs/markdown/000-intro/30-who-are-you.md
diff --git a/docs/markdown/010-once-upon-a-time/00-title.md b/docs/markdown/010-once-upon-a-time/00-title.md
@@ -0,0 +1,5 @@
+<!-- .slide: class="transition"-->
+
+# Evolution of Data Stack
+
+## Module 1
diff --git a/docs/markdown/010-once-upon-a-time/10-architectures.md b/docs/markdown/010-once-upon-a-time/10-architectures.md
@@ -0,0 +1,72 @@
+# Data Platforms evolution
+
+![full-center](./assets/images/docs/markdown/10-once-upon-a-time/data-architectures.svg)
+
+Notes:
+Datalake
+
+- Scalable and flexible storage for diverse, raw data types
+
+Data Warehouse
+
+- Centralized storage and analysis system for structured business data.
+
+Lakehouse
+
+- Unified data platform combining data lake and data warehouse capabilities.
+
+Data Mesh
+
+- Domain-oriented, self-serve data products
+
+##==##
+
+# 2012 Data Landscape
+
+![center hm-800](./assets/images/docs/markdown/10-once-upon-a-time/2012-data-landscape.png)
+
+##==##
+
+# 2021 Data Landscape
+
+![center hm-800](./assets/images/docs/markdown/10-once-upon-a-time/2021-data-landscape.png)
+
+##==##
+
+# Data Platform
+
+## From ETL...
+
+![center full-width](./assets/images/docs/markdown/10-once-upon-a-time/data-platforms.svg)
+
+##==##
+
+# Modern Data Platform
+
+## ...to ELT
+
+![center full-width](./assets/images/docs/markdown/10-once-upon-a-time/modern-data-platforms.svg)
+
+##==##
+
+<!-- .slide: class="two-column" -->
+
+# <u>E</u>xtract <u>T</u>ransform <u>L</u>oad
+
+![sfeir-icons big](git-merge) data is **extracted** from various sources
+
+![sfeir-icons big](tool) **transformed** into a suitable form
+
+![sfeir-icons big](upload) and then **loaded** into a target system for analysis.
+
+##--##
+
+<!-- .slide: data-background="var(--black)" -->
+
+# <u>E</u>xtract <u>L</u>oad <u>T</u>ransform
+
+![sfeir-icons big](git-merge) data is **extracted** from various sources
+
+![sfeir-icons big](upload) **loaded** into a target system without immediate transformation
+
+![sfeir-icons big](tool) **transformation** occurs when needed, to expose data products and use cases
diff --git a/docs/markdown/012-dbt/00-title.md b/docs/markdown/012-dbt/00-title.md
@@ -0,0 +1,5 @@
+<!-- .slide: class="transition"-->
+
+# Introduction to _dbt_
+
+## Module 2
diff --git a/docs/markdown/012-dbt/10-dbt.md b/docs/markdown/012-dbt/10-dbt.md
@@ -0,0 +1,34 @@
+# Core concepts
+
+<!-- .slide: class="two-column" -->
+
+![center hm-800](./assets/images/docs/markdown/12-dbt/analytics-engineering-dbt.png)
+
+##--##
+
+<!-- .slide: data-background="var(--black)"-->
+
+#
+
+<br/>
+
+![sfeir-icons big](edit) Transformation are expressed with SQL `SELECT` statement only
+
+![sfeir-icons big](refresh-ccw) Reference between models are automatically build
+
+![sfeir-icons big](target) Tests ensure model accuracy
+
+![sfeir-icons big](book) Documentation is accessible and easily updated
+
+![sfeir-icons big](terminal) KISS: use macros to write reusable SQL
+
+##==##
+
+# Open Source but not only
+
+|                     | dbt Core                                                            | dbt Cloud                                                                                                                                                           |
+| ------------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Advantage**       | More flexible and customizable                                      | Short time to value                                                                                                                                                 |
+| **Model**           | Command-line interface (CLI)                                        | Web-integrated IDE                                                                                                                                                  |
+| **Pricing**         | Free to user                                                        | <ul><li>Developer ⇒ free for one developper seat </li><li>Team ⇒ $100 / dev / month (max 8 seats) </li><li>Custom ⇒ $300 / dev / month (min 5 seats)</li>           |
+| **Functionalities** | Possibility to add features and enhancements while it’s open-source | IDE browser-based, job scheduling, job logging, monitoring, alerting, job documentation, version control integration, SSO, role-based access control, access to API |
diff --git a/docs/markdown/015-installation/00-title.md b/docs/markdown/015-installation/00-title.md
@@ -0,0 +1,5 @@
+<!-- .slide: class="transition"-->
+
+# Introduction to dbt
+
+## Installation
Original file line number	Diff line number	Diff line change
Expand Up		@@ -10,7 +10,6 @@

		<h2>Henri-Maxime <span style="padding-right:0">Ducoulombier</span></h2>


		### Senior Data Architect

		<!-- .element: class="icon-rule icon-first" -->
Expand Down