Adjusting deployment

.github/workflows/adoc_build.yml

@@ -2,8 +2,6 @@ name: Export and publish document
 
 on:
   workflow_dispatch:
-  push:
-    branches: [ main ]
 
 permissions:
   contents: read

.github/workflows/deploy.yml

@@ -1,7 +1,9 @@
-name: deploy-book
+name: deploy
 
 on:
   workflow_dispatch:
+  push:
+    branches: [ main ]
 
 jobs:
   deploy-book:
@@ -20,38 +22,28 @@ jobs:
           cache: true
           frozen: true
 
-      # Install dependencies
-      - name: Set up Python 3.11
-        uses: actions/setup-python@v3
-        with:
-          python-version: 3.11
-
-      - name: Install dependencies
-        run: |
-          pip install -r requirements.txt
-
       # (optional) Cache your executed notebooks between runs
       # if you have config:
       # execute:
       #   execute_notebooks: cache
       - name: cache executed notebooks
         uses: actions/cache@v3
         with:
-          path: _build/.jupyter_cache
-          key: jupyter-book-cache-${{ hashFiles('requirements.txt') }}
+          path: rtqc/_build/.jupyter_cache
+          key: jupyter-book-cache-${{ hashFiles('pixi.toml') }}
 
       # Build the book
       - name: Build the book
         run: |
-          jupyter-book build .
+          pixi run jupyter-book build rtqc
 
       # Upload the book's HTML as an artifact
       - name: Upload artifact
         uses: actions/upload-pages-artifact@v3
         with:
-          path: "_build/html"
+          path: "rtqc/_build/html"
 
       # Deploy the book's HTML to GitHub Pages
       - name: Deploy to GitHub Pages
         id: deployment
-        uses: actions/deploy-pages@v2
+        uses: actions/deploy-pages@v4

pixi.toml

@@ -9,5 +9,6 @@ platforms = ["osx-arm64", "linux-64", "win-64"]
 [tasks]
 
 [dependencies]
+python = "~=3.12.0"
 jupyter-book = ">=1.0.3,<2"
 matplotlib = ">=3.9.2,<4"

rtqc/_toc.yml

@@ -6,6 +6,5 @@ root: intro
 options:
   numbered: true
 chapters:
-- file: realtimeqc
 - file: qc_tests
 - file: references

rtqc/intro.md

@@ -1,11 +1,95 @@
-# Welcome to your Jupyter Book
+# Realtime Quality Control for Underwater Gliders
 
-This is a small sample book to give you a feel for how book content is
-structured.
-It shows off a few of the major file types, as well as some sample content.
-It does not go in-depth into any particular topic - check out [the Jupyter Book documentation](https://jupyterbook.org) for more information.
+## Credits
 
-Check out the content pages bundled with this sample book to see more.
+(List of contributors including brief description of the contribution(s))
 
-```{tableofcontents}
-```
+[List of contributors](section-contributors)
+
+## Statement of Purpose
+
+## Inbox - Content to be processed
+
+Here I'll add texts from our discussions that should be processed and organized
+in this new document. Mostly copy-n-paste so we don't loose anything.
+
+Topics that reached a consensus
+
+### What defines real-time QC?
+
+## Main Document
+
+<!-- Underwater gliders only -->
+This document is focused on underwater gliders, which use changes in buoyancy as the main propulsion, thus characterized by relatively slow propagation velocities.
+Other types of platforms with different natures of operation, such as wave gliders and propelled underwater autonomous vehicles, might take advantage of this document but there is no intention on covering specific aspects of those other platforms.
+
+<!-- Why should we do RTQC? -->
+There are a wide range of applications that cannot afford waitting for the delayed mode product due to time constraints, such as data assimilation for weather and sea state forecasts, thus requiring a real-time quality control with low latency. Given the diverse environments where gliders are operated and differences among glider models, the operators themselves are the best suited to evaluate their own measurements. Despite that, different users might have different priorities as well as tolerance for what they consider as useful data. While for data assimilation one might unforgive bad measurements and rather use less data with higher quality, a monitoring or alerting system cannot afford wrongfully flaggin and miss a single extreme event. Although there is no one unique optimal flagging for everyone, communication with the final users and understanding of the expectations can help fine tunning the QC criteria and desired informative flags. It should be expected that some users will apply their own QC procedure in addition to the QC from the operators, but all it takes is one obviously bad data point to damage the credibility of the whole dataset. Even before the final users, the glider operation itlsef benefits from a continuous real-time QC. A prompt detection of sensor degradation might trigger an early recovery and swap sensors or the full platform when possible. Catastrophic failures are sometimes preceded by anomalous behaviour, thus high rate of errors should raise the alertness of the pilots. Some spurious measurements are inevitable for any ocean observing system. Despite the constraints imposed by telemetry and pressure for low latency, real-time QC has clear benefits and should be part of all glider operation.
+
+<!-- Do not modify the original data -->
+The methods described here augment the original data to guide the user on the decision of what to trust and hence what to use. The raw data must not be modified, but instead  classification quality flags should be aggregated, therefore, allowing users to apply alternative quality control methods without limitations. For instance, advanced methods might be developed in the future, requiring the original data to avoid error propagation. Quality control in this document does not consider calibration but is limited to a classification. Whenever it is considered important to provide the best data possible ready to use, the recommendation is to aggregate such corrected data while preserving the original values directly available.
+
+<!-- Do not limit to automatic procedures -->
+Real-time QC is commonly associated with automatic checks, which indeed better suit the fast response requirements for a real-time data stream, but it should not be limited to automatic procedures. One of the distinct characteristics of underwater gliders is the close monitoring from pilots and scientists as frequently as hourly to at most every couple of days. 
+Complex cases, otherwise missed by the automatic procedures, but identified by the operators should be flagged and updated in the data stream. With the assumption of a low rate of mistakes by the automatic procedure, it is best to do not hold the automatic procedure for manual confirmation, but employ a fast automatic assessment and stream a posterior correction when necessary.
+
+include::tests.adoc[leveloffset=+1]
+
+## Procedures
+
+#### Argo BGC
+
+9. Spike test (median)
+
+Red Sea and Mediterranean Sea, flag 4 if test value > 1 micromole/kg
+
+Other places, flag 4 if test value > 5 micromole/kg
+
+## History
+
+Virtual synchronous meeting on XXX.
+(add participants/contributors on each step.
+
+### Virtual workshop
+
+Between May 11 - 15 2021
+
+(I think we had one single sync meeting. Find the correct date/time, participants, and notes taken)
+
+### Async discussions from Apr 10 2021 to July 19 2021
+Title Best Practices on Realtime QC
+
+### Called: Main document
+
+Contacted people by email.
+
+Asynchronous discussions through a Google Docs initiated on Feb 2021 and contributions extended up to Oct 2021.
+
+A Google Doc was choosen at that time to reduce the technical barrier and expand
+the participation and contributions.
+
+Participants:
+
+* Guilherme Castelao (Gui), [email protected], Scripps Institution of Oceanography
+* Soeren Thomsen, [email protected],	LOCEAN, OceanGliders.org Best Practice coordinator
+* Mark Bushnell, [email protected]
+* Pierre Testor, [email protected]
+* Emma Slater, [email protected]
+* Justin Buck, [email protected]
+* Mun Woo, [email protected],	IMOS Ocean Gliders
+* Thierry Carval, [email protected], IFREMER
+* Corentin Guyot, [email protected], IFREMER
+* Sylvie Pouliquen, [email protected], IFREMER and Euro*Argo ERIC
+* Victor Turpin, [email protected], OceanOPS
+* Nikolaos Zarokanellos, [email protected], SOCIB
+* Christoph Waldmann, [email protected], MARUM
+* John Kerfoot, [email protected]
+* Claire Gourcuff, [email protected]
+
+(section-contributors)=
+## Contributors
+
+List of all contributors an their contributions.
+
+*  Schmechtig (@catsch), CNRS
+   PR-1: Details on Argo's Nitrate test

rtqc/references.md

@@ -0,0 +1,4 @@
+(bibliography)=
+# References
+```{bibliography}
+```