Update data management (#152)

Align the data management strategy with EPP and simplify the example project.

---------

Co-authored-by: Hans-Martin von Gaudecker <[email protected]>

CHANGES_template.md

@@ -1,5 +1,13 @@
 # Release Notes
 
+## v0.7.2 -- July 2024
+
+Complete re-write of how to obtain templates, simplifying the example project, and
+aligning the data management part with that of the newest EPP iteration.
+
+- Remove cookiecutter (#149, @timmens, @hmgaudecker)
+- Update data management (#152, @timmens, @hmgaudecker)
+
 ## v0.7.1 -- December 2023
 
 - Replace `Pdb++` by `Pdb+` (#142, @janosg).

docs_template/scripts/latex/root_bld_src.tex

@@ -36,9 +36,9 @@
                 \nodepart{two}
                 \texttt{bld}
                 \nodepart{three}
-                \texttt{template\_project.pdf}
+                \texttt{paper.pdf}
                 \nodepart{four}
-                \texttt{template\_project\_pres.pdf}
+                \texttt{presentation.pdf}
                 \nodepart{five}
                 \texttt{src}
                 \nodepart{six}
@@ -47,14 +47,15 @@
 
             \node (2) [
                 rectangle split,
-                rectangle split parts=6,
+                rectangle split parts=7,
                 rectangle split part fill={
                         gray,
                         yellow!75,
                         yellow!75,
                         yellow!75,
                         yellow!75,
-                        yellow!75
+                        yellow!75,
+                        yellow!75,
                     },
                 draw,
                 text width=2.50cm,
@@ -75,6 +76,8 @@
                 \texttt{models}
                 \nodepart{six}
                 \texttt{predictions}
+                \nodepart{seven}
+                \texttt{documents}
             };
 
             \node (3) [
@@ -85,9 +88,7 @@
                         blue!75,
                         blue!75,
                         blue!75,
-                        blue!50,
-                        blue!50,
-                        blue!50,
+                        blue!75,
                         blue!50,
                     },
                 draw,

docs_template/scripts/latex/src.tex

@@ -50,12 +50,11 @@
 
         \node (5) [
             rectangle split,
-            rectangle split parts=4,
+            rectangle split parts=3,
             rectangle split part fill={
                 gray,
                 blue!50,
                 blue!50,
-                blue!50,
             },
             draw,
             text width=3.0cm,
@@ -70,8 +69,6 @@
                 \texttt{\_\_init\_\_.py}
             \nodepart{three}
                 \texttt{clean\_data\{.py,.r,.jl,.do\}}
-            \nodepart{four}
-                \texttt{data\_info.yaml}
         };
 
         \node (4) [

docs_template/scripts/task_create_visualization_for_docs.py

@@ -82,7 +82,7 @@ def _visualize_organisational_steps(case):
     )
     fig.update_yaxes(
         tickvals=[1, 2],
-        ticktext=["marital-status", "qualification"],
+        ticktext=["marital-status", "highest-qualification"],
         title="Variable",
     )
 

docs_template/source/guides_explanations/porting_existing_project.md

@@ -9,8 +9,11 @@ the template, understand what is happening and why, and move on.
 1. Start with the data management code and move your data files to the spot where they
    belong under the new structure.
 1. Move (the first steps of) your data management code to the folder under the
-   templates. Modify the `task_xxx` files accordingly or create new ones.
+   templates. Create new `task_...` files.
 1. Run `pytask`, adjusting the code for the errors you'll likely see.
 1. Move on step-by-step like this.
-1. Delete the example files and the corresponding sections of the `task_xxx` files / the
-   entire files in case you created new ones.
+1. Once you feel secure enough that you do not need the template files any more, delete
+   all files carrying a `_template` in their names. You will also need to adjust the
+   documents so they do not refer to figures and tables created by the template any
+   more. Delete the build directory to make sure you do not rely on outputs from tasks
+   that you removed.

docs_template/source/guides_explanations/starting_from_scratch.md

@@ -11,5 +11,8 @@ in {ref}`template_setup`, my recommendation would be as follows.
    necessary or create new ones.
 1. Remove the build directory regularly to make sure you do not rely on outputs from
    tasks that do not exist any more — this is a frequent source of confusion.
-1. Once you got the hang of how things work, remove the examples (both the data files
-   and the code in the `task_xxx` files). Also remove the build directory.
+1. Once you feel secure enough that you do not need the template files any more, delete
+   all files carrying a `_template` in their names. You will also need to adjust the
+   documents so they do not refer to figures and tables created by the template any
+   more. Delete the build directory to make sure you do not rely on outputs from tasks
+   that you removed.

environment.yml

@@ -20,7 +20,6 @@ dependencies:
   - pytest-xdist
 
   # Python template project dependencies
-  - pyyaml
   - statsmodels
   - numpy
   - pandas >=2.2
@@ -33,6 +32,7 @@ dependencies:
   - r-plyr
   - r-precommit
   - r-yaml
+  - r-forcats
 
   # Install project
   - pip: [-e ., kaleido]

inst/WORDLIST

@@ -722,6 +722,7 @@ FO
 Fomel
 FomelHennenfent
 foR
+forcats
 formatters
 fp
 Fp