using a venv with github action to render docstring example

[DOH-FAA3303]

I'm feeling defeated. I have a simple example in a docstring that I want to render in my quartodoc github page, like this:

def test(df):
   """
   Examples
   --------
   ```{python}
   import polars as pl
   df = pl.DataFrame(
            {
                "a": ["a", "b", "a", "b", "c"],
                "b": [1, 2, 1, 3, 3],
                "c": [5, 4, 3, 2, 1],
            }
        )
   # the output from this will execute and render
   df.group_by("a").agg(pl.col("b"), pl.col("c"))  
   ```
   """

I can successfully install my dependencies, but when my github action gets to the publish step it errors out saying There is an unactivated Python environment in .venv. Did you forget to activate it?

Here is my full github action yaml:

on:
  workflow_dispatch:
  push:
    branches: 
      - main

name: Render and Publish

jobs:
  build-deploy:
    runs-on: ubuntu-latest
    
    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.10'
          cache: 'pip' # caching pip dependencies
      # - run: pip install -r requirements.txt
      - run: |
          python -m venv .venv
          source .venv/bin/activate
          pip install --upgrade pip
          pip install jupyter
          pip install pyyaml
          pip install -r requirements.txt
          echo PATH=$PATH >> $GITHUB_ENV
        
      - name: Set up Quarto
        uses: quarto-dev/quarto-actions/setup@v2
        with:
          version: '1.5.22'
        
      - name: Prep CURL install
        run: sudo apt-get update

      - name: Install CURL Headers
        run: sudo apt-get install libcurl4-openssl-dev

      - name: Build quartodoc
        run: cd docs/ && quartodoc build

      - name: Publish to GitHub Pages (and render)
        uses: quarto-dev/quarto-actions/[email protected]
        with:
          target: gh-pages
          path: docs/_site
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # this secret is always available for github actions
          PYTHON_ENV: .venv/bin/activate

I have tried to pip install within the Publish step, save and activate the venv in that step, and more. Any ideas on how I can make sure my venv gets activated in the publish step??

[DOH-FAA3303]

i figured it out. I separated the quarto render step from the publish step, and I specifically activated the venv in the render step:

on:
  workflow_dispatch:
  push:
    branches: 
      - main

name: Render and Publish

jobs:
  build-deploy:
    runs-on: ubuntu-latest
    
    env:
      RENV_PATHS_ROOT: ~/.cache/R/renv
      
    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.10'
          cache: 'pip' # caching pip dependencies
      # - run: pip install -r requirements.txt
      - run: |
          python -m venv .venv
          source .venv/bin/activate
          pip install --upgrade pip
          pip install jupyter
          pip install pyyaml
          pip install -r requirements.txt
          echo PATH=$PATH >> $GITHUB_ENV
        
      - name: Set up Quarto
        uses: quarto-dev/quarto-actions/setup@v2
        with:
          version: '1.5.22'
        
      - name: Prep CURL install
        run: sudo apt-get update

      - name: Install CURL Headers
        run: sudo apt-get install libcurl4-openssl-dev

      - name: Build quartodoc
        run: |
          python -m venv .venv
          source .venv/bin/activate
          pip install --upgrade pip
          pip install jupyter
          pip install pyyaml
          pip install -r requirements.txt
          echo PATH=$PATH >> $GITHUB_ENV
          cd docs/ && quartodoc build

      - name: Render Quarto documents
        run: |
          . .venv/bin/activate
          cd docs/
          quarto render

      - name: Publish to GitHub Pages (and dont render)
        uses: quarto-dev/quarto-actions/[email protected]
        with:
          target: gh-pages
          path: docs/_site
          render: "false"
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # this secret is always available for github actions

On a side note, I was able to get more crazy with my docstring example by reference a dataset in an external folder and referencing a custom function in a different folder:

def reason_null(specimen,reason,lab):
  """ Check if Reason Column NULL

    Examples
    --------
    ```{python}
    import os
    import sys
    import polars as pl
    import polars.selectors as cs

    module_path = os.path.abspath(os.path.join('../..'))
    if module_path not in sys.path:
        sys.path.append(module_path)
    # This allows you to import the desired function from the module hierarchy:

    import notebooks.functions as qa

    df = pl.read_csv('../../notebooks/test_data.csv')

    (
        df
        .with_columns(
        flag_reason_null = qa.reason_null(
            specimen="specimen",
            reason="reason",
            lab="lab"
            )
        )
    )
    ```
  """

It is cumbersome to say the least. If anyone has a more elegant way than needing to change the absolute path for every docstring function let me know!

[DOH-FAA3303]

Here's a less cumbersome but still not sexy way of doing what I want above:

make a _setup.qmd in your docs folder where you can pull in the relevant paths to call in your custom functions:

import os
import sys

module_path = os.path.abspath(os.path.join('../..'))
if module_path not in sys.path:
    sys.path.append(module_path)
# This allows you to import the desired function from the module hierarchy:

Then you can call that into your Example docstring and set echo: false to prevent the code from rendering and an include shortcode if you dont want to copy/paste that code into all of your docstring examples:

```{python}
#| echo: false
{{< include "../_setup.qmd" >}}
```

full example:

def reason_null(specimen,reason,lab):
  """ Check if Reason Column NULL

    reason column cannot be null

    Usage
    -----
    To be run against a table, should be pathogen agnostic.

    Parameters
    ----------
    specimen: string
        Specimen equals Yes or No. Indicates whether the specimen was sequenced or not.
    reason: string
        The reason for sequencing. Typically 'SENTINEL SURVEILLANCE' for COVID-19
    lab: string
        The lab that did the sequencing. If NULL then there you have other issues!

    Examples
    --------
    ```{python}
    #| echo: false
    {{< include "../_setup.qmd" >}}
    ```
    ```{python}
    import polars as pl
    import polars.selectors as cs
    import notebooks.functions as qa

    df = pl.read_csv('../../notebooks/test_data.csv')

    (
        df
        .with_columns(
        flag_reason_null = qa.reason_null(
            specimen="specimen",
            reason="reason",
            lab="lab"
            )
        )
    )
    ```
  """

and output