Don't deploy outputs when using workflow_dispatch trigger on main

[zaneselvans]

Overview

Due to some faulty logic in our altogether too complex conditionals for deciding when/where to deploy build outputs, running a workflow_dispatch deployment on main results in the outputs being deployed to the following paths:

s3://pudl.catalyst.coop/main/
gs://pudl.catalyst.coop/main/

Tasks

Preview Give feedback

1. Simplify the nightly/stable/workflow_dispatch logic as outlined below
2. Attempt to create a Zenodo sandbox data release for workflow_dispatch builds
3. Stop deploying to parquet.catalyst.coop since we publish all parquet files now
4. Use gcloud instead of AWS CLI since it can manage S3 now

[jdangerx]

Poked around at the gcp_pudl_etl.sh. I think there's lots of room for improvement with minimal effort.

I think we want this general structure for the file:

# set BUILD_TYPE to be "nightly", "stable", or "ad-hoc" based on GITHUB_ACTION_TRIGGER, GITHUB_REF, etc, one time

# blah blah function definitions here. we already have most of these

# run ETL
# clean up postgres
# save outputs to internal bucket

if [[ $BUILD_TYPE == "nightly" ]]; then
    # merge tag nightly-yyyy-mm-dd to nightly
    # update datasette
    # distribute parquet
    # clean up outputs
    # distribute to S3 nightly location
    # distribute to GCS nightly location
    # make Zenodo sandbox record
elif [[ $BUILD_TYPE == "stable" ]]; then
    # merge tag v20yy.m.v to stable
    # distribute parquet
    # clean up outputs
    # distribute to S3 versioned location
    # distribute to S3 stable location
    # distribute to GCS versioned location
    # distribute to GCS stable location
    # make Zenodo prod record draft
    # lock GCS versioned outputs
elif [[ $BUILD_TYPE == "ad-hoc" ]]; then
    # throw something in the logs like "ad-hoc build, not distributing anything"
fi

# finish uploading logs
# notify slack based on success

We'd get to take out a lot of the "figure out where we're putting stuff" logic from the various function definitions, too, and just pass the targets in at the call sites, which would make our conditionals a lot simpler too.

Then when something like this pops up, it's most likely an error in the "set $BUILD_TYPE" logic.

If this design seems reasonable, I'm happy to take a crack at it @zaneselvans . Or you can if you're looking for some tech work...

[jdangerx]

Punting this until next sprint since we're not focusing on the build script refactor/quality for this sprint.

[e-belfer]

Putting this into next sprint, since we'll be doing yet another release in November and this seems to be a problem again.