feat: development environment seeding data script improvements #584
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kahlstrm
force-pushed
the
feat/seeding-script-improvements
branch
2 times, most recently
from
9fb5f06 to
28968fa
Compare
kahlstrm
force-pushed
the
feat/seeding-script-improvements
branch
from
28968fa to
4dc1fe2
Compare
joonatanaatos
approved these changes
Jan 5, 2025
| @@ -13,6 +13,7 @@ Prerequisites: | |||
| - [Node.js](https://nodejs.org/en/), usage of [fnm](https://github.com/Schniz/fnm) or [nvm](https://github.com/nvm-sh/nvm) recommended | |||
| - [Docker](https://www.docker.com/) and docker-compose | |||
| - [MongoDB Database Tools](https://www.mongodb.com/docs/database-tools/installation/installation/) (for seeding the dev-db). | |||
| - [bun](https://bun.sh/) (for scripts that seed the dev-db & fetch uploads from prod). | |||
There was a problem hiding this comment.
Do the scripts not work with Node?
scripts/generate_seeding_data.sh
Outdated
| echo "Copying uploads from apps/cms/uploads to db_data/gen/uploads..." | ||
| cp -r apps/cms/uploads data/gen | ||
| echo "cleaning up excess versions from versioned collections..." | ||
| bun run scripts/remove_excess_versions.ts |
There was a problem hiding this comment.
Here, could it try to default to Node if bun is not present?
kahlstrm
added a commit
that referenced
this pull request
Jan 6, 2025
…nts (#584) * feat: create script for fetching contents for upload collections from prod * feat: create script for removing excess versions from mongo export * chore: run import from prod with new scripts * fix: add fallback to node for generate_seeding_data script
kahlstrm
added a commit
that referenced
this pull request
Jan 21, 2025
…nts (#584) * feat: create script for fetching contents for upload collections from prod * feat: create script for removing excess versions from mongo export * chore: run import from prod with new scripts * fix: add fallback to node for generate_seeding_data script
kahlstrm
added a commit
that referenced
this pull request
Feb 4, 2025
…nts (#584) * feat: create script for fetching contents for upload collections from prod * feat: create script for removing excess versions from mongo export * chore: run import from prod with new scripts * fix: add fallback to node for generate_seeding_data script
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Previously when the dev db seeding was export from production DB, the versioned collections
didn't adhere to the 2-version limit due to production using higher version count retainment (100).
Also no images production were downloaded, leaving the development environment in a somewhat broken
state.
This PR addresses these two concerns with the following changes:
Create script for fetching contents for upload collections from prod
apps/cms/src/scripts/import-uploads.tswill iteratethe local payload instance for the upload collections, check if the appropriate
image file exists already locally, and if not it tries to download file from
PAYLOAD_PRODUCTION_URL.Create script for removing excess versions from mongo export
scripts/remove-excess-versions.tsrun import from prod with new excess removal script
Due to Payload 2 being very nice and having SCSS imports mixed with the package, running the payload
script with Node doesn't work, hence the added requirement for
bun. Maybe there could bebetter workaround, but
¯\_(ツ)_/¯Before submitting the PR, please make sure you do the following
Formatting and linting
pnpm formatand lint the project withpnpm lint