chore: (WIP) Added script to drop tables from DB

[syedimranhassan]

---

Make sure that the following steps are done before merging:

• Have a Site Reliability Engineer review the PR if you don't own all of the services impacted.
• If you are adding any new default values that need to be overridden when this change goes live, update internal repos and add an entry to the top of the CHANGELOG.
• Performed the appropriate testing.

[robrap]

@syedimranhassan:

1. The list of tables in https://2u-internal.atlassian.net/browse/GSRE-2673 are very specific. Can you confirm that you will limit your script to run for only those tables listed if they pass your checks? I am not saying that any table that has not changed in 12 months is ok to delete.
2. In the list of tables, you'll see that there is a foreign-key between 2 of the tables to be deleted, so order will matter, as well as removing that foreign-key.

[syedimranhassan]

2. In the list of tables, you'll see that there is a foreign-key between 2 of the tables to be deleted, so order will matter, as well as removing that foreign-key.

Thanks @robrap, I’ve updated the script to operate only on the predefined list of tables instead of taking input from the user. It now checks if a table has had any changes in the last 12 months before proceeding to drop either the foreign key or the table. The script ensures foreign keys are dropped only for the necessary tables before removing them to prevent dependency issues. Let me know if you have any feedback!

[robrap]

Thanks.

1. Is there already a way to run the script in a dryrun mode, or does that need to be explicitly added?
2. Could you get the dryrun output for the suggested tables? Do we need logging for what was removed?
3. The 1 year check is just for extra safety, so the full list should be deleted. If not, we need to understand why.
4. To test your last activity code, you could tun in dryrun mode against a table with recent activity.

Thank you.

[syedimranhassan]

Currently, the script does not have an explicit dry-run mode, but I am working on adding that functionality. This will allow us to see which tables and foreign keys would be dropped without actually performing the deletions.

Previously, the script simply printed the actions it performed, but I have replaced those with logging for better visibility. With the 1-year check, the dry-run mode will help identify any tables that have had activity within the last 12 months. Additionally, when running the script without dry-run, the logs will provide insight into why a table was skipped.

[robrap]

Once your script is ready, you can run in dryrun mode locally (make sure it doesn't do anything harmful), and then in stage, to see the actual output and that it will delete what was predicted on the ticket. Thanks

UPDATE: I'd recommend running dryrun in prod and edge too, but you can wait until you are actually ready to perform the clean-up.