From 0fce6696acc049013462221119b26401106ae390 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Thu, 28 Nov 2024 15:37:18 -0500 Subject: [PATCH 1/9] WIP: 017 - added empty Impact section --- adr/017-connection-pooling.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/adr/017-connection-pooling.md b/adr/017-connection-pooling.md index 2ca6749ac..b89369b56 100644 --- a/adr/017-connection-pooling.md +++ b/adr/017-connection-pooling.md @@ -13,6 +13,19 @@ Accepted ## Context Connection pooling is vital to the project for many reasons including resource management and conservation, performance improvement, latency reduction, scalability, and most importantly here managing connections. +## Impact + +### Positive + +- + +### Negative + +- + +### Risks + +- ### Related Issues From a2b978cccee5698b2ca0d99b33aeca8edc42e436 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Thu, 28 Nov 2024 16:05:01 -0500 Subject: [PATCH 2/9] 017 - added impact section --- adr/017-connection-pooling.md | 40 ++++++++++++++++++++++++++++++++--- 1 file changed, 37 insertions(+), 3 deletions(-) diff --git a/adr/017-connection-pooling.md b/adr/017-connection-pooling.md index b89369b56..9cf8545e7 100644 --- a/adr/017-connection-pooling.md +++ b/adr/017-connection-pooling.md @@ -17,15 +17,49 @@ Connection pooling is vital to the project for many reasons including resource m ### Positive -- +- **High Performance:** HikariCP outperforms other connection pool libraries in terms of throughput and latency, enhancing application responsiveness. + + +- **Stability and Reliability:** Active maintenance ensures stability, addressing potential bugs and vulnerabilities. + + +- **Ease of Use:** Comprehensive documentation simplifies integration and configuration for developers. + + +- **Resource Efficiency:** Limits the number of open connections, reducing load on the database server. + + +- **Scalability:** Handles large-scale applications effectively, supporting high concurrency with low overhead. + + +- **Enhanced Diagnostics:** Built-in monitoring and logging features facilitate troubleshooting and performance tuning. ### Negative -- +- **Dependency Overhead:** Adds an external library dependency that must be maintained and updated over time. + + +- **Configuration Complexity:** Suboptimal configuration (e.g., pool size, connection timeout) can degrade performance or cause issues under load. + + +- **Learning Curve:** Developers unfamiliar with connection pooling or HikariCP may require additional time for onboarding. + + +- **Troubleshooting Complexity**: Adds another layer to the stack, potentially complicating debugging of database-related issues. ### Risks -- +- **Connection Leaks:** Improperly handled connections could lead to resource exhaustion, affecting application stability. +- Incorrect Configuration: Poorly tuned settings may result in underutilized or overburdened pools. + + +- **Driver Compatibility:** Potential issues with certain database drivers if not thoroughly tested. + + +- **Operational Challenges:** Monitoring and maintaining the pool in production environments may require additional effort. + + +- **Dependency Lock-in:** Switching to another connection pooling library in the future may require significant refactoring. ### Related Issues From 9c95dcc92ffa604b1f1cbd1a802cf24dce9bba06 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Thu, 28 Nov 2024 16:05:12 -0500 Subject: [PATCH 3/9] 018 - added impact section --- adr/018-sql-migrations.md | 41 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/adr/018-sql-migrations.md b/adr/018-sql-migrations.md index 2c97a44b8..a6ebf386e 100644 --- a/adr/018-sql-migrations.md +++ b/adr/018-sql-migrations.md @@ -13,6 +13,47 @@ Accepted ## Context As part of having CI/CD pipelines we need tooling inside of the project to allow us to deploy and rollback changes in an automated way. Using Liquibase enables us to store SQL migration scripts inside of the project and update them as needed. We can then set up the project deployment pipelines to automatically pick up the new changes and make the updates to the database. Liquibase will also allow for non-manual rollbacks should the need for them arise. As part of this implementation we will be using the Liquibase generated Github Actions in our pipelines to achieve this automation. +## Impact + +### Positive + +- **Rollback Functionality:** Liquibase’s ability to perform rollbacks in the open-source version ensures we can quickly recover from migration issues without additional cost. + + +- **Improved CI/CD Integration:** Automating migrations reduces the need for manual intervention and decreases deployment errors. + + +- **Version Control of Schema:** Keeping migration scripts within the project ensures that database changes are tracked alongside application changes, improving traceability. + + +- **Extensive Documentation:** Liquibase’s comprehensive documentation simplifies onboarding and troubleshooting. + + +- **Flexibility:** Liquibase supports a wide range of databases and migration formats, making it adaptable for future needs. + +### Negative + +- **Learning Curve:** Teams unfamiliar with Liquibase may require time to learn its configuration and scripting syntax. + + +- **Setup Complexity:** Initial integration with CI/CD pipelines may require extra effort to align with existing workflows. + + +- **Dependency Overhead:** Adds a dependency to the project, which must be maintained and updated over time. + +### Risks + +- **Migration Errors:** Incorrectly defined migration scripts could cause deployment failures or data integrity issues. + + +- **Rollback Limitations:** Not all changes (e.g., destructive data operations) can be automatically rolled back, requiring careful planning for such scenarios. + + +- **Version Drift:** If environments are not consistently updated with migrations, discrepancies could arise between development, staging, and production. + + +- **GitHub Action Reliability:** Relying on prebuilt GitHub Actions introduces a dependency on external tools, which may have compatibility or update issues over time. + ### Related Issues From ce59f8c1d33fa47db259f17a3357b27b13b11de3 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Thu, 28 Nov 2024 16:05:23 -0500 Subject: [PATCH 4/9] 019 - added impact section --- adr/019-link-message-ids.md | 38 +++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/adr/019-link-message-ids.md b/adr/019-link-message-ids.md index 598f37ddc..52b9c5e31 100644 --- a/adr/019-link-message-ids.md +++ b/adr/019-link-message-ids.md @@ -16,6 +16,44 @@ As part of the requirement to link an order with its corresponding result(s), we We decided to create a new table for this purpose, but we also discussed the option to instead add a new field to the `metadata` table with an array of link IDs. Even though this option would reduce the complexity in our database and code, we decided to create the new table because we'll soon need to pull data for each ID in order to populate the response from the `metadata` endpoint, which is a usecase that the separate table will fit better. If we find that the tradeoff is worth it, we may decide in the future to refactor the code and use the array field instead of the table. +## Impact + +### Positive + +- **Data Integrity:** Decouples the link storage from the `metadata` table, ensuring links remain intact even if metadata fields change. + + +- **Flexibility:** The `message_link` table can adapt to future requirements without altering the `metadata` schema. + + +- **Efficient Querying:** Simplifies the process of retrieving linked data by isolating relationships in a dedicated table. + + +- **Scalability:** A separate table is better suited for handling large-scale relationships, particularly as the volume of linked data grows. + + + +### Negative + +- **Increased Complexity:** Adds another table to the database, which introduces additional queries and joins in the codebase. + + +- **Data Synchronization Overhead:** Requires careful management to ensure consistency between the `metadata` table and the `message_link` table. + + +- **Initial Implementation Effort:** More time and resources needed for implementation compared to the simpler array field approach. + +### Risks + +- **Data Consistency:** Potential risks of mismatches between `metadata` and `message_link` entries due to bugs or incomplete synchronization. + + +- **Performance Concerns:** Queries involving multiple joins with the `message_link` table might impact database performance if not optimized. + + +- **Schema Evolution:** Future changes to linking logic or requirements may necessitate updates to the `message_link` table structure. + + ### Related Issues #621 From f277c9962016c15e197d69525b5039b7ab7aacb8 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 4 Dec 2024 16:40:18 -0500 Subject: [PATCH 5/9] 017 - fixed formatting --- adr/017-connection-pooling.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/adr/017-connection-pooling.md b/adr/017-connection-pooling.md index 9cf8545e7..47a4e4264 100644 --- a/adr/017-connection-pooling.md +++ b/adr/017-connection-pooling.md @@ -50,7 +50,9 @@ Connection pooling is vital to the project for many reasons including resource m ### Risks - **Connection Leaks:** Improperly handled connections could lead to resource exhaustion, affecting application stability. -- Incorrect Configuration: Poorly tuned settings may result in underutilized or overburdened pools. + + +- **Incorrect Configuration:** Poorly tuned settings may result in underutilized or overburdened pools. - **Driver Compatibility:** Potential issues with certain database drivers if not thoroughly tested. From 7728b0f5c18e9a27b02a1edefe494af6273f935a Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 4 Dec 2024 16:41:11 -0500 Subject: [PATCH 6/9] 018 - added recommended changelog structure issues --- adr/018-sql-migrations.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/adr/018-sql-migrations.md b/adr/018-sql-migrations.md index a6ebf386e..e9fb1212a 100644 --- a/adr/018-sql-migrations.md +++ b/adr/018-sql-migrations.md @@ -52,9 +52,17 @@ As part of having CI/CD pipelines we need tooling inside of the project to allow - **Version Drift:** If environments are not consistently updated with migrations, discrepancies could arise between development, staging, and production. -- **GitHub Action Reliability:** Relying on prebuilt GitHub Actions introduces a dependency on external tools, which may have compatibility or update issues over time. +- **GitHub Action Reliability:** Relying on prebuilt GitHub Actions introduces a dependency on external tools, which may have compatibility or update issues over time. +- **Changelog Structure Issues:** + + - Referencing `metadata.received_message_id` in the `message_link` table creates dependencies that may complicate schema migrations. + + - Changes to the metadata table structure, such as renaming or modifying the `received_message_id` column, require careful coordination with the `message_link` table to prevent breaking changes. + + - Liquibase changelog design must ensure that migrations are properly ordered and modular to accommodate these dependencies. + ### Related Issues - 753 From e72a6376e0ec1ce250c155dc4c295f01d9200c37 Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 4 Dec 2024 16:47:02 -0500 Subject: [PATCH 7/9] Revert "018 - added recommended changelog structure issues" This reverts commit 7728b0f5c18e9a27b02a1edefe494af6273f935a. --- adr/018-sql-migrations.md | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/adr/018-sql-migrations.md b/adr/018-sql-migrations.md index e9fb1212a..a6ebf386e 100644 --- a/adr/018-sql-migrations.md +++ b/adr/018-sql-migrations.md @@ -52,17 +52,9 @@ As part of having CI/CD pipelines we need tooling inside of the project to allow - **Version Drift:** If environments are not consistently updated with migrations, discrepancies could arise between development, staging, and production. -- **GitHub Action Reliability:** Relying on prebuilt GitHub Actions introduces a dependency on external tools, which may have compatibility or update issues over time. +- **GitHub Action Reliability:** Relying on prebuilt GitHub Actions introduces a dependency on external tools, which may have compatibility or update issues over time. -- **Changelog Structure Issues:** - - - Referencing `metadata.received_message_id` in the `message_link` table creates dependencies that may complicate schema migrations. - - - Changes to the metadata table structure, such as renaming or modifying the `received_message_id` column, require careful coordination with the `message_link` table to prevent breaking changes. - - - Liquibase changelog design must ensure that migrations are properly ordered and modular to accommodate these dependencies. - ### Related Issues - 753 From 749504b5aa674dff844dab697bc8d0a155d2152c Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 4 Dec 2024 16:55:19 -0500 Subject: [PATCH 8/9] Updated 018 - updated risks - Changelog Structure and Dependencies --- adr/018-sql-migrations.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/adr/018-sql-migrations.md b/adr/018-sql-migrations.md index a6ebf386e..8ca4b6986 100644 --- a/adr/018-sql-migrations.md +++ b/adr/018-sql-migrations.md @@ -31,6 +31,7 @@ As part of having CI/CD pipelines we need tooling inside of the project to allow - **Flexibility:** Liquibase supports a wide range of databases and migration formats, making it adaptable for future needs. + ### Negative - **Learning Curve:** Teams unfamiliar with Liquibase may require time to learn its configuration and scripting syntax. @@ -49,12 +50,14 @@ As part of having CI/CD pipelines we need tooling inside of the project to allow - **Rollback Limitations:** Not all changes (e.g., destructive data operations) can be automatically rolled back, requiring careful planning for such scenarios. -- **Version Drift:** If environments are not consistently updated with migrations, discrepancies could arise between development, staging, and production. +- **Version Drift:** If environments are not consistently updated with migrations, discrepancies could arise between development, staging, and production. - **GitHub Action Reliability:** Relying on prebuilt GitHub Actions introduces a dependency on external tools, which may have compatibility or update issues over time. +- **Changelog Structure and Dependencies:** A poorly structured changelog can create challenges in managing references between tables. For example, if the `message_link` table references the `received_message_id` column in the `metadata` table, structural changes in `metadata` could require significant changes to the migrations, increasing complexity. This risk can make future schema modifications or rollbacks harder to implement without breaking dependencies https://docs.liquibase.com/start/design-liquibase-project.html. + ### Related Issues - 753 From 59aea242840bdfee5dcfa5948435cca4a2a4c81e Mon Sep 17 00:00:00 2001 From: Tiffini Johnson Date: Wed, 4 Dec 2024 17:31:19 -0500 Subject: [PATCH 9/9] Updated 019 - added suggested risks - added suggested risk language in regards to adding another table increases potential rollback complexity --- adr/019-link-message-ids.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/adr/019-link-message-ids.md b/adr/019-link-message-ids.md index 52b9c5e31..4d2c82703 100644 --- a/adr/019-link-message-ids.md +++ b/adr/019-link-message-ids.md @@ -14,7 +14,7 @@ Accepted As part of the requirement to link an order with its corresponding result(s), we need to match fields in the `metadata` table to find the messages to link. Considering that the matching fields may change in the future, we want to store the linked IDs to preserve the relationship even if the matching fields change. -We decided to create a new table for this purpose, but we also discussed the option to instead add a new field to the `metadata` table with an array of link IDs. Even though this option would reduce the complexity in our database and code, we decided to create the new table because we'll soon need to pull data for each ID in order to populate the response from the `metadata` endpoint, which is a usecase that the separate table will fit better. If we find that the tradeoff is worth it, we may decide in the future to refactor the code and use the array field instead of the table. +We decided to create a new table for this purpose, but we also discussed the option to instead add a new field to the `metadata` table with an array of link IDs. Even though this option would reduce the complexity in our database and code, we decided to create the new table because we'll soon need to pull data for each ID in order to populate the response from the `metadata` endpoint, which is a use-case that the separate table will fit better. If we find that the tradeoff is worth it, we may decide in the future to refactor the code and use the array field instead of the table. ## Impact @@ -47,6 +47,7 @@ We decided to create a new table for this purpose, but we also discussed the opt - **Data Consistency:** Potential risks of mismatches between `metadata` and `message_link` entries due to bugs or incomplete synchronization. +- **Rollback Complexity:** Adding a new table increases the complexity of rollbacks during migrations, especially if records are linked inconsistently or if dependent data is removed from the `message_link` table prematurely. - **Performance Concerns:** Queries involving multiple joins with the `message_link` table might impact database performance if not optimized.