[TRACE-X] ⭐🚀 Determination of the optimal resource setup for Trace-X

[mkanal]

Overview

Explain the topic in 2 sentences

The determination of the optimal resource setup for Trace-X involves assessing various factors such as workload, scalability, and availability to ensure efficient, reliable, and cost-effective performance. This effort will help balance the resources in a way that meets both operational and budgetary constraints.

What's the benefit?

Optimizing the resource setup will improve the overall performance of the Trace-X application, ensuring better response times, reliability, and scalability while also reducing unnecessary resource consumption and costs. It allows the system to handle peak loads efficiently and minimizes the risk of downtime.

What are the Risks/Dependencies ?

Risk of resource misconfiguration, which could lead to performance bottlenecks or under-utilized systems.

Detailed explanation

The optimal resource setup for Trace-X requires an in-depth analysis of the current resource allocation and usage patterns. The current configuration may not be fully optimized for peak performance, which could lead to inefficiencies or higher operational costs. The new setup should take into account factors such as the number of cores, memory allocation, storage needs, and the ability to scale resources based on demand. Thread pooling, database optimization, and the usage of a managed, scalable database instance are critical factors that need adjustment to improve performance and reliability.

Current implementation

Currently, Trace-X uses a default resource setup that has not been thoroughly optimized for workload variability or scalability. The existing database setup may not be fully optimized for high-demand scenarios, and the thread pool configurations may not efficiently handle peak loads.

Proposed improvements

The improvements proposed include identifying and documenting the optimal number of cores, memory, and storage for Trace-X, along with configuring thread pools to handle concurrent tasks more efficiently. Additionally, database settings will be tuned for performance under defined workloads, and Trace-X will shift to a managed, scalable database instance to better handle variable loads and ensure availability.

Feature Team

Contributor

• Cofinity-X @rogocof
• Cofinity-X @mkanal

Committer

• @ds-jhartmann
• @ds-mwesener
• @ds-lcapellino

---

User Stories

As a team,
I want to determine the optimal resource setup for the Trace-X application,
So that the application performs efficiently, reliably, and cost-effectively, taking into account various factors such as workload, scalability, availability, and budget constraints.

Acceptance Criteria

• Optimal resource setup for trace-X is known (cores, memory, storage requirements) including documentation and proof why we use this setup
• Optimal Thread Pooling and thread pool configurations
• Database settings are optimized for defines scenario
• Trace-X uses a managed scalable database instance

Test Cases

Test Case 1

Verify that the Trace-X application is running with the optimal resource setup in terms of CPU cores, memory, and storage.

Steps

1. Deploy Trace-X with the defined optimal resource setup (e.g., number of cores, memory, storage).
2. Simulate various workload scenarios, including peak and low usage periods.
3. Monitor system performance and resource utilization during each scenario.

Expected Result

1. Trace-X performs efficiently without resource overutilization or underutilization.
2. Response times remain within acceptable limits across different workloads.
3. No resource exhaustion (e.g., CPU or memory) occurs during peak loads, and resource consumption scales down appropriately during low usage.

Test Case 2

Verify that the thread pooling configuration efficiently handles concurrent requests and scales based on the workload.

Steps

1. Configure thread pools as per the proposed optimal settings.
2. Simulate multiple concurrent API requests with varying load levels (light, moderate, heavy).
3. Monitor system performance and response times.

Expected Result

1. Trace-X handles concurrent requests efficiently without delays or performance degradation.
2. Thread pooling dynamically adjusts to accommodate varying request volumes, preventing thread starvation or overflow.
3. Resource usage (CPU and memory) remains within optimal limits during heavy loads.

Architectural Relevance

The following items are ensured (answer: yes) after this issue is implemented:

• This feature aligns with our current architectural guidelines
  • Data Sovereignty: All data sharing activities across company boundaries follow the Catena-X Regulatory Framework, in particular the Data Exchange Governance, and the Dataspace Protocol via a compliant Connector (like the tractusx-edc or similar, see Connector KIT)
    • CX-0010 Business Partner Number
    • CX-0013 Identity of Member Companies
    • CX-0018 Data Space Connectivity (EDC)
    • CX-0049 DID Document Schema
    • CX-0050 Framework Agreement Credential
    • CX-0149 Verified Company Identity
  • Interoperability: Digital Twins are used (compliant to the Digital Twin KIT and the Industry Core KIT)
    • CX-0001 EDC Discovery API
    • CX-0002 Digital Twins in Catena-X
    • CX-0018 Data Space Connectivity (EDC)
  • Data Format:
    • The data model is based on a published Semantic Model
• The impact on the overall system architecture has been assessed. The Feature does not require changes to the architecture or any existing standard? Please have a look here on the overarching architecture
• Potential risks or conflicts with existing architecture has been assessed

Justification: (Fill this out, if at least one of the checkboxes above cannot be ticked. Contact the Architecture Management Committee to get an approval for the justification)

Additional information

• I am aware that my request may not be developed if no developer can be found for it. I'll try to contribute a developer (bring your own developer)

[stephanbcbauer]

Some hints from Release Management (@ther3sa) and Tractus-X Project Lead (@stephanbcbauer)

• Status currently in Backlog -> Since for Backlog some requirements are missing ... please update them
• Please add missing sections from the feature template, or fill them out
• Please add assignee (overall responsible person who drives the feature)

[mkanal]

Feature is moved from r25.3 to 25.6 due to other priorities.

[stephanbcbauer]

I deleted the milestone, since the milestone is set together in the open planning session. This means ⇾ commitment to work on the feature. THX

[stephanbcbauer]

was not presented in open planning. Nobody was available.