Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@yunks128
yunks128 authored Nov 30, 2023
1 parent 8ef6400 commit 75724cb
Showing 1 changed file with 117 additions and 129 deletions.
246 changes: 117 additions & 129 deletions docs/guides/software-lifecycle/continuous-testing/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,171 +1,143 @@
# Continuous Testing

## Introduction

Welcome to the Continuous Testing page. This page provides an overview of continuous testing, offers a ready-to-use continuous testing plan template, and addresses valuable feedback from our community members.
<pre align="center">This page provides an overview of continuous testing, offers a ready-to-use continuous testing plan template, and addresses valuable feedback from our community members.</pre>

## What is Continuous Testing?
## Introduction

Continuous Testing (CT) is integral to modern software development, automating systematic tests throughout the software lifecycle. It ensures your project's goals are met by providing rapid feedback on code quality and functionality. Beyond bug detection, CT validates performance, enhances security, and ensures seamless functionality.
**Background**: Continuous Testing (CT) is essential in addressing the challenges of modern software development, providing an automated and systematic approach to testing throughout the software lifecycle. In the ever-evolving landscape of software, CT goes beyond mere bug detection. It stands as a proactive solution that not only validates performance and enhances security but also ensures the seamless functionality of your software. By offering rapid feedback on code quality and functionality, CT becomes a crucial ally in meeting project goals, ultimately leading to more reliable and efficient software releases. Implementing CT promises not only the immediate benefits of bug identification but also the long-term advantages of improved performance, heightened security, and overall robust software functionality.

## Continuous Testing Philosophy
At the core of our continuous testing philosophy is the strategic utilization of open-source tools. We prioritize the adoption of a singular, comprehensive tool that seamlessly addresses various aspects across the continuous testing phases, optimizing our testing processes for efficiency and effectiveness.
**Philosophy**: At the core of our continuous testing philosophy is the strategic utilization of open-source tools. We prioritize the adoption of a singular, comprehensive tool that seamlessly addresses various aspects across the continuous testing phases, optimizing our testing processes for efficiency and effectiveness.

## Continuous Testing Phases

**1. Planning and Design:**
- Develop a comprehensive testing plan aligned with SLIM recommendations.
- Establish a shared plan for Continuous Testing.
- Define test requirements, specifying what to test and the reasons behind it.
- Identify roles for testers and implementers.
- Tools: TestRail (Test management tool for test case planning and design)
**Use Cases**:
- Ensuring consistent and thorough validation of new code changes
- Identifying and addressing potential performance bottlenecks
- Validating security measures
- Ensuring seamless integration of new features
- Providing timely feedback for developers to streamline the debugging process
- Automating repetitive and time-consuming testing tasks
- Facilitating quicker releases
- Fostering a culture of continuous improvement in software development

---

**2. Test Implementation:**
## Prerequisites
- CT Tools (These tools encompass both open-source and widely adopted commercial solutions)
- **Unit Testing:**
- Jest (Javascript), Pytest (Python)
- Jest (Javascript), Pytest (Python)
- **Integration Testing:**
- Jest (Javascript), Pytest (Python)
- Jest (Javascript), Pytest (Python)
- **Regression Testing:**
- Selenium, Playwright
- Selenium, Playwright
- **Performance Testing:**
- Apache JMeter
- Apache JMeter
- **Security Testing:**
- Dependabot, SonarQube
- Dependabot, SonarQube
- **Verification and Validation (V&V):**
- SonarQube, ESLint (Javascript), PyLint (Python)
- SonarQube, ESLint (Javascript), PyLint (Python)
- **UI/UX Testing**
- Selenium, Playwright
- Selenium, Playwright
- Understanding of the software development lifecycle
- Knowledge of the application architecture and dependencies
- An established set of test cases for initial implementation

---

**3. Improvement and Reporting:**
- Conduct retrospectives and feedback sessions for continuous enhancement.
- Regularly update tests to accommodate changes in functionality, new features, or optimizations.
- **Reporting and Analysis:**
- SonarQube
- **Continuous Improvement:**
- Jira
## Continuous Testing Phases

```mermaid
graph TD;
A
B
C
A --> B;
B --> C;
## Continuous Testing Plan Template
subgraph A[Planning and Design]
D[Develop a comprehensive testing plan aligned with SLIM recommendations]
E[Establish a shared plan for Continuous Testing]
F[Define test requirements, specifying what to test and the reasons behind it]
G[Identify roles for testers and implementers]
H[Tools: TestRail]
### 1. Project Overview
- **Project Name:** [INSERT PROJECT NAME HERE]
- **Project Description:** [INSERT SHORT PROJECT DESCRIPTION HERE]
- **Testing Lead:** [INSERT PROJECT LEAD NAME HERE]
end
subgraph B[Test Implementation]
I[Unit]
J[Integration]
K[Regression]
L[Performance]
M[Security]
N[V&V]
O[UI/UX Testing]
end
subgraph C[Improvement and Reporting]
P[Conduct retrospectives and feedback sessions for continuous enhancement]
Q[Regularly update tests to accommodate changes in functionality, new features, or optimizations]
R[Reporting and Analysis]
S[Continuous Improvement]
end
```

### 2. Test Requirements
- **Objective:** Why are we testing? [INSERT OBJECTIVE HERE]
- **Test Artifacts:** What are we testing? [INSERT ARTIFACTS HERE]

### 3. Testing Workflow Architecture
---
## Quick Start

#### [Insert your workflow diagram here]
To facilitate the implementation of Continuous Testing, we present a comprehensive template below. This template outlines key steps and considerations across various testing phases, along with recommended tools for each category. Whether you're a seasoned developer or new to continuous testing, this template serves as a valuable guide to streamline your testing processes and enhance software delivery. The tools recommended are chosen for their effectiveness, and they span both open-source and widely adopted commercial options, providing flexibility based on your specific needs and preferences.

### 4. Test Specifications
[Continuous Testing Plan Template](continuous-testing-plan-template.md)

Before customizing the table entries below, evaluate your project's needs and requirements. Add, remove, or modify rows in the table to best represent the phases, tests, tools, and people associated with your testing approach.
[Continuous Testing Plan Example](continuous-testing-plan-example.md)

#### [Insert your table here]

### 5. Implementation Checklist

- [ ] Continuous Testing Plan (this document) defined, including all relevant parts
- [ ] Security Testing tools, people, and tests implemented
- [ ] Unit Testing tools, people, and tests implemented
- [ ] All planned test cases defined in the test plan are implemented.
- [ ] Defects are reported and tracked.
- [ ] Test summary report is generated and shared with stakeholders.

### Example
```
### [INSERT PROJECT NAME HERE] Continuous Testing Plan
---

#### Introduction:
This document aims to provide a general approach to Continuous Testing for [INSERT PROJECT NAME HERE]. It encompasses planning, test phases, tool recommendations, and test specifications.
## Step-by-Step Guide

---
1. **Step 1: Define Your Continuous Testing Goals**
- Clearly outline the objectives and goals you aim to achieve with continuous testing. Identify key performance indicators and success criteria.

#### **1. Project Overview**
- **Project Name:** [INSERT PROJECT NAME HERE]
- **Project Description:** [INSERT SHORT PROJECT DESCRIPTION HERE]
- **Testing Lead:** [INSERT PROJECT LEAD NAME HERE]
2. **Step 2: Select Continuous Testing (CT) Tools**
- Choose a CT tool that aligns with your development environment.

#### **2. Test Requirements**
- **Objective:** Why are we testing? [INSERT OBJECTIVE HERE]
<!-- Example: "Ensure that the codebase remains free of critical vulnerabilities and maintains high performance." -->
3. **Step 3: Set Up Automated Unit Testing**
- Implement automated unit testing using suitable frameworks for your programming language (e.g., JUnit for Java, Pytest for Python).

- **Test Artifacts:** What are we testing? [INSERT ARTIFACTS HERE]
<!-- Example: "Web APIs, Front-end components, Database layer." -->
4. **Step 4: Implement Integration Testing**
- Set up automated integration tests to verify the interactions between different components. Use frameworks like Jest (JavaScript) or Pytest (Python).

5. **Step 5: Establish Regression Testing**
- Implement regression tests to ensure new code changes do not introduce issues in existing functionalities. Tools like Selenium and Playwright are popular for UI regression testing.

#### **3. Testing Workflow Architecture**
```
```mermaid
graph TD;
A[Code]
B
C
D
A --> B;
B --> C;
C --> D;
subgraph B[Local Testing Venue]
E[Unit Testing]
F[Security Testing]
G[Code Linting]
H[...]
end
subgraph C[Integration Testing Venue]
I[Regression Testing]
J[Performance Testing]
K[Security Testing]
L[UI Testing]
M[...]
end
6. **Step 6: Integrate Performance Testing**
- Use tools like Apache JMeter to perform load and performance testing. Ensure your system can handle various conditions and workloads.

subgraph D[Monitoring and Feedback]
N[Monitor Test Results]
O[Collect Metrics]
P[Generate Reports]
end
7. **Step 7: Implement Security Testing**
- Integrate security testing tools such as Dependabot for dependency scanning and SonarQube for code security analysis.

```
8. **Step 8: Verification and Validation (V&V)**
- Set up tools like ESLint (JavaScript), PyLint (Python), or other linters for static code analysis. Ensure code quality and adherence to coding standards.

```
#### **4. Test Specifications**
Before customizing the table entries below, evaluate your project's specific needs and requirements. Add, remove, or modify rows in the table to best represent the phases, tests, tools, and people associated with your testing approach.
| Phase | High-level Tests | Recommended Tool | Description | Starter Kit | Key People |
|-------|------------------|-----------------|-------------|-------------|------------|
| Security Testing | Vulnerability Scan | [GitHub Dependabot](https://dependabot.com/) | Monitors dependencies for known vulnerabilities. | [GitHub Docs](https://docs.github.com/en/code-security/supply-chain-security/keeping-your-dependencies-updated-automatically/about-dependabot-version-updates) | [INSERT NAME HERE] |
| | Secure Code Review | [SonarQube](https://www.sonarqube.org/) | Continuous inspection of code quality. | [SonarQube Docs](https://docs.sonarqube.org/latest/) | [INSERT NAME HERE] |
| Unit Testing | Function Validity | [JUnit](https://junit.org/junit5/) | Verifies individual units of Java software. | [JUnit 5 Guide](https://junit.org/junit5/docs/current/user-guide/) | [INSERT NAME HERE] |
| | Null Checks | [xUnit](https://xunit.net/) | Unit testing tool for .NET. | [xUnit.net Docs](https://xunit.github.io/docs/) | [INSERT NAME HERE] |
| Regression Testing | Feature Consistency | [Cucumber](https://cucumber.io/) | Supports behavior-driven development (BDD). | [Cucumber Starter](https://cucumber.io/docs/guides/10-minute-tutorial/) | [INSERT NAME HERE] |
| | User Flow Validation | [Selenium](https://www.selenium.dev/) | Ensures that new code changes do not adversely affect existing functionalities. | [Selenium HQ](https://www.selenium.dev/documentation/en/getting_started_with_webdriver/) | [INSERT NAME HERE] |
| Integration Testing | API Contract Validation | [REST Assured](https://rest-assured.io/) | Java DSL for simplifying testing of REST based services. | [REST Assured Guide](https://github.com/rest-assured/rest-assured/wiki/GettingStarted) | [INSERT NAME HERE] |
| | Data Flow Checks | [Postman](https://www.postman.com/) | Validates the interfaces and interactions between different software modules. | [Postman Learning Center](https://learning.postman.com/) | [INSERT NAME HERE] |
| Performance Testing | Load Testing | [JMeter](https://jmeter.apache.org/) | Measures system performance under various conditions. | [JMeter User Manual](https://jmeter.apache.org/usermanual/index.html) | [INSERT NAME HERE] |
| Requirements Verification & Validation | Requirement Traceability | [TestRail](https://www.gurock.com/testrail) | Ensures that the system meets the defined requirements. | [TestRail Docs](https://www.gurock.com/testrail/docs/) | [INSERT NAME HERE] |
| Deployment Testing | Cloud Deployment | [Terraform](https://www.terraform.io/) | Infrastructure as code for cloud provisioning. | [Terraform Get Started](https://learn.hashicorp.com/terraform/getting-started/install.html) | [INSERT NAME HERE] |
| | Chaos Testing | [Chaos Monkey](https://github.com/Netflix/chaosmonkey) | Simulates random failures to test system resilience. | [Chaos Monkey Wiki](https://github.com/Netflix/chaosmonkey/wiki) | [INSERT NAME HERE] |
#### 5. Implementation Checklist
- [ ] Continuous Testing Plan (this document) defined, including all relevant parts
- [ ] Security Testing tools, people, and tests implemented
- [ ] Unit Testing tools, people, and tests implemented
- [ ] All planned test cases defined in the test plan are implemented.
- [ ] Defects are reported and tracked.
- [ ] Test summary report is generated and shared with stakeholders.
```
9. **Step 9: UI/UX Testing**
- Implement automated UI/UX testing using tools like Selenium or Playwright to validate the user interface and overall user experience.

10. **Step 10: Reporting and Analysis**
- Utilize tools like SonarQube for comprehensive code quality and analysis reporting. Leverage reporting to identify areas for improvement.

11. **Step 11: Collaboration with Jira**
- Integrate with project management tools like Jira to enhance collaboration, track issues, and manage tasks efficiently.

## Feedback
12. **Step 12: Monitor and Adjust**
- Continuously monitor the performance of your continuous testing process. Make adjustments based on feedback, changing project requirements, and evolving best practices.

We value the feedback from our community members, and we've taken your input into account:


---

## Frequently Asked Questions (FAQ)

- **User Interface Testing:** The role of usability testing and user interface testing is an important consideration. We're actively exploring how to integrate this into our continuous testing model.

Expand All @@ -179,5 +151,21 @@ We value the feedback from our community members, and we've taken your input int

- **Iterative Implementation:** Iteration is key to successful continuous testing. We understand that it may take time to get it right, and we encourage iterative test implementation.

## Resources
[Use Cases and Frameworks](testing-frameworks)

---

## Credits

**Authorship**:
- [Kyongsik Yun](https://github.com/yunks128)

**Acknowledgements**:
* We express our gratitude to Rishi Verma and John Engelke for their invaluable reviews and insightful comments, which significantly contributed to the enhancement of this work.
* [Use Cases and Frameworks](testing-frameworks.md)


---

## Feedback and Contributions

We welcome feedback and contributions to help improve and grow this page. Please see our [contribution guidelines](https://nasa-ammos.github.io/slim/docs/contribute/contributing/).

0 comments on commit 75724cb

Please sign in to comment.