Skip to content

Commit

Permalink
Add a few guides to UI Coverage
Browse files Browse the repository at this point in the history
  • Loading branch information
jennifer-shehane committed Dec 30, 2024
1 parent 55f4e91 commit 4910251
Show file tree
Hide file tree
Showing 16 changed files with 351 additions and 12 deletions.
2 changes: 1 addition & 1 deletion docs/ui-coverage/configuration/_category_.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"label": "Configuration",
"position": 30
"position": 40
}
20 changes: 9 additions & 11 deletions docs/ui-coverage/get-started/introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -42,16 +42,14 @@ From there, you can easily customize reports to fit your needs with flexible con

## How it Works

:::caution
UI Coverage provides an interactive, visual map of test coverage for your application, powered by Cypress [Test Replay](/cloud/features/test-replay) (requires Cypress v13+). These reports are automatically generated for every unique state reached during your Cypress tests, whether in end-to-end or component testing.

UI Coverage generates reports using [Test Replay](/cloud/features/test-replay) data and requires Cypress v13+.
- **Effortless Setup**: No extra configuration is required. UI Coverage uses the same capture protocol as Test Replay, so no additional code or configuration is needed.
- **Dynamic Coverage Mapping**: Each interactive element is identified and highlighted as tested or untested, giving you a clear view of your test coverage across all pages and components.
- **DOM Snapshots**: Each tested and untested element is accompanied by a full-page, inspectable DOM snapshot, highlighting the exact location and context of element.
- **Comprehensive Scoring**: A UI Coverage score is calculated by comparing tested elements to the total interactable elements in your application.
- **Actionable Reports**: Sortable and filterable views provide insights into which areas are tested and which need improvement.
- **Flexible configuration**: Customize and fine-tune UI Coverage to suit specific needs and scenarios like ignoring views or elements or grouping similar elements together.
- **Configurable CI Integration**: The [Results API](/ui-coverage/results-api) allows you to programmatically control your CI pipeline's behavior based on UI Coverage scores.

:::

UI Coverage creates a dynamic visual map of test coverage across every page of your application. Each interactive element is identified and highlighted as either tested or untested, providing a clear and actionable overview of your test coverage.

UI Coverage captures "snapshots" of every page visited by your Cypress tests (and every component mounted in Component Testing). Snapshots represent all of the unique states found across your test run, and these states are analyzed to identify all of the unique elements that make up your application.

As a result, a UI Coverage score is generated. This score quantifies coverage across your application by measuring the ratio of the unique elements that have been tested to the total unique elements that are found across your application.

Because Cypress UI Coverage uses data captured through Cypress Test Replay, it is subject to Test Replay limitations. Read [How Test Replay Works](/cloud/features/test-replay#How-Test-Replay-Works) to discover which data is captured and which browsers are supported.
Read more about how it works in the [Core Concepts](/ui-coverage/core-concepts/interactivity) section.
4 changes: 4 additions & 0 deletions docs/ui-coverage/guides/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"label": "Guides",
"position": 30
}
131 changes: 131 additions & 0 deletions docs/ui-coverage/guides/address-coverage-gaps.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
---
sidebar_label: Address coverage gaps
title: 'Address coverage gaps | Cypress UI Coverage Documentation'
sidebar_position: 20
---

# Address coverage gaps

After [identifying test coverage gaps](/ui-coverage/guides/identify-coverage-gaps) using Cypress UI Coverage, the next step is to address these gaps to ensure your application is comprehensively tested. This guide outlines best practices and strategies for improving coverage and closing the identified gaps effectively.

## Step 1: Prioritize Gaps

Not all coverage gaps are equally critical. Use the information provided in the UI Coverage reports to prioritize testing efforts based on:

- **Critical Views**: Focus on views or components that represent high-priority user journeys, such as checkout pages, login screens, or submission forms.
- **Untested Links**: Address testing pages that are not being visited by your current test suite. You won't get insight into the untested elements on these pages until you visit them.

By prioritizing based on application context and business impact, you can address the most significant gaps first.

## Step 2: Enhance Test Coverage

Once you've identified priority areas, create or update tests to cover these gaps.

### Write Targeted Tests

Focus on creating tests that interact with the specific untested elements or pages identified in the coverage reports. For example:

```js
describe('Dashboard', () => {
it('Submits form on landing page', () => {
cy.visit('/request-trial')

// Interact with previously untested elements
cy.get('[data-cy="email"]').type('[email protected]')
cy.contain('Request Trial').click()
// UI Coverage will now surface the coverage of the thank you page
cy.url().should('include', '/thank-you')
});
});
```

### Cover Untested Links

Use the **Untested Links** section of the UI Coverage report to identify pages your tests haven't visited. Add navigation steps to your tests to include these pages:

```js
describe('Cover Untested Links', () => {
it('Visits untested pages', () => {
const untestedLinks = [
'/about-us',
'/contact',
'/pricing'
];

untestedLinks.forEach(link => {
cy.visit(link)
// Perform basic checks to ensure the page loads correctly
cy.get('h1').should('exist')
// UI Coverage will now surface the coverage of these pages
})
})
})
```

## Step 3: Refine Tests

### Ensure Element Visibility

Some gaps occur because elements are hidden or not rendered during tests. Update your tests to reveal these elements:

```js
cy.get('[data-cy="dropdown-toggle"]').click() // Reveal hidden elements
cy.get('[data-cy="dropdown-item"]').should('be.visible').click()
```

### Handle Dynamic Content

If coverage gaps are caused by dynamic or conditional rendering, ensure your tests account for various application states:

```js
// Login to render elements that only display after login
cy.get('[data-cy="login-button"]').click()
cy.get('[data-cy="user-profile"]')
```

## Step 4: Optimize Configuration

To maximize the effectiveness of UI Coverage, consider refining your configuration:

- Element Filters: Exclude irrelevant elements (e.g., placeholders, ads) from coverage reports.
- Significant Attributes: Define custom attributes that accurately identify elements.
- Attribute Filters: Remove auto-generated attributes to prevent redundant element identification.

Refer to the [Configuration Guide](/ui-coverage/configuration/overview) to learn how to customize UI Coverage to address these common needs:

- **Filtering**: Exclude specific elements or views from coverage reports.
- [Element Filters](/ui-coverage/configuration/elementfilters): Exclude specific elements from coverage reports.
- [View Filters](/ui-coverage/configuration/viewfilters): Exclude specific views from coverage reports.
- **Grouping**: Group similar elements together for easier analysis.
- [Elements](/ui-coverage/configuration/elements): Specify selectors to uniquely identify elements, even when they lack stable identifiers across snapshots.
- [Element Grouping](/ui-coverage/configuration/elementgrouping): Group similar elements together for easier analysis.
- [Views](/ui-coverage/configuration/views): Group views together based on defined URL patterns.
- **Defining Attribute Patterns**: Define patterns for identifying and grouping elements by attributes.
- [Attribute Filters](/ui-coverage/configuration/attributefilters): Specify patterns for attributes and their values that should not be used for identifying and grouping elements.
- [Significant Attributes](/ui-coverage/configuration/significantattributes): Define selectors to prioritize above the default attributes Cypress uses for the purpose of identification and grouping.

## Step 5: Iterate and Monitor

### Review Coverage Reports

After updating your tests, record them again to Cypress Cloud and review the new coverage reports. Verify that:

- Untested elements and links have been addressed.
- Overall coverage score has improved.

### Automate Coverage Enforcement

Use the [Results API](ui-coverage/results-api) to integrate coverage checks into your CI/CD pipeline. Set thresholds for coverage scores to enforce quality standards. This ensures your application maintains high test coverage over time.

## Step 6: Collaborate with Your Team

Improving test coverage often requires collaboration. Share insights from the UI Coverage reports with your team to:

- Prioritize testing efforts collectively.
- Align on critical areas that require attention.
- Distribute tasks for writing or updating tests.

## Next Steps

You can also leverage UI Coverage to reduce test duplication to optimize your test suite further. Learn how to [reduce test duplication](/ui-coverage/guides/reduce-test-duplication) with UI Coverage to streamline your testing process.

127 changes: 127 additions & 0 deletions docs/ui-coverage/guides/identify-coverage-gaps.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
---
sidebar_label: Identify coverage gaps
title: 'Identify coverage gaps | Cypress UI Coverage Documentation'
sidebar_position: 10
---

# Identify coverage gaps

Understanding your application's test coverage is crucial for ensuring quality and reliability. Cypress's UI Coverage tool provides insights into which parts of your application are tested and highlights untested areas. This guide will help you get started with UI Coverage to identify and address coverage gaps effectively.

## Step 1: Run Tests

To identify coverage gaps, you need to first run and record Cypress tests to the Cloud. If you're new to Cypress, refer to the [Cypress documentation](/app/end-to-end-testing/writing-your-first-end-to-end-test) to get started with writing tests.

### Example: Automated Sitemap-Based Testing

If your project lacks existing Cypress tests, a common approach is to understand test coverage from a sitemap or an array of target URLs. These URLs can be used to perform light interactions and capture the initial gaps in testing. Below is an example of how to automate this process by using a sitemap:

```js
describe('UI Coverage Scan', () => {
it('Checks ui coverage with the sitemap.xml', () => {
cy.request('https://<YOUR_WEBSITE>/sitemap.xml').then((response) => {
const xmlString = response.body
const parser = new DOMParser()
(loc) => loc.textContent
})
Cypress._.each(URLs, (URL) => {
cy.visit(URL)
})
})
})
```

## Step 2: Review Coverage Reports

Once your tests have recorded to Cypress Cloud, you can analyze the coverage reports to identify gaps. Click on the runs in your project in [Cypress Cloud](https://on.cypress.io/cloud) to access the UI Coverage reports. This report provides a visual representation of your application's test coverage, highlighting tested and untested elements.

### Overall Score

The first metric to review is the **overall coverage score**. This score is calculated by comparing the number of tested elements to the total number of [interactive elements](ui-coverage/core-concepts/interactivity) in your application. A higher score indicates better coverage, while a lower score indicates areas that need additional testing. The score will display on the runs page and within individual runs.

<DocsImage
src="/img/ui-coverage/guides/cypress-ui-coverage-runs-list-with-uicov-score.png"
alt="Cypress Cloud screenshot cropping to show a list of 4 runs with the details of the run like git commit, committer, branch, and the UI Coverage score of 11%"
/>

### Views

Within a run's **UI Coverage** tab, you'll find a **Views** section. Views represent different pages or components of your application. Each view in the list displays the URL or component path, the number of snapshots, the number of untested and tested elements in that view, and the coverage score.

<DocsImage
src="/img/ui-coverage/guides/cypress-ui-coverage-list-of-views.png"
alt="Cypress Cloud screenshot cropping to show a list of views within the UI Coverage tab including a lefthand navigation, a list of views and a filter"
/>

#### View Drilldown

Clicking into a view shows a detailed breakdown of the tested and untested elements within that view. You can use this information to inspect the DOM snapshot of the element using your browser's developer tools and understand the context of any coverage gaps. The view includes:

- **Untested Elements**: A list of interactive elements that were not interacted with during the test run.
- **Tested Elements**: A list of interactive elements that were interacted with during the test run.
- **DOM Snapshot**: A full-page, inspectable DOM snapshot of the view as it appeared during the test run. Tested elements highlight as green, while untested elements highlight as red.
- **Snapshot Navigation**: Navigate between snapshots to see the state of the view at different points during the test run.
- **Snapshot Coverage Score**: The coverage score for the specific snapshot based on the number of tested elements.
- **Test Replay**: A link to the Test Replay for the specific snapshot.
- **URL** and **Viewport size**: Metadata for the view.

<DocsImage
src="/img/ui-coverage/guides/cypress-ui-coverage-view-drilldown.png"
alt="Cypress Cloud screenshot cropping a single view within the cypress-documentation project showing the view's URL, the list of tested and untested elements and a DOM snapshot of the view"
/>

### Untested links

In the **UI Coverage** tab, the **Untested links** section lists all the links not interacted with during the test run. This can help you identify pages of your application that are not being visited and tested. Use this to identify unvisited pages and prioritize testing.

<DocsImage
src="/img/ui-coverage/guides/cypress-ui-coverage-untested-links.png"
alt="Cypress Cloud screenshot cropping the untested links section with a list of links that were not interacted with during the test run"
/>

#### Referrer Drilldown

Clicking a referrer link redirects you to the referrer's view, where the untested link is highlighted in red.

<DocsImage
src="/img/ui-coverage/guides/cypress-ui-coverage-untested-link-view.png"
alt="Cypress Cloud screenshot showing a view with the untested link highlighted in red"
/>


### Untested elements

The **Untested Elements** section in the **UI Coverage** tab lists all interactive elements not interacted with during the test run. This can help you identify specific elements that are not being tested across views. Use this information to prioritize testing for these elements.

<DocsImage
src="/img/ui-coverage/guides/cypress-ui-coverage-untested-elements-expanded.png"
alt="Cypress Cloud screenshot cropping the untested elements section with a list of elements that were not interacted with during the test run"
/>

#### View Drilldown

Clicking into an untested element's view shows a detailed breakdown of the element, including the element's selector, the number of times the element was interacted with, and the views without interactions. You can use this information to inspect the DOM snapshot of the element using your browser's developer tools and understand the context of any coverage gaps.

<DocsImage
src="/img/ui-coverage/guides/cypress-ui-coverage-untested-elements-view-drilldown.png"
alt="Cypress Cloud screenshot cropping a single untested element showing the element's selector, the number of times the element was interacted with, the views without interactions, and a full DOM snapshot of the view"
/>

## Step 3: Configure UI Coverage

While UI Coverage is designed to work seamlessly out of the box, there are instances where custom configuration may be necessary to address unique application structures, testing requirements, or edge cases. Refer to the [Configuration Guide](/ui-coverage/configuration/overview) to learn how to customize UI Coverage to address these common needs:

- **Filtering**: Exclude specific elements or views from coverage reports.
- [Element Filters](/ui-coverage/configuration/elementfilters): Exclude specific elements from coverage reports.
- [View Filters](/ui-coverage/configuration/viewfilters): Exclude specific views from coverage reports.
- **Grouping**: Group similar elements together for easier analysis.
- [Elements](/ui-coverage/configuration/elements): Specify selectors to uniquely identify elements, even when they lack stable identifiers across snapshots.
- [Element Grouping](/ui-coverage/configuration/elementgrouping): Group similar elements together for easier analysis.
- [Views](/ui-coverage/configuration/views): Group views together based on defined URL patterns.
- **Defining Attribute Patterns**: Define patterns for identifying and grouping elements by attributes.
- [Attribute Filters](/ui-coverage/configuration/attributefilters): Specify patterns for attributes and their values that should not be used for identifying and grouping elements.
- [Significant Attributes](/ui-coverage/configuration/significantattributes): Define selectors to prioritize above the default attributes Cypress uses for the purpose of identification and grouping.

## Next Steps

By leveraging these tools and techniques, you can effectively identify test coverage gaps. Next, read our guide on [addressing coverage gaps](/ui-coverage/guides/address-coverage-gaps) to ensure a robust and reliable application.
Loading

0 comments on commit 4910251

Please sign in to comment.