Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for summarized and scheduled notifications #3925

Open
wants to merge 94 commits into
base: feature-322-scheduled-notifications
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
fa06057
added javacron dependency
MM-msr Apr 29, 2024
30fd0cf
added scheduled properties in NotificationRule, added configurable de…
MM-msr Apr 29, 2024
02b815e
changed type of cron configuration to string
MM-msr Apr 29, 2024
bd1cddf
added fallback for last execution time if not set
MM-msr Apr 29, 2024
5ac8ade
moved scheduled properties from NotificationRule to new class
MM-msr May 2, 2024
465624f
added persistence entry for ScheduledNotificationRule
MM-msr May 2, 2024
5f9a6cf
added scheduled crud methods to query managers
MM-msr May 2, 2024
8edf14f
added api for scheduled notification rules
MM-msr May 2, 2024
69e29dd
added some minor validation in scheduled api
MM-msr May 3, 2024
49352eb
fixed wrong database usage (data stored in notificationrule table), w…
MM-msr May 3, 2024
53ae7c3
Updated NotificationQueryManager to use UTC time for ScheduledNotific…
MM-msr May 7, 2024
5b79e26
Add new methods for retrieving new policy violations and vulnerabilit…
MM-msr May 7, 2024
ee6745b
Added basic Task for sending scheduled notifications (originates main…
MM-msr May 7, 2024
e13092c
added update method for last execution after scheduled task completion
MM-msr May 8, 2024
d3d50be
fixed VulnerabilityQueryManager SQL query for new vulnerabilities
MM-msr May 10, 2024
3e50a58
added basic support for scheduled publishing in notification publishers
MM-msr May 13, 2024
5d42aba
Added API endpoints for filtering publishers
MM-msr May 13, 2024
0b4bc6f
Unique serialVersionUID for ScheduledNotificationRule instead of same…
MM-msr May 14, 2024
468bfd3
fixed setting last execution time on update
MM-msr May 14, 2024
96e0ae1
fixed wrong method usage for updating last execution time in QueryMan…
MM-msr May 15, 2024
ef9c21b
fixed last execution to only update after successful publishing
MM-msr May 15, 2024
d4e06e0
code cleanup
MM-msr May 16, 2024
fec62ef
abstracted NotificationRule with interface for reusing existing Publi…
MM-msr May 16, 2024
601e5fc
basic rebuild of scheduled publish task to match idea of multiple pub…
MM-msr May 16, 2024
ed8107b
null checks in query managers for new events since last scheduled exe…
MM-msr May 17, 2024
e321106
generation of basic notification content in task
MM-msr May 17, 2024
69cd738
fixed missing header part in Rule
MM-msr May 24, 2024
1992656
fixed query in policy and vulnerability querymanagers when project li…
MM-msr May 24, 2024
8a76884
changed retrieval of default publishers from db to support multiple d…
MM-msr May 24, 2024
d43990b
fixed missing detach for scheduled notification items
MM-msr May 24, 2024
62d77bf
added scheduled default publisher with testing email template, suppor…
MM-msr May 27, 2024
4b7e26c
modified scheduled task to deliver test data with new subject classes
MM-msr May 28, 2024
44e272f
added cron task management on CRUD operations with automatic re-sched…
MM-msr May 28, 2024
bba7f8f
removed test date in scheduled task
MM-msr May 28, 2024
2a828d1
fixed missing update of last execution time after successful publish
MM-msr May 28, 2024
5606fe4
initialize scheduled notification tasks at startup
MM-msr May 29, 2024
4fbcd2e
added option to run scheduled notification rule manually instant
MM-msr May 29, 2024
2dd4f71
support to read default cron expression from environment variables
MM-msr May 29, 2024
80b9e01
update last execution time of rule without publishing, if no errors o…
MM-msr May 29, 2024
70a4fbe
added informational logging
MM-msr May 29, 2024
54034ae
removed author tags
MM-msr May 29, 2024
10ebedc
removed unnecessary code in publisher task
MM-msr May 29, 2024
624a96b
moved notification title and content generation to NotificationUtil c…
MM-msr May 29, 2024
439b7fb
removed check for instant execution api payload to match UI changes (…
MM-msr May 30, 2024
8c57996
added json serializer for ZonedDateTime for better readability in api…
MM-msr May 31, 2024
6ead108
Merge branch 'msr-scheduled-tests' into msr-issue-322
MM-msr Jun 3, 2024
d6bec6b
added new data models to match new provided pebble template
MM-msr Jun 5, 2024
7e9f5c8
changed depending classes to use new template models
MM-msr Jun 5, 2024
74b757e
fixed wrong query in getting findings with since-date-filter
MM-msr Jun 5, 2024
9031bd9
fixed typo in overview model
MM-msr Jun 5, 2024
904a8a9
fixed StackOverflowException due to missing method definition
MM-msr Jun 6, 2024
49b597b
fixed ignore of suppressed violations, fixed error on pebble template…
MM-msr Jun 6, 2024
0328433
added child projects audit in scheduled notification mail
MM-msr Jun 6, 2024
bff4c79
ignore version label in template if not set
MM-msr Jun 6, 2024
c64efc1
fixed detach in scheduled task to avoid implicit modification of noti…
MM-msr Jun 6, 2024
66d7ac2
fixed determination of affected project in scheduled notification rul…
MM-msr Jun 7, 2024
4465e63
updated console default publisher and template to support scheduled n…
MM-msr Jun 10, 2024
30eb2b2
hide details part in mail if no new vulnerabilities were found
MM-msr Jun 10, 2024
82f2cd4
added new policy violation processing in scheduled notification, refa…
MM-msr Jun 11, 2024
b05e627
refactored scheduled models to retrieve data from db only once
MM-msr Jun 12, 2024
7f5602b
changed PolicyViolationDetails back to PolicyViolation base type to r…
MM-msr Jun 12, 2024
8d77dca
fixed typo
MM-msr Jun 12, 2024
c6cc570
added publisher tests
MM-msr Jun 13, 2024
b0b0d50
removed notification level in scheduled rule from persistence and usage
MM-msr Jun 13, 2024
1456ea6
minor refactoring
MM-msr Jun 13, 2024
a7523f8
removed unneeded tests
MM-msr Jun 13, 2024
defde9b
fixed Exception in PublishContext due missing notificationlevel
MM-msr Jul 2, 2024
d14613e
changed suppressed type in template models to bool due to better json…
MM-msr Jun 14, 2024
c900ff1
fixed json structure of template models
MM-msr Jun 14, 2024
a2d8b54
added documentation for notification models in templates
MM-msr Jun 14, 2024
5f0b493
changed notification title + content to support single occurrences
MM-msr Jun 14, 2024
bf70f30
fixed total vulnerabilities determination (without regarding last not…
MM-msr Jun 14, 2024
8f49c2b
added new publisher types in documentation
MM-msr Jun 14, 2024
d5dc87d
updated mail template
MM-msr Jun 14, 2024
f698d0c
added class comments for all relevant new introduced classes
MM-msr Jun 14, 2024
c63d4f7
removed debug code
MM-msr Jun 14, 2024
267f50d
fixed not resolvable characters in policy mail template part
MM-msr Jul 3, 2024
8038447
use system local time instead of UTC for last execution time in sched…
MM-msr Jul 3, 2024
575840c
use ISO string instead of manual date format string
MM-msr Jul 3, 2024
1aeab35
fixed test failures
MM-msr Jul 8, 2024
2504e6f
added tests, removed some unused imports in tests
MM-msr Jul 28, 2024
47f4a3e
fixed missing filter for suppressed findings
MM-msr Jul 28, 2024
4f9ef27
restructured scheduled payload to records with factory
MM-msr Jul 28, 2024
2bfd123
added mail content tests for vulnerabilities and policy violations
MM-msr Aug 1, 2024
ab6fda4
removed unnecessary database fetching
MM-msr Aug 1, 2024
c791f29
fixed enum and cron related test errors
MM-msr Aug 12, 2024
e3d25a7
fixed test error for invalid cron config (add mandatory setters)
MM-msr Sep 2, 2024
cec723a
ci-build fix
mge-mm Oct 31, 2024
531a1ca
test fixes
mge-mm Oct 31, 2024
c34d93d
ScheduledNotificationRuleResourceTest fix, commented createScheduledN…
mge-mm Nov 4, 2024
1f1a361
test and code changes for tests CI
mge-mm Nov 5, 2024
ccc6d19
SendmailPublisherTest fix
mge-mm Nov 6, 2024
847f5c7
Fix DefectDojo unit test cases
rbt-mm Nov 12, 2024
6379119
Fix test for ScheduledPolicyViolation mail
rbt-mm Nov 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
203 changes: 191 additions & 12 deletions docs/_docs/integrations/notifications.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,16 +61,18 @@ multiple levels, while others can only ever have a single level.
A notification publisher is a Dependency-Track concept allowing users to describe the structure of a notification (i.e. MIME type, template) and how to send a notification (i.e. publisher class).
The following notification publishers are included by default :

| Publisher | Description |
|------------|-----------------------------------------------------|
| Slack | Publishes notifications to Slack channels |
| Teams | Publishes notifications to Microsoft Teams channels |
| Mattermost | Publishes notifications to Mattermost channels |
| WebEx | Publishes notifications to Cisco WebEx channels |
| Webhook | Publishes notifications to a configurable endpoint |
| Email | Sends notifications to an email address |
| Console | Displays notifications on the system console |
| Jira | Publishes notifications to Jira |
| Publisher | Description |
| ----------------- | ---------------------------------------------------------------------------------------------- |
| Slack | Publishes notifications to Slack channels |
| Teams | Publishes notifications to Microsoft Teams channels |
| Mattermost | Publishes notifications to Mattermost channels |
| WebEx | Publishes notifications to Cisco WebEx channels |
| Webhook | Publishes notifications to a configurable endpoint |
| Email | Sends notifications to an email address |
| Console | Displays notifications on the system console |
| Jira | Publishes notifications to Jira |
| Scheduled Email | Sends a summary of all subscribed events since last notification to an email address |
| Scheduled Console | Displays a slim summary of all subscribed events since last notification to the system console |

### Templating

Expand All @@ -96,7 +98,7 @@ The template context is enhanced with the following variables :
> subject will be present at all times. Some fields are optional since the underlying fields in the datamodel are optional.
> The section below will describe the portfolio notifications in JSON format.

#### NEW_VULNERABILITY
#### NEW_VULNERABILITY (per event)
This type of notification will always contain:
* 1 component
* 1 vulnerability
Expand Down Expand Up @@ -160,6 +162,96 @@ This type of notification will always contain:

> The `cwe` field is deprecated and will be removed in a later version. Please use `cwes` instead.

#### NEW_VULNERABILITY (scheduled summary)

```json
{
"notification": {
"level": "INFORMATIONAL",
"scope": "PORTFOLIO",
"group": "NEW_VULNERABILITY",
"timestamp": "2024-05-16T23:26:22.961",
"title": "123 new Vulnerabilities in 45 components in Scheduled Rule 'ABC'",
"content": "Find below a summary of new vulnerabilities since 2024-05-16T00:00:00Z in Scheduled Notification Rule 'ABC'.",
"subject": {
"overview": {
"affectedProjectsCount": 7,
"newVulnerabilitiesCount": 123,
"affectedComponentsCount": 45,
"suppressedNewVulnerabilitiesCount": 0,
"newVulnerabilitiesBySeverity": {
"CRITICAL": 13,
"HIGH": 24,
"MEDIUM": 56,
"LOW": 10,
"INFO": 17,
"UNASSIGNED": 3
}
},
"summary": {
"projectSummaries": [
{
"project": {
"uuid": "6fb1820f-5280-4577-ac51-40124aabe307",
"name": "Acme Example",
"version": "1.0.0"
},
"summary": {
"newVulnerabilitiesBySeverity": {
"CRITICAL": 3,
"HIGH": 4,
"LOW": 2,
"INFO": 7
},
"totalProjectVulnerabilitiesBySeverity": {
"CRITICAL": 35,
"HIGH": 57,
"MEDIUM": 13,
"LOW": 105,
"INFO": 23,
"UNASSIGNED": 13
},
"suppressedNewVulnerabilitiesBySeverity": {
"HIGH": 2,
"LOW": 5,
"INFO": 1
}
}
}
]
},
"details": {
"projectDetails": [
{
"project": {
"uuid": "6fb1820f-5280-4577-ac51-40124aabe307",
"name": "Acme Example",
"version": "1.0.0"
},
"findings": [
{
"componentUuid": "4d0da61c-b462-4895-b296-da0b4bb34744",
"componentName": "axis",
"componentVersion": "1.4",
"componentGroup": "apache",
"vulnerabilitySource": "NVD",
"vulnerabilityId": "CVE-2012-5784",
"vulnerabilitySeverity": "MEDIUM",
"analyzer": "OSSINDEX_ANALYZER",
"attributionReferenceUrl": "https://ossindex.sonatype.org/vulnerability/CVE-2012-5784",
"attributedOn": "2024-05-16T12:34:39Z",
"analysisState": "IN_TRIAGE",
"suppressed": false
}
]
}
]
}
}
}
}
```

#### NEW_VULNERABLE_DEPENDENCY
This type of notification will always contain:
* 1 project
Expand Down Expand Up @@ -324,7 +416,7 @@ This type of notification will always contain:
}
```

#### POLICY_VIOLATION
#### POLICY_VIOLATION (per event)

```json
{
Expand Down Expand Up @@ -368,6 +460,93 @@ This type of notification will always contain:
}
```

#### POLICY_VIOLATION (scheduled summary)

```json
{
"notification": {
"level": "INFORMATIONAL",
"scope": "PORTFOLIO",
"group": "POLICY_VIOLATION",
"timestamp": "2022-05-12T23:07:59.611303",
"title": "2 new Policy Violations in 2 components in Scheduled Rule 'Policy Guard'",
"content": "Find below a summary of new policy violations since 2022-05-12T00:00:00Z in Scheduled Notification Rule 'Policy Guard'.",
"subject": {
"overview": {
"affectedProjectsCount": 1,
"newViolationsCount": 2,
"affectedComponentsCount": 2,
"suppressedNewViolationsCount": 0,
"newViolationsByRiskType": {
"LICENSE": 0,
"SECURITY": 0,
"OPERATIONAL": 2
}
},
"summary": {
"affectedProjectSummaries": [
{
"project": {
"uuid": "7a36e5c0-9f09-42dd-b401-360da56c2abe",
"name": "Acme Example",
"version": "1.0.0"
},
"summary": {
"newViolationsByRiskType": {
"OPERATIONAL": 2
},
"totalProjectViolationsByRiskType": {
"LICENSE": 5,
"OPERATIONAL": 2
},
"suppressedNewViolationsByRiskType": {
}
}
}
]
}
"details": {
"projectDetails": [
{
"project": {
"uuid": "7a36e5c0-9f09-42dd-b401-360da56c2abe",
"name": "Acme Example",
"version": "1.0.0"
},
"violations": [
{
"component": {
"uuid": "4e04c695-9acd-46fc-9bf6-ed23d7eb551e",
"group": "apache",
"name": "axis",
"version": "1.4"
},
"violation": {
"uuid": "c82fcb50-029a-4636-a657-96242b20680e",
"type": "OPERATIONAL",
"timestamp": "2022-05-12T20:34:46Z",
"policyCondition": {
"uuid": "8e5c0a5b-71fb-45c5-afac-6c6a99742cbe",
"subject": "COORDINATES",
"operator": "MATCHES",
"value": "{\"group\":\"apache\",\"name\":\"axis\",\"version\":\"*\"}",
"policy": {
"uuid": "6d4c7398-689a-4ec7-b5c5-9abb6b5393e9",
"name": "Banned Components",
"violationState": "FAIL"
}
}
}
}
]
}
]
}
}
}
}
```

#### USER_CREATED

```json
Expand Down
16 changes: 16 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@
<lib.cvss-calculator.version>1.4.2</lib.cvss-calculator.version>
<lib.owasp-rr-calculator.version>1.0.1</lib.owasp-rr-calculator.version>
<lib.cyclonedx-java.version>9.1.0</lib.cyclonedx-java.version>
<lib.jakarta-validation.version>3.0.2</lib.jakarta-validation.version>
<lib.greenmail.version>2.1.0</lib.greenmail.version>
<lib.jackson.version>2.18.0</lib.jackson.version>
<lib.jackson-databind.version>2.18.0</lib.jackson-databind.version>
Expand All @@ -114,17 +115,20 @@
<lib.pebble.version>3.2.2</lib.pebble.version>
<lib.protobuf-java.version>4.28.3</lib.protobuf-java.version>
<lib.resilience4j.version>2.2.0</lib.resilience4j.version>
<lib.swagger.version>2.2.25</lib.swagger.version>
<lib.swagger-parser.version>2.1.22</lib.swagger-parser.version>
<lib.system-rules.version>1.19.0</lib.system-rules.version>
<lib.testcontainers.version>1.20.3</lib.testcontainers.version>
<lib.wiremock.version>2.35.2</lib.wiremock.version>
<lib.woodstox.version>7.0.0</lib.woodstox.version>
<lib.junit.version>4.13.2</lib.junit.version>
<lib.junit-params.version>1.1.1</lib.junit-params.version>
<lib.signpost-core.version>2.1.1</lib.signpost-core.version>
<lib.httpclient.version>4.5.14</lib.httpclient.version>
<lib.httpclient5.version>5.4</lib.httpclient5.version>
<lib.log4j-over-slf4j.version>2.0.16</lib.log4j-over-slf4j.version>
<lib.org-kohsuke-github-api.version>1.323</lib.org-kohsuke-github-api.version>
<lib.com-asahaf-javacron.version>1.4.0</lib.com-asahaf-javacron.version>
<!-- JDBC Drivers -->
<lib.jdbc-driver.mssql.version>12.8.1.jre11</lib.jdbc-driver.mssql.version>
<lib.jdbc-driver.mysql.version>8.2.0</lib.jdbc-driver.mysql.version>
Expand Down Expand Up @@ -216,6 +220,12 @@
<scope>provided</scope>
</dependency>

<dependency>
<groupId>jakarta.validation</groupId>
<artifactId>jakarta.validation-api</artifactId>
<version>${lib.jakarta-validation.version}</version>
</dependency>

<dependency>
<groupId>com.github.package-url</groupId>
<artifactId>packageurl-java</artifactId>
Expand Down Expand Up @@ -411,6 +421,12 @@
<version>${lib.org-kohsuke-github-api.version}</version>
</dependency>

<dependency>
<groupId>com.asahaf.javacron</groupId>
<artifactId>javacron</artifactId>
<version>${lib.com-asahaf-javacron.version}</version>
</dependency>

<!-- Test Dependencies -->
<dependency>
<groupId>junit</groupId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ public enum ConfigPropertyConstants {
ACCESS_MANAGEMENT_ACL_ENABLED("access-management", "acl.enabled", "false", PropertyType.BOOLEAN, "Flag to enable/disable access control to projects in the portfolio", true),
NOTIFICATION_TEMPLATE_BASE_DIR("notification", "template.baseDir", SystemUtils.getEnvironmentVariable("DEFAULT_TEMPLATES_OVERRIDE_BASE_DIRECTORY", System.getProperty("user.home")), PropertyType.STRING, "The base directory to use when searching for notification templates"),
NOTIFICATION_TEMPLATE_DEFAULT_OVERRIDE_ENABLED("notification", "template.default.override.enabled", SystemUtils.getEnvironmentVariable("DEFAULT_TEMPLATES_OVERRIDE_ENABLED", "false"), PropertyType.BOOLEAN, "Flag to enable/disable override of default notification templates"),
NOTIFICATION_CRON_DEFAULT_EXPRESSION("notification", "cron.default.expression", SystemUtils.getEnvironmentVariable("DEFAULT_SCHEDULED_CRON_EXPRESSION", "0 12 * * *"), PropertyType.STRING, "The default interval of scheduled notifications as cron expression"),
TASK_SCHEDULER_LDAP_SYNC_CADENCE("task-scheduler", "ldap.sync.cadence", "6", PropertyType.INTEGER, "Sync cadence (in hours) for LDAP"),
TASK_SCHEDULER_GHSA_MIRROR_CADENCE("task-scheduler", "ghsa.mirror.cadence", "24", PropertyType.INTEGER, "Mirror cadence (in hours) for Github Security Advisories"),
TASK_SCHEDULER_OSV_MIRROR_CADENCE("task-scheduler", "osv.mirror.cadence", "24", PropertyType.INTEGER, "Mirror cadence (in hours) for OSV database"),
Expand Down
Loading
Loading