From 75482c2323b0a1798223f35e666d62f03edfb32d Mon Sep 17 00:00:00 2001 From: Charles Wilson <87022759+Motional-Charles-Wilson@users.noreply.github.com> Date: Mon, 11 Sep 2023 12:25:47 -0400 Subject: [PATCH 1/4] corrected error in markdown / corrected grammar --- Future.md | 54 +++++++++++++++++++++++++----------------------------- 1 file changed, 25 insertions(+), 29 deletions(-) diff --git a/Future.md b/Future.md index e52609fe..15723adf 100644 --- a/Future.md +++ b/Future.md @@ -1,57 +1,53 @@ # The Future of SARIF - -The SARIF project started with a relatively simple mission... define a standard format to enable the interchange of static analysis results. However, the ecosystem of software analysis tools has grown dramatically and there are now a broad variety of ways to analyse software. For example: + +The SARIF project started with a relatively simple mission, define a standard format to enable the interchange of static analysis results. However, the ecosystem of software analysis tools has grown dramatically and there are now a broad variety of ways to analyse software. For example: + * Static Analysis * Dynamic Analysis * Interactive Analysis * Runtime Protection * Observability -* Fuzzing +* Fuzz Testing * Manual Penetration Testing * Manual Code Review * Bug Bounty Programs - -Ultimately, the SARIF standard would enable the automatic exchange of results from all types of software analysis tools. This page details some of the efforts and investigations that the SARIF team is planning in order to achieve that goal. +Ultimately, the SARIF standard would enable the automatic exchange of results from all types of software analysis tools. This page details some of the efforts and investigations that the SARIF team is planning in order to achieve that goal. + +## Add Support for Dynamic (DAST) and Interactive (IAST) Tools + +Modern software analysis often involves observation of running software. This can occur from the outside (dynamic) where the inputs and outputs of the software are manipulated and observed to identify issues with the software works. Or it can occur from within the software (interactive) where techniques like profiling and instrumentation are used to directly observe the software as it runs. Supporting these ways of analyzing software will require SARIF to support findings that include: -# Add Support for Dynamic (DAST) and Interactive (IAST) Tools - -Modern software analysis often involves observation of running software. This can occur from the outside (dynamic) where the inputs and outputs of the softare are manipulated and observed to identify issues with the software works. Or it can occur from within the software (interactive) where techniques like profiling and instrumentation are used to directly observe the software as it runs. Supporting these ways of analyzing software will require SARIF to support findings that include: * various types of input (such as HTTP requests and URL schemes) * various types of output (such as HTTP responses and JSON payloads) * stack traces * data flow and other application state (with real data captured at runtime) * information about interaction with backend systems - - -# Coverage Reporting - -For software analysis to be trusted, we need to understand the scope of and confidence in the testing performed. SARIF could do a better job at reporting exactly what coverage was achieved. For example, SARIF could include details of: -* the exact tests that are performed on the software -* the confidence that these tests have thorougly and correctly achieved their goal -* details of software that was not covered in the test for any reason (stack depth, library, plugin, dynamic, etc...) - -# Add Support for "Positive" and "Architecture" Findings +## Coverage Reporting -Rather than only reporting vulnerabilities, some software analysis tools identify other attributes of software that need to be specified in a machine readable format so that they can be interchanged with other tools. Some examples include: -* positive findings that capture evidence that a particular secure coding practice has been correctly followed -* findings about the software architecture, such as upstream technolology, exposed routes, attack surface, backend connections, frameworks, etc... -* metrics about the software, such as number of database queries, queues supported, socket connections, etc... +For software analysis to be trusted, we need to understand the scope of and confidence in the testing performed. SARIF could do a better job at reporting exactly what coverage was achieved. For example, SARIF could include details of: +* the exact tests that are performed on the software +* the confidence that these tests have thoroughly and correctly achieved their goal +* details of software that was not covered in the test for any reason
(stack depth, library, plugin, dynamic, ...) -# Integrate with SBOM (CycloneDX and SPDX) +## Add Support for "Positive" and "Architecture" Findings -SARIF has primarily focused on tools that look at code problems and SBOM standards have focused on issues related to dependencies, both components and services. Together, they provide a much more complete picture of software security. In fact, in most cases, understanding both code and dependencies is required for fully understanding the context and risk associated with an issue. So this work stream is to investigate how SARIF and SBOM standards can be generated, linked, and used together. The goal is to achieve that elusive 1+1=3 result. +Rather than only reporting vulnerabilities, some software analysis tools identify other attributes of software that need to be specified in a machine readable format so that they can be interchanged with other tools. Some examples include: +* positive findings that capture evidence that a particular secure coding practice has been correctly followed +* findings about the software architecture, such as upstream technology, exposed routes, attack surface, backend connections, frameworks, ... +* metrics about the software, such as number of database queries, queues supported, socket connections, ... -# Integrate with OpenTelemetry - -Software doesn't exist in a vacuum. Modern software often runs in a complex ecosystem of systems that have a wide range of security attributes. As software trends towards being deployed in smaller chunks (web app -> web api -> serverless function), the need to understand the context becomes increasingly critical. +## Integrate with SBOM (CycloneDX and SPDX) -One way to tackle this problem is to identify a way to link SARIF into a model of that ecosystem, such as what is generated by "observability" tools that leverage OpenTelemetry. Linking of this nature has benefits in both directions. People looking at the entire ecosystem can drill into SARIF details from the global picture. People more focused on individual software components can enrich SARIF results with an understanding of context. +SARIF has primarily focused on tools that look at code problems and SBOM standards have focused on issues related to dependencies, both components and services. Together, they provide a much more complete picture of software security. In fact, in most cases, understanding both code and dependencies is required for fully understanding the context and risk associated with an issue. So this work stream is to investigate how SARIF and SBOM standards can be generated, linked, and used together. The goal is to achieve that elusive **1 + 1 = 3** result. -For example, imagine that SARIF identifies a "critical" SQL injection problem in an application. Adding global context would enable adding risk factors that temper that result. The application might only contain read-only, public data. Or it might only be accessible by administrators. Or it might be protected by a runtime protection module. The contextual risk calculation may be only "low". +## Integrate with OpenTelemetry +Software doesn't exist in a vacuum. Modern software often runs in a complex ecosystem of systems that have a wide range of security attributes. As software trends towards being deployed in smaller chunks (web app -> web api -> serverless function), the need to understand the context becomes increasingly critical. +One way to tackle this problem is to identify a way to link SARIF into a model of that ecosystem, such as what is generated by "observability" tools that leverage OpenTelemetry. Linking of this nature has benefits in both directions. People looking at the entire ecosystem can drill into SARIF details from the global picture. People more focused on individual software components can enrich SARIF results with an understanding of context. +For example, imagine that SARIF identifies a "critical" SQL injection problem in an application. Adding global context would enable adding risk factors that temper that result. The application might only contain read-only, public data. Or it might only be accessible by administrators. Or it might be protected by a runtime protection module. The contextual risk calculation may be only "low". From 5b136628f49dd404d5f5cac8e9707d351d183bb8 Mon Sep 17 00:00:00 2001 From: Charles Wilson <87022759+Motional-Charles-Wilson@users.noreply.github.com> Date: Mon, 11 Sep 2023 13:04:59 -0400 Subject: [PATCH 2/4] added SARIF Field Compatibility table --- Future.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/Future.md b/Future.md index 15723adf..c6013d83 100644 --- a/Future.md +++ b/Future.md @@ -14,6 +14,20 @@ The SARIF project started with a relatively simple mission, define a standard fo Ultimately, the SARIF standard would enable the automatic exchange of results from all types of software analysis tools. This page details some of the efforts and investigations that the SARIF team is planning in order to achieve that goal. +## SARIF Field Compatibility + +The following table shows mapping between current SARIF terminology as used for static analysis and that of other possible use cases. + +| Process | Context | Run | Test | Location | Issue | +|---------|---------|-----|------|----------|-------| +| Static Analysis | Common | Common | Checker | File / Line | Violation | +| Threat Modeling | Common | Common | Rule | Graph Edge | Violation +| Fuzz Testing | Common | Common | Checker | File / Line | Fault +| Dynamic Analysis | Common | Common | Checker | (variable resolution location) | Failure | +| Penetration Testing | Common | Common | Test | Test Step | Failure + +**Note:** Both **contest** and **run** have direct mapping into all of the above domains. + ## Add Support for Dynamic (DAST) and Interactive (IAST) Tools Modern software analysis often involves observation of running software. This can occur from the outside (dynamic) where the inputs and outputs of the software are manipulated and observed to identify issues with the software works. Or it can occur from within the software (interactive) where techniques like profiling and instrumentation are used to directly observe the software as it runs. Supporting these ways of analyzing software will require SARIF to support findings that include: From 0856c22c251c93d80cadb7bf505f86a1247d9ab4 Mon Sep 17 00:00:00 2001 From: Charles Wilson <87022759+Motional-Charles-Wilson@users.noreply.github.com> Date: Thu, 11 Jan 2024 08:14:04 -0500 Subject: [PATCH 3/4] added section for acronym change / added section for domain-specific schema subsetting --- Future.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/Future.md b/Future.md index c6013d83..b663ddec 100644 --- a/Future.md +++ b/Future.md @@ -14,6 +14,10 @@ The SARIF project started with a relatively simple mission, define a standard fo Ultimately, the SARIF standard would enable the automatic exchange of results from all types of software analysis tools. This page details some of the efforts and investigations that the SARIF team is planning in order to achieve that goal. +## Acronym Change + +With an expanded scope the **SARIF** acronym's underlying expansion would change from **S**tatic **A**nalysis **R**esults **I**nterchange **F**ormat to **S**ystematic **A**nalysis **R**esults **I**nterchange **F**ormat. + ## SARIF Field Compatibility The following table shows mapping between current SARIF terminology as used for static analysis and that of other possible use cases. @@ -65,3 +69,11 @@ Software doesn't exist in a vacuum. Modern software often runs in a complex ecos One way to tackle this problem is to identify a way to link SARIF into a model of that ecosystem, such as what is generated by "observability" tools that leverage OpenTelemetry. Linking of this nature has benefits in both directions. People looking at the entire ecosystem can drill into SARIF details from the global picture. People more focused on individual software components can enrich SARIF results with an understanding of context. For example, imagine that SARIF identifies a "critical" SQL injection problem in an application. Adding global context would enable adding risk factors that temper that result. The application might only contain read-only, public data. Or it might only be accessible by administrators. Or it might be protected by a runtime protection module. The contextual risk calculation may be only "low". + +## Provide Domain-specific Schema Subsets + +In order to reduce the overhead of working with the complete SARIF schema and also accommodate the diverse needs of automated software pipelines, domain-specific schema subsets would be created. These would be strict subsets in that they would conform to the master SARIF schema. + +Schema subsetting would also allow for the adoption of SARIF in these new domains as the weight of the current schema acts as a barrier to entry. + +Additionally, these schema subsets should accelerate the creation of SARIF data manipulation within the build pipeline. From 3a5443367779a47a43c1954fdc0ff78d1c145d7f Mon Sep 17 00:00:00 2001 From: Charles Wilson <87022759+Motional-Charles-Wilson@users.noreply.github.com> Date: Thu, 11 Jan 2024 12:32:11 -0500 Subject: [PATCH 4/4] Update Future.md removed domain-specific subsetting --- Future.md | 8 -------- 1 file changed, 8 deletions(-) diff --git a/Future.md b/Future.md index b663ddec..537c7f1c 100644 --- a/Future.md +++ b/Future.md @@ -69,11 +69,3 @@ Software doesn't exist in a vacuum. Modern software often runs in a complex ecos One way to tackle this problem is to identify a way to link SARIF into a model of that ecosystem, such as what is generated by "observability" tools that leverage OpenTelemetry. Linking of this nature has benefits in both directions. People looking at the entire ecosystem can drill into SARIF details from the global picture. People more focused on individual software components can enrich SARIF results with an understanding of context. For example, imagine that SARIF identifies a "critical" SQL injection problem in an application. Adding global context would enable adding risk factors that temper that result. The application might only contain read-only, public data. Or it might only be accessible by administrators. Or it might be protected by a runtime protection module. The contextual risk calculation may be only "low". - -## Provide Domain-specific Schema Subsets - -In order to reduce the overhead of working with the complete SARIF schema and also accommodate the diverse needs of automated software pipelines, domain-specific schema subsets would be created. These would be strict subsets in that they would conform to the master SARIF schema. - -Schema subsetting would also allow for the adoption of SARIF in these new domains as the weight of the current schema acts as a barrier to entry. - -Additionally, these schema subsets should accelerate the creation of SARIF data manipulation within the build pipeline.