Add rule that new versions SHOULD only be released on significant changes

[bernhardreiter]

In the wild we saw new document versions without a change in contents.
For example this Red Hat VEX document:

/document/publisher/namespace https://www.redhat.com
/document/tracking/id CVE-2012-5842

has only a change in the version of the generator.

zdiff -u ./redhat.com.2024-01-11/white/2012/cve-2012-5842.json.gz ./redhat.com.2024-03-15/white/2012/cve-2012-5842.json.gz
--- /dev/fd/5   2024-06-06 17:24:16.585853570 +0200
+++ -   2024-06-06 17:24:16.591738917 +0200
@@ -37,12 +37,12 @@
     ],
     "title": "10.0.11) (MFSA 2012-91)",
     "tracking": {
-      "current_release_date": "2023-05-11T21:12:36+00:00",
+      "current_release_date": "2024-01-25T05:02:18+00:00",
       "generator": {
-        "date": "2023-10-12T18:25:44+00:00",
+        "date": "2024-01-25T05:02:18+00:00",
         "engine": {
           "name": "Red Hat SDEngine",
-          "version": "3.23.0"
+          "version": "3.26.1"
         }
       },
       "id": "CVE-2012-5842",
@@ -57,10 +57,15 @@
           "date": "2023-05-11T21:12:36+00:00",
           "number": "2",
           "summary": "Current version"
+        },
+        {
+          "date": "2024-01-25T05:02:18+00:00",
+          "number": "3",
+          "summary": "Last generated version"
         }
       ],
       "status": "final",
-      "version": "1"
+      "version": "3"
     }
   },
   "product_tree": {

this should not have triggered a new version of the document in my view.
Even if the new generator has changed things outside of the document (like doing a better signature) , a new document version will trigger CSAF Management Systems into flagging this as new information that would possibly to be reviewed . For larger documents it also wastes computing cycles and backup space, resources which environmentally concious software should preserve.

So a proposal could be to introduce a SHOULD rule to only for only releasing a new revision for significant changes.

For instance a less relevant typo in on of the free text sections should also not trigger a new document. It could wait for a significant change or a number of small changes.

[mprpic]

This is expected and not avoidable at this time. In the future, we'd like to refactor how we implement the revision history in our CSAF files to produce more meaningful revision history entries, but it's still a backloged task unfortunately.

This example is also a little misleading. This diff is meaningful because it reflects a partial update to how we report the revision history (adding another entry). In most cases, the files will only be regenerated when something significant changes. In some cases, we may need to regenerate all files after releasing certain bug fixes, so you may see some updated files even when those file contents don't change significantly. The significant changes will be found in other CSAF files, which are generated together with / as a set with these files. This has been the case for the past several rounds of bug fixes (all files must be regenerated afterwards). You can find details about recent changes to our CSAF files in our official Red Hat Security Data Changelog: https://access.redhat.com/articles/5554431.

Red Hat generates 20k+ CSAF documents and it's not practical to manually decide what is and isn't a relevant update to a file to be able to produce a perfect revision history, and re-generate a file only when needed. Though, if you have a proposal of how we could improve this (at scale), we're open to feedback (here, or [email protected], or Jira).

[bernhardreiter]

@mprpic thanks for looking at the example and responding.

I've suggested a SHOULD rule so that exceptions can happen.

As far as I can understand Red Hat is in the process of adapting CSAF as one of the early adopters and promoters. And while doing so is improving the processes. So it is okay if things like this happen in the phase.

For the standard it would be good to give a recommendation.

preventing the example?

Before doing the full regeneration, you will have done a test run of the new generator.
And there will be the source data somewhere.

Some ideas (though you have probably thought of them already):

• Keep last generation date and last change date in the source database. Only regenerate if last generation date is before last change.
• Compare a small sample of the test run documents to the real documents, if you see something like above, implement a migration rule for this run that does not do a new version if there is no other change. You could do so by having a blacklist for this very regeneration.
• Implement a check that compares the new test documents to the old one and only allow for substancial changes (could be a bit more work, but not so much)
• Have a button "publish" now for the editor in the source data. If this is not pressed, the change was trivial and there is no publication of the new revision (it would happen with the next button press).

[tschmidtb51]

An idea could be also to diff the two documents and generate the revision_history entry if something changed...