Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLORA-61] Start the data model for security advisories #762

Merged
merged 1 commit into from
Nov 13, 2024

Conversation

tchoutri
Copy link
Contributor

@tchoutri tchoutri commented Oct 11, 2024

Proposed changes

This PR introduces the initial data model for security advisories.

The database components are as follow:

  • security_advisories, which encode most of the metadata, and to where affected packages refer
  • affected_packages represent the packages affected by an advisory, pointing to the advisory. Affected packages have one or multiple affected version ranges associated.
  • affected_version_ranges represent a link between an affected package, and the releases where the vulnerability is introduced and fixed.

Linked

Contributor checklist

@tchoutri tchoutri added the Hackathon Tickets that can be started or done during a Hackathon label Oct 11, 2024
Copy link
Contributor

@MangoIV MangoIV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this, perhaps we can continue on it later :)

src/advisories/Advisories/Model/Advisory/Types.hs Outdated Show resolved Hide resolved
src/datatypes/Advisories/HsecId/Orphans.hs Show resolved Hide resolved
src/datatypes/Advisories/HsecId/Orphans.hs Outdated Show resolved Hide resolved
src/datatypes/OSV/Reference/Orphans.hs Show resolved Hide resolved
migrations/20241011154110_create_affected_packages.sql Outdated Show resolved Hide resolved
@tchoutri
Copy link
Contributor Author

@MangoIV The standalone syntax for NFData does not work if the type does not implement Generic

@tchoutri tchoutri force-pushed the security-advisories-db-schema branch 2 times, most recently from c98b332 to 3623a55 Compare October 14, 2024 14:52
@MangoIV
Copy link
Contributor

MangoIV commented Oct 14, 2024

if the type does not implement Generic

I mistook that for granted.

@tchoutri tchoutri force-pushed the security-advisories-db-schema branch 4 times, most recently from 83ebac9 to 7f19d24 Compare October 17, 2024 21:37
@tchoutri tchoutri changed the title Start the data model for security advisories [FLORA-61] Start the data model for security advisories Oct 17, 2024
@tchoutri tchoutri requested a review from MangoIV October 17, 2024 22:13
@tchoutri
Copy link
Contributor Author

@TristanCacqueray @blackheaven @frasertweedale Please let me know what you think of this. For the moment I'm filing advisories of the GHC components (haddock included) in the ghc package.

Copy link
Contributor

@MangoIV MangoIV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had expected some place to asynchronously propagate a package to be affected to all its reverse dependencies, am I missing where that happens? Or is that a futurework?

I think the data model is fine etc. though.

With regards to actually using this, here's what I imagine:

  • a new internal endpoint (or whatever mechanism you use) to invoke the download of the advisories)
  • this spawns a process that starts to go through the advisories and for each of them, fetches the reverse deps which each gets a process itself that goes through their reverse deps
  • to make sure that not too many processes are spawned, you can use a QSem (quantity semaphore) which the processes block on if there's already enough of them chewing.

src/advisories/Advisories/Import.hs Show resolved Hide resolved
src/advisories/Advisories/Import.hs Show resolved Hide resolved
src/advisories/Advisories/Import.hs Show resolved Hide resolved
src/advisories/Advisories/Import/Error.hs Show resolved Hide resolved
test/fixtures/Cabal/hackage/hledged-web-1.23.cabal Outdated Show resolved Hide resolved
@tchoutri tchoutri force-pushed the security-advisories-db-schema branch 2 times, most recently from 3087f97 to 6be5f6a Compare November 8, 2024 21:16
@tchoutri
Copy link
Contributor Author

tchoutri commented Nov 8, 2024

@MangoIV yes, the propagation of the information is the next PR, I just wanted to focus on the data model here. :)

Copy link
Contributor

@blackheaven blackheaven left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're very tightly coupled to hsec, but, aside of that, it seems coherent.

Failure failures ->
throwError failures
Success advisories ->
forM_ advisories $ \advisory -> processAdvisory advisory
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
forM_ advisories $ \advisory -> processAdvisory advisory
mapM_ processAdvisory advisories

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@blackheaven I'm interested in how I could become less coupled to hsec! :)

@tchoutri tchoutri force-pushed the security-advisories-db-schema branch 2 times, most recently from ed708a5 to 97426f4 Compare November 13, 2024 13:24
@tchoutri tchoutri added the squash+merge me Tell Mergify bot to squash and merge the PR label Nov 13, 2024
@tchoutri tchoutri force-pushed the security-advisories-db-schema branch 2 times, most recently from 676a8c3 to 3e40540 Compare November 13, 2024 14:08
@tchoutri tchoutri force-pushed the security-advisories-db-schema branch from 3e40540 to 29cd72d Compare November 13, 2024 14:08
@tchoutri tchoutri added merge me Tell Mergify bot to merge the PR and removed squash+merge me Tell Mergify bot to squash and merge the PR labels Nov 13, 2024
@mergify mergify bot merged commit 5bfd934 into development Nov 13, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Hackathon Tickets that can be started or done during a Hackathon merge me Tell Mergify bot to merge the PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Store HSEC advisories in the database
3 participants