Document not found (404)
+This URL is invalid, sorry. Please use the navigation bar or search to continue.
+ +diff --git a/v4.7.2/.nojekyll b/v4.7.2/.nojekyll new file mode 100644 index 0000000000..f17311098f --- /dev/null +++ b/v4.7.2/.nojekyll @@ -0,0 +1 @@ +This file makes sure that Github Pages doesn't process mdBook's output. diff --git a/v4.7.2/404.html b/v4.7.2/404.html new file mode 100644 index 0000000000..7f83abf34b --- /dev/null +++ b/v4.7.2/404.html @@ -0,0 +1,219 @@ + + +
+ + +This URL is invalid, sorry. Please use the navigation bar or search to continue.
+ +The following sections give a conceptual overview of how Clair works internally.
+ + +Internal endpoints are underneath /api/v1/internal
and are meant for
+communication between Clair microservices. If Clair is operating in combo mode,
+these endpoints may not exist. Any sort of API ingress should disallow clients
+to talk to these endpoints.
We do not formally expose these APIs in our OpenAPI spec. +Further information and usage is an effort left to the reader.
+The update_diff/
endpoint exposes the api for diffing two update operations.
+This is used by the notifier to determine the added and removed vulnerabilities on security databsae update.
The update_operation
endpoint exposes the api for viewing updaters' activity.
+This is used by the notifier to determine if new updates have occured and triggers an update diff to see what has changed.
The affected_manifest
endpoint exposes the api for retreiving affected manifests given a list of Vulnerabilities.
+This is used by the notifier to determine the manifests that need to have a notification generated.
Previous versions of Clair used jwtproxy to gate authentication. For ease of +building and deployment, v4 handles authentication itself.
+Authentication is configured by specifying configuration objects underneath the
+auth
key of the configuration. Multiple authentication configurations may be
+present, but they will be used preferentially in the order laid out below.
Clair implements JWT-based authentication using a pre-shared key.
+The auth
stanza of the configuration file requires two parameters: iss
, which
+is the issuer to validate on all incoming requests; and key
, which is a base64
+encoded symmetric key for validating the requests.
auth:
+ psk:
+ key: >-
+ MDQ4ODBlNDAtNDc0ZC00MWUxLThhMzAtOTk0MzEwMGQwYTMxCg==
+ iss: 'issuer'
+
+
+ The Indexer service is responsble for "indexing a manifest".
+Indexing involves taking a manifest representing a container image and computing its constituent parts. The indexer is trying to discover what packages exist in the image, what distribution the image is derived from, and what package repositories are used within the image. Once this information is computed it is persisted in an IndexReport.
+The IndexReport is an intermediate data structure describing the contents of a container image. This report can be fed to a Matcher node for vulnerability analysis.
+ClairV4 treats all manifests and layers as content addressable. In the context of ClairV4 this means once we index a specific manifest we will not index it again unless it's required, and likewise with individual layers. This allows a large reduction in work.
+For example, consider how many images in a registry may use "ubuntu:artful" as a base layer. It could be a large majority of images if the developers prefer basing their images off ubuntu. Treating the layers and manifests as content addressable means we will only fetch and scan the base layer once.
+There are of course conditions where ClairV4 should re-index a manifest.
+When an internal component such as a package scanner is updated, Clair will know to perform the scan with the new package scanner. Clair has enough information to determine that a component has changed and the IndexReport may be different this time around.
+A client can track ClairV4's index_state
endpoint to understand when an internal component has changed and subsequently issue re-indexes. See our api guide to learn how to view our api specification.
In summary, you should understand that Indexing is the process Clair uses to understand the contents of layers.
+For a more indepth look at indexing check out the ClairCore Documentation
+ +A Matcher node is responsible for matching vulnerabilities to a provided IndexReport.
+Matchers by default are also responsible for keeping the database of vulnerabilities up to date. Matchers will typically run a set of Updaters which periodically probe their data sources for new contents, storing new vulnerabilities in the database when discovered.
+The matcher API is designed to be called often and will always provide the most up-to-date VulnerabilityReport when queried. This VulnerabilityReport summaries both a manifest's contents and any vulnerabilities affecting the contents.
+See our api guide to learn how to view our api specification and work with the Matcher api.
+A remote matcher behaves similarly to a matcher, except that it uses api calls to fetch vulnerability data for a provided IndexReport. +Remote matchers are useful when it is not possible to persist data from a given source into the database.
+The crda
remote matcher is responsible for fetching vulnerabilities from Red Hat Code Ready Dependency Analytics (CRDA).
+By default, this matcher serves 100 requests per minute.
+The rate-limiting can be lifted by requesting a dedicated API key, which is done via this form.
In summary you should understand that a Matcher node provides vulnerability reports given the output of an Indexing process. By default it will also run background Updaters keeping the vulnerability database up-to-date.
+For a more indepth look at indexing check out the ClairCore Documentation
+ +ClairV4 implements a notification system.
+The notifier service will keep track of new security database updates and inform an interested client if new or removed vulnerabilites affect an indexed manifest.
+The interested client can subscribe to notifications via several mechanisms:
+Configuring the notifier is done via the yaml configuration.
+See the "Notifier" object in our config reference
+When the notifier becomes aware of new vulnerabilities affecting a previously indexed manifest, it will use the configured method(s) to issue notifications about the new changes. Any given notification expresses the most severe vulnerability discovered because of the change. This avoids creating a flurry of notifications for the same security database update.
+Once a client receives a notification, it should issue a new request against the matcher to receive an up-to-date vulnerability report.
+The notification schema is the JSON marshaled form of the following types:
+// Reason indicates the catalyst for a notification
+type Reason string
+const (
+ Added Reason = "added"
+ Removed Reason = "removed"
+ Changed Reason = "changed"
+)
+type Notification struct {
+ ID uuid.UUID `json:"id"`
+ Manifest claircore.Digest `json:"manifest"`
+ Reason Reason `json:"reason"`
+ Vulnerability VulnSummary `json:"vulnerability"`
+}
+type VulnSummary struct {
+ Name string `json:"name"`
+ Description string `json:"description"`
+ Package *claircore.Package `json:"package,omitempty"`
+ Distribution *claircore.Distribution `json:"distribution,omitempty"`
+ Repo *claircore.Repository `json:"repo,omitempty"`
+ Severity string `json:"severity"`
+ FixedInVersion string `json:"fixed_in_version"`
+ Links string `json:"links"`
+}
+
+See the "Notifier.Webhook" object in the config reference for complete configuration details.
+When you configure notifier for webhook delivery you provide the service with the following pieces of information:
+When the notifier has determined an updated security database has changed the affected status of an indexed manifest, it will deliver the following JSON body to the configured target:
+{
+ "notifiction_id": {uuid_string},
+ "callback": {url_to_notifications}
+}
+
+On receipt, the server can immediately browse to the URL provided in the callback field.
+The URL returned in the callback field brings the client to a paginated result.
+The callback endpoint specification follows:
+GET /notifier/api/v1/notification/{id}?[page_size=N][next=N]
+{
+ page: {
+ size: int, // maximum number of notifications in the response
+ next: string, // if present, the next id to fetch.
+ }
+ notifications: [ Notification… ] // array of notifications; max len == page.size
+}
+
+The GET callback request implements a simple bare-minimum paging mechanism.
+The "page_size" url param controls how many notifications are returned in a single page. +If not provided a default of 500 is used.
+The "next" url param informs Clair the next set of paged notifications to return. If not provided the 0th page is assumed.
+A page object accompanying the notification list specifies "next" and "size" fields.
+The "next" field returned in the page must be provided as the subsequent request's "next" url parameter to retrieve the next set of notifications.
+The "size" field will simply echo back the request page_size parameter.
+When the final page is served to the client the returned "page" data structure will not contain a "next" member.
+Therefore the following loop is valid for obtaining all notifications for a notification id in pages of a specified size.
+{ page, notifications } = http.Get("http://clairv4/notifier/api/v1/notifications/{id}?page_size=1000")
+
+while (page.Next != None) {
+ { page, notifications } = http.Get("http://clairv4/notifier/api/v1/notifications/{id}?next={page.Next},page_size=1000")
+}
+
+Note: If the client specifies a custom page_size it must specify this page_size on every request for accurate responses.
+While not mandatory, the client may issue a delete of the notification via a DELETE method. See api to view the delete api.
+Deleting a notification ID will clean up resources in the notifier quicker. Otherwise the notifier will wait a predetermined length of time before clearing delivered notifications from its database.
+See the "Notifier.AMQP" object in our config reference for complete configuration details.
+The notifier also supports delivering to an AMQP broker. With AMQP delivery you can control whether a callback is delivered to the broker or whether notifications are directly delivered to the queue.
+This allows the developer of the AMQP consumer to determine the logic of notification processing.
+Note that AMQP delivery only supports AMQP 0.x protocol (e.g. RabbitMQ). If you need to publish notifications on AMQP 1.x message queue (e.g. ActiveMQ), you can use STOMP delivery.
+If the notifier's configuration specifies direct: true
for AMQP, notifications will be delivered directly to the configured exchange.
When direct
is set, the rollup
property may be set to instruct the notifier to send a max number of notifications in a single AMQP message. This allows a balance between size of the message and number of messages delivered to the queue.
The notifier has a testing mode enabled when it sees the "NOTIFIER_TEST_MODE" environment variable set. It can be set to any value as we only check to see if it exists.
+When this environment variable is set, the notifier will begin sending fake notifications to the configured delivery mechanism every "poll_interval" interval. This provides an easy way to implement and test new or existing deliverers.
+The notifier will run in this mode until the environment variable is cleared and the service is restarted.
+ +Clair utilizes go packages we call "updaters" that encapsulate the logic of +fetching and parsing different vulnerability databases. Updaters are usually +pared with a matcher to interpret if and how any vulnerability is related to a +package.
+Operators may wish to update the vulnerability database less frequently or not +import vulnerabilities from databases that they know will not be used.
+Updaters can be configured by updaters
key at the top of the configuration. If
+updaters are being run automatically within the matcher processes, as is the
+default, the period for running updaters is configured under the matcher's
+configuration stanza.
Specific sets of updaters can be selected by the sets
list. If not present,
+the defaults of all upstream updaters will be used.
updaters:
+ sets:
+ - rhel
+
+Configuration for specific updaters can be passed by putting a key underneath
+the config
member of the updaters
object. The name of an updater may be
+constructed dynamically; users should examine logs to double-check names.
+The specific object that an updater expects should be covered in the updater's
+documentation.
For example, to have the "rhel" updater fetch a manifest from a different +location:
+updaters:
+ config:
+ rhel:
+ url: https://example.com/mirror/oval/PULP_MANIFEST
+
+For additional flexibility, Clair supports running updaters in a different
+environment and importing the results. This is aimed at supporting installations
+that disallow the Clair cluster from talking to the Internet directly. An update
+procedure needs to arrange to call the relevant clairctl
command in an
+environment with access to the Internet, move the resulting artifact across the
+airgap according to site policy, and then call the relevant clairctl
command
+to import the updates.
For example:
+# On a workstation, run:
+clairctl export-updaters updates.json.gz
+
+# Move the resulting file to a place reachable by the cluster:
+scp updates.json.gz internal-webserver:/var/www/
+
+# On a pod inside the cluster, import the file:
+clairctl import-updaters http://web.svc/updates.json.gz
+
+Note that a configuration file is needed to run these commands.
+Matcher processes should have the disable_updaters
key set to disable
+automatic updaters running.
matcher:
+ disable_updaters: true
+
+indexer:
+ airgap: true
+
+indexer:
+ scanner:
+ package:
+ name:
+ key: value
+ repo:
+ name:
+ key: value
+ dist:
+ name:
+ key: value
+
+
+ The following sections provides information on how to contribute to Clair.
+The Clair project utilizes well structured commits to keep the history useful and help with release automation. +We suggest signing off on your commits as well.
+A typical commit will take on the following structure:
+<scope>: <subject>
+
+<body>
+Fixes #1
+Pull Request #2
+
+Signed-Off By: <email>
+
+The header of the commit is regexp checked before commit and your commit will be kicked back if it does not conform.
+This is the section of code this commit influences.
+You will often see scopes such as "notifier", "auth", "chore", "cicd".
+We use this field to group commits by scope in our automated changelog generation.
+It would be wise to take a look at our changelog before contributing to get an idea of the common scopes we use.
+Subject is a short and concise summary of the change the commit is introducing. It should be a sentence fragment without starting capitalization and ending punctuation and limited to about 60 characters, to allow for the scope prefix and decoration in the git log.
+Body should be full of detail.
+Explain what this commit is doing and why it is necessary.
+You may include references to issues and pull requests as well. Our automated changelog process will discover references prefixed with "Fixes", "Closed" and "Pull Request"
+ +Clair releases are cut roughly every three months and actively maintained for +six.
+This means that bugfixes should be landed on master
(if applicable) and then
+marked for backporting to a minor version's release branch. The process for
+doing this is not yet formalized.
When cutting a new minor release, two things need to be done: creating a tag and +creating a release branch. This can be done like so:
+git tag -as v4.x.0 HEAD
+git push upstream HEAD:release-4.x tag v4.x.0
+
+Then, a "release" needs to be created in the Github UI using the created tag.
+A patch release is just like a minor release with the caveat that minor version +tags should only appear on release branches and a new branch does not need to +be created.
+git checkout release-4.x
+git tag -as v4.x.1 HEAD
+git push upstream tag v4.x.1
+
+Then, a "release" needs to be created in the Github UI using the created tag.
+Clair's artifact release process is automated and driven off the releases in +Github.
+Publishing a new release in the Github UI automatically triggers the creation of
+a complete source archive and a container. The archive is attached to the
+release, and the container is pushed to the
+quay.io/projectquay/clair
+repository.
This is all powered by a Github Action in .github/workflows/cut-release.yml
.
The following sections provide instructions on accomplish specific goals in Clair.
+ + +