-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: naveensrinivasan <[email protected]>
- Loading branch information
1 parent
d8a01c9
commit 76a0670
Showing
3 changed files
with
108 additions
and
23 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
name: Build and Push Docker Image | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
|
||
jobs: | ||
build-and-test: | ||
runs-on: ubuntu-latest | ||
|
||
permissions: | ||
contents: read | ||
packages: write | ||
|
||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v2 | ||
- name: Set up Docker | ||
uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db # v3.6.1 | ||
- name: Set up Go | ||
uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v2 | ||
with: | ||
go-version: '1.22.5' | ||
|
||
- name: Docker | ||
run: make docker-build | ||
|
||
- name: Log in to GHCR | ||
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v2 | ||
with: | ||
registry: ghcr.io | ||
username: ${{ github.actor }} | ||
password: ${{ secrets.GITHUB_TOKEN }} | ||
|
||
- name: Run build | ||
run: make docker-build | ||
|
||
- name: Push Docker images | ||
run: | | ||
docker push ghcr.io/bitbomdev/bomfactory:latest |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -17,3 +17,6 @@ clean: | |
rm -rf bin/ | ||
|
||
.PHONY: all build lint clean | ||
|
||
docker-build: | ||
./build-image.sh |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,62 +1,103 @@ | ||
|
||
# bomfactory | ||
|
||
## Why? | ||
## Overview | ||
|
||
`bomfactory` makes it easy to get SBOMs quickly, saving time and ensuring important projects. We aimed to test thousands of SBOMs for [minefield](https://github.com/bitbomdev/minefield), a simple graph database for managing dependencies. Using Roaring Bitmaps, it provides very fast O(1) queries on large datasets, and `bomfactory` made this process simple. | ||
`bomfactory` is a powerful command-line tool that simplifies and automates the process of downloading Software Bill of Materials (SBOMs) for multiple repositories. SBOMs are essential for software testing and security analysis, and `bomfactory` streamlines the task of acquiring them efficiently. | ||
|
||
If you are looking for thousands of SBOMs for testing and research, https://github.com/bitbomdev/bom-silo repository contains a large collection of SBOMs that you can use which was created by using `bomfactory`. | ||
Inspired by the [criticality_score](https://github.com/ossf/criticality_score) project, `bomfactory` specifically targets critical projects, making it an indispensable tool for anyone involved in software security, testing, or research. | ||
|
||
## What is bomfactory? | ||
## Why Use bomfactory? | ||
|
||
`bomfactory` is a powerful CLI tool designed to automate the downloading of Software Bill of Materials (SBOMs) for multiple repositories. SBOMs are crucial for software testing and security analysis, and `bomfactory` simplifies the process of obtaining them. This project draws inspiration from the [criticality_score](https://github.com/ossf/criticality_score) project to target critical projects. | ||
Working with thousands of SBOMs can be time-consuming and complex. We developed `bomfactory` to facilitate the rapid acquisition of SBOMs, ensuring that important projects are well-supported. This tool was crucial in testing thousands of SBOMs for [minefield](https://github.com/bitbomdev/minefield), a simple graph database for managing dependencies. By leveraging Roaring Bitmaps, it allows for O(1) query performance on large datasets, significantly simplifying the process. | ||
|
||
## Features | ||
If you require a large collection of SBOMs for testing or research, check out the [bom-silo](https://github.com/bitbomdev/bom-silo) repository, which was created using `bomfactory`. | ||
|
||
- **Download Criticality Score CSV**: Easily download the Criticality Score CSV file. | ||
- **Load CSV into SQLite**: Load CSV data into an SQLite database for efficient querying. | ||
- **Query Repositories**: Perform complex queries on the SQLite database to find repositories based on various criteria. | ||
- **Download SBOMs**: Automatically download SBOMs for repositories matching your query. | ||
## Key Features | ||
|
||
- **Download Criticality Score CSV**: Quickly download a CSV file containing criticality scores. | ||
- **Load CSV into SQLite**: Import CSV data into an SQLite database for efficient querying. | ||
- **Advanced Querying**: Perform complex queries on the SQLite database to identify repositories based on various criteria. | ||
- **Automated SBOM Downloads**: Download SBOMs automatically for repositories that match your query criteria. | ||
|
||
## Quickstart | ||
|
||
### Installation | ||
> **Note:** Replace `~/temp` with the path to your preferred directory. | ||
### Step 1: Download the CSV file containing criticality scores | ||
|
||
```bash | ||
docker run --rm -v ~/temp:/app/data ghcr.io/bitbomdev/bomfactory download-csv -o /app/data/data.csv | ||
``` | ||
|
||
### Step 2: Load the CSV data into SQLite | ||
|
||
```bash | ||
docker run --rm -v ~/temp:/app/data ghcr.io/bitbomdev/bomfactory load -d /app/data/data.db -c /app/data/data.csv --start 1 --end 1000 | ||
``` | ||
|
||
### Step 3: Query the SQLite data | ||
|
||
```bash | ||
docker run --rm -v ~/temp:/app/data ghcr.io/bitbomdev/bomfactory q -d /app/data/data.db -f "repo_language:=:Go" | ||
``` | ||
|
||
### Step 4: Download SBOMs for repositories | ||
|
||
```bash | ||
docker run --rm -v ~/temp:/app/data ghcr.io/bitbomdev/bomfactory ds -d /app/data/data.db -f "repo_language:=:Go" --dir /app/data | ||
``` | ||
|
||
## Advanced Usage | ||
|
||
The following example demonstrates how to download 1,000 SBOMs for Go repositories hosted on Google, skipping the first 9,000 repositories and downloading 10 SBOMs concurrently: | ||
|
||
Clone the repository and install the dependencies: | ||
```bash | ||
docker run --rm -v ~/temp:/app/data ghcr.io/bitbomdev/bomfactory ds --filter "repo_language:=:Go" --filter "repo_url:LIKE:%google/%" -m 1000 --dir /app/data/sboms/go -d /app/data/data.db -s 9000 --cd 10 | ||
``` | ||
|
||
> **Tip:** For a complete dataset, ensure that you load the entire CSV data into the SQLite database before performing advanced queries. | ||
## Installation | ||
|
||
```sh | ||
To install `bomfactory`, clone the repository and build the project: | ||
|
||
```bash | ||
git clone https://github.com/bitbomdev/bomfactory.git | ||
cd bomfactory | ||
make build | ||
``` | ||
|
||
### Usage | ||
## Detailed Usage | ||
|
||
#### 1. Download the CSV file | ||
### 1. Download the CSV File | ||
|
||
```sh | ||
```bash | ||
bomfactory download-csv --url https://www.googleapis.com/download/storage/v1/b/ossf-criticality-score/o/2024.07.05%2F143335%2Fall.csv?generation=1721362287412491&alt=media --output data.csv | ||
``` | ||
|
||
#### 2. Load the CSV data into SQLite | ||
### 2. Load the CSV Data into SQLite | ||
|
||
```sh | ||
```bash | ||
bomfactory load --csv data.csv --db data.db --start 1 --end 0 | ||
``` | ||
|
||
#### 3. Query the SQLite data | ||
### 3. Query the SQLite Data | ||
|
||
```sh | ||
```bash | ||
bomfactory query --filter "repo_language:==:Go" --filter "repo_star_count:>:100" --db data.db | ||
``` | ||
|
||
#### 4. Download SBOMs for repositories | ||
### 4. Download SBOMs for Repositories | ||
|
||
```sh | ||
```bash | ||
bomfactory download-sbom --filter "repo_language:==:Go" --token my_github_token --dir sbom_files --db data.db | ||
``` | ||
|
||
## Contributions and Support | ||
|
||
We welcome contributions and feedback! If you have any questions or need assistance, feel free to open an issue in the repository. | ||
|
||
--- | ||
|
||
Feel free to open an issue if you have any questions or need further assistance! | ||
This revised README improves readability and organization, ensuring that users can quickly understand the purpose of `bomfactory` and how to use it effectively. Let me know if there's anything else you'd like to add or modify! |