Skip to content

Commit

Permalink
Some tweaks
Browse files Browse the repository at this point in the history
Signed-off-by: naveensrinivasan <[email protected]>
  • Loading branch information
naveensrinivasan committed Aug 30, 2024
1 parent d8a01c9 commit 76a0670
Show file tree
Hide file tree
Showing 3 changed files with 108 additions and 23 deletions.
41 changes: 41 additions & 0 deletions .github/workflows/docker.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Build and Push Docker Image

on:
push:
branches:
- main

jobs:
build-and-test:
runs-on: ubuntu-latest

permissions:
contents: read
packages: write

steps:
- name: Checkout code
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v2
- name: Set up Docker
uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db # v3.6.1
- name: Set up Go
uses: actions/setup-go@0a12ed9d6a96ab950c8f026ed9f722fe0da7ef32 # v2
with:
go-version: '1.22.5'

- name: Docker
run: make docker-build

- name: Log in to GHCR
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Run build
run: make docker-build

- name: Push Docker images
run: |
docker push ghcr.io/bitbomdev/bomfactory:latest
3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,6 @@ clean:
rm -rf bin/

.PHONY: all build lint clean

docker-build:
./build-image.sh
87 changes: 64 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,62 +1,103 @@

# bomfactory

## Why?
## Overview

`bomfactory` makes it easy to get SBOMs quickly, saving time and ensuring important projects. We aimed to test thousands of SBOMs for [minefield](https://github.com/bitbomdev/minefield), a simple graph database for managing dependencies. Using Roaring Bitmaps, it provides very fast O(1) queries on large datasets, and `bomfactory` made this process simple.
`bomfactory` is a powerful command-line tool that simplifies and automates the process of downloading Software Bill of Materials (SBOMs) for multiple repositories. SBOMs are essential for software testing and security analysis, and `bomfactory` streamlines the task of acquiring them efficiently.

If you are looking for thousands of SBOMs for testing and research, https://github.com/bitbomdev/bom-silo repository contains a large collection of SBOMs that you can use which was created by using `bomfactory`.
Inspired by the [criticality_score](https://github.com/ossf/criticality_score) project, `bomfactory` specifically targets critical projects, making it an indispensable tool for anyone involved in software security, testing, or research.

## What is bomfactory?
## Why Use bomfactory?

`bomfactory` is a powerful CLI tool designed to automate the downloading of Software Bill of Materials (SBOMs) for multiple repositories. SBOMs are crucial for software testing and security analysis, and `bomfactory` simplifies the process of obtaining them. This project draws inspiration from the [criticality_score](https://github.com/ossf/criticality_score) project to target critical projects.
Working with thousands of SBOMs can be time-consuming and complex. We developed `bomfactory` to facilitate the rapid acquisition of SBOMs, ensuring that important projects are well-supported. This tool was crucial in testing thousands of SBOMs for [minefield](https://github.com/bitbomdev/minefield), a simple graph database for managing dependencies. By leveraging Roaring Bitmaps, it allows for O(1) query performance on large datasets, significantly simplifying the process.

## Features
If you require a large collection of SBOMs for testing or research, check out the [bom-silo](https://github.com/bitbomdev/bom-silo) repository, which was created using `bomfactory`.

- **Download Criticality Score CSV**: Easily download the Criticality Score CSV file.
- **Load CSV into SQLite**: Load CSV data into an SQLite database for efficient querying.
- **Query Repositories**: Perform complex queries on the SQLite database to find repositories based on various criteria.
- **Download SBOMs**: Automatically download SBOMs for repositories matching your query.
## Key Features

- **Download Criticality Score CSV**: Quickly download a CSV file containing criticality scores.
- **Load CSV into SQLite**: Import CSV data into an SQLite database for efficient querying.
- **Advanced Querying**: Perform complex queries on the SQLite database to identify repositories based on various criteria.
- **Automated SBOM Downloads**: Download SBOMs automatically for repositories that match your query criteria.

## Quickstart

### Installation
> **Note:** Replace `~/temp` with the path to your preferred directory.
### Step 1: Download the CSV file containing criticality scores

```bash
docker run --rm -v ~/temp:/app/data ghcr.io/bitbomdev/bomfactory download-csv -o /app/data/data.csv
```

### Step 2: Load the CSV data into SQLite

```bash
docker run --rm -v ~/temp:/app/data ghcr.io/bitbomdev/bomfactory load -d /app/data/data.db -c /app/data/data.csv --start 1 --end 1000
```

### Step 3: Query the SQLite data

```bash
docker run --rm -v ~/temp:/app/data ghcr.io/bitbomdev/bomfactory q -d /app/data/data.db -f "repo_language:=:Go"
```

### Step 4: Download SBOMs for repositories

```bash
docker run --rm -v ~/temp:/app/data ghcr.io/bitbomdev/bomfactory ds -d /app/data/data.db -f "repo_language:=:Go" --dir /app/data
```

## Advanced Usage

The following example demonstrates how to download 1,000 SBOMs for Go repositories hosted on Google, skipping the first 9,000 repositories and downloading 10 SBOMs concurrently:

Clone the repository and install the dependencies:
```bash
docker run --rm -v ~/temp:/app/data ghcr.io/bitbomdev/bomfactory ds --filter "repo_language:=:Go" --filter "repo_url:LIKE:%google/%" -m 1000 --dir /app/data/sboms/go -d /app/data/data.db -s 9000 --cd 10
```

> **Tip:** For a complete dataset, ensure that you load the entire CSV data into the SQLite database before performing advanced queries.
## Installation

```sh
To install `bomfactory`, clone the repository and build the project:

```bash
git clone https://github.com/bitbomdev/bomfactory.git
cd bomfactory
make build
```

### Usage
## Detailed Usage

#### 1. Download the CSV file
### 1. Download the CSV File

```sh
```bash
bomfactory download-csv --url https://www.googleapis.com/download/storage/v1/b/ossf-criticality-score/o/2024.07.05%2F143335%2Fall.csv?generation=1721362287412491&alt=media --output data.csv
```

#### 2. Load the CSV data into SQLite
### 2. Load the CSV Data into SQLite

```sh
```bash
bomfactory load --csv data.csv --db data.db --start 1 --end 0
```

#### 3. Query the SQLite data
### 3. Query the SQLite Data

```sh
```bash
bomfactory query --filter "repo_language:==:Go" --filter "repo_star_count:>:100" --db data.db
```

#### 4. Download SBOMs for repositories
### 4. Download SBOMs for Repositories

```sh
```bash
bomfactory download-sbom --filter "repo_language:==:Go" --token my_github_token --dir sbom_files --db data.db
```

## Contributions and Support

We welcome contributions and feedback! If you have any questions or need assistance, feel free to open an issue in the repository.

---

Feel free to open an issue if you have any questions or need further assistance!
This revised README improves readability and organization, ensuring that users can quickly understand the purpose of `bomfactory` and how to use it effectively. Let me know if there's anything else you'd like to add or modify!

0 comments on commit 76a0670

Please sign in to comment.