Skip to content

Commit

Permalink
Adds CM commands to run stable diffusion reference implementation (#1678
Browse files Browse the repository at this point in the history
)

Co-authored-by: Miro <[email protected]>
  • Loading branch information
arjunsuresh and mrmhodak authored May 21, 2024
1 parent 101bb5b commit 8fcd0d0
Showing 1 changed file with 11 additions and 2 deletions.
13 changes: 11 additions & 2 deletions text_to_image/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,19 @@ This is the reference implementation for MLPerf Inference text to image

| Data | Description |
| ---- | ---- |
| Coco-2014 | We use a subset of 5000 images and captions of the coco 2014 validation dataset, so that there is exaclty one caption per image. The model takes as input the caption of the image and generates an image from it. The original images and the generated images are used to compute FID score. The caption and the generated images are used to compute the CLIP score. We provide a [script](tools/coco.py) to automatically download the dataset |
| Coco-2014 (calibration) | We use a subset of 500 captions and images of the coco 2014 training dataset, so that there is exaclty one caption per image. The subset was generated using this [script](tools/coco_generate_calibration.py). We provide the [caption ids](../calibration/COCO-2014/coco_cal_captions_list.txt) and a [script](tools/coco_calibration.py) to download them. |
| Coco-2014 | We use a subset of 5000 images and captions of the coco 2014 validation dataset, so that there is exactly one caption per image. The model takes as input the caption of the image and generates an image from it. The original images and the generated images are used to compute FID score. The caption and the generated images are used to compute the CLIP score. We provide a [script](tools/coco.py) to automatically download the dataset |
| Coco-2014 (calibration) | We use a subset of 500 captions and images of the coco 2014 training dataset, so that there is exactly one caption per image. The subset was generated using this [script](tools/coco_generate_calibration.py). We provide the [caption ids](../calibration/COCO-2014/coco_cal_captions_list.txt) and a [script](tools/coco_calibration.py) to download them. |

## Automated command to run the benchmark via MLCommons CM

```
python3 -m pip install cmind
cm pull repo mlcommons@ck
cm run script --tags=run-mlperf,inference --model=sdxl --implementation=reference --backend=pytorch --device=cpu --precision=float32 --scenario=Offline --quiet
```
* `--device=cuda` can be used to run on Nvidia GPUs and `--device=rocm` can be used to run on AMD GPUs
* `--precision=float16` or `--precision=bfloat16` can be used to change the model precision

## Setup
Set the following helper variables
```bash
Expand Down

0 comments on commit 8fcd0d0

Please sign in to comment.