This repository has been archived by the owner on Oct 11, 2024. It is now read-only.
dbarbuzzi triggered nightly on refs/heads/main #132
dbarbuzzinightly.yml
on: schedule
BUILD-TEST
/
...
/
BUILD
34m 22s
BUILD-TEST
/
...
/
TEST
4h 3m
BUILD-TEST
/
...
/
TEST
5h 8m
BUILD-TEST
/
...
/
TEST-ACCURACY-SMOKE
8m 0s
BUILD-TEST
/
...
/
TEST-ACCURACY-FULL
BUILD-TEST
/
...
/
BENCHMARK_REPORT
35s
BUILD-TEST
/
...
/
PUBLISH
Annotations
2 errors and 1 warning
|
BUILD-TEST / TEST-SOLO / TEST
Process completed with exit code 1.
|
|
BUILD-TEST / BENCHMARK / BENCHMARK_REPORT
# :warning: **Performance Alert** :warning:
Possible performance regression was detected for benchmark **'smaller_is_better'**.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold `1.10`.
| Benchmark suite | Current: d69a34a3194efb8dd34c1f293af91aac19b5b992 | Previous: 59cf939c70173f2419d143a738505180c37465a4 | Ratio |
|-|-|-|-|
| `{"name": "mean_ttft_ms", "description": "VLLM Serving - 2:4 Sparse\nmodel - neuralmagic/OpenHermes-2.5-Mistral-7B-pruned2.4\nmax-model-len - 4096\nsparsity - semi_structured_sparse_w16a16\nbenchmark_serving {\n \"nr-qps-pair_\": \"1500,5\",\n \"dataset\": \"sharegpt\"\n}", "gpu_description": "NVIDIA A10G x 1", "vllm_version": "0.3.0", "python_version": "3.10.12 (main, May 10 2024, 13:42:25) [GCC 9.4.0]", "torch_version": "2.3.0+cu121"}` | `257.79683584202337` ms | `229.12089890001153` ms | `1.13` |
This comment was automatically generated by [workflow](https://github.com/neuralmagic/nm-vllm/actions?query=workflow%3ANightly) using [github-action-benchmark](https://github.com/marketplace/actions/continuous-benchmark).
Comment was generated at https://github.com/neuralmagic/nm-vllm/commit/d69a34a3194efb8dd34c1f293af91aac19b5b992#commitcomment-142164299
|
|
BUILD-TEST / BENCHMARK / BENCHMARK_REPORT
Performance alert! Previous value was 229.12089890001153 and current value is 257.79683584202337. It is 1.1251563566644613x worse than previous exceeding a ratio threshold 1.1
|
Artifacts
Produced during runtime
| Name | Size | |
|---|---|---|
|
3.10.12-nm-vllm-nightly-0.3.0.20240518.tar.gz
Expired
|
534 KB |
|
|
9136217407-aws-avx2-32G-a10g-24G
Expired
|
124 KB |
|
|
cc-vllm-html-aws-avx2-192G-4-a10g-96G
Expired
|
2.23 MB |
|
|
cc-vllm-html-aws-avx2-32G-a10g-24G
Expired
|
2.23 MB |
|
|
gh_action_benchmark_jsons-9136217407-aws-avx2-32G-a10g-24G
Expired
|
29.3 KB |
|
|
nm_vllm_nightly-0.3.0.20240518-cp310-cp310-manylinux_2_17_x86_64.whl
Expired
|
103 MB |
|