Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.
Nightly

dbarbuzzi triggered nightly on refs/heads/main #170

Sign in to view logs

dbarbuzzi triggered nightly on refs/heads/main

dbarbuzzi triggered nightly on refs/heads/main #170

Triggered via schedule June 7, 2024 01:26
@dbarbuzzidbarbuzzi
⁠ 87571b8
main
Status Failure
Total duration 1d 19h 9m 42s
Artifacts 19

nightly.yml

on: schedule
PYTHON-3-10  /  ...  /  BUILD
49m 23s
PYTHON-3-10 / BUILD / BUILD
PYTHON-3-11  /  ...  /  BUILD
54m 40s
PYTHON-3-11 / BUILD / BUILD
PYTHON-3-8  /  ...  /  BUILD
57m 47s
PYTHON-3-8 / BUILD / BUILD
PYTHON-3-9  /  ...  /  BUILD
59m 33s
PYTHON-3-9 / BUILD / BUILD
PYTHON-3-10  /  ...  /  BENCHMARK
5h 14m
PYTHON-3-10 / BENCHMARK / BENCHMARK
PYTHON-3-10  /  ...  /  TEST
4h 31m
PYTHON-3-10 / TEST-SOLO / TEST
PYTHON-3-10  /  ...  /  TEST
5h 39m
PYTHON-3-10 / TEST-MULTI / TEST
PYTHON-3-10  /  ...  /  TEST-ACCURACY-SMOKE
9m 37s
PYTHON-3-10 / TEST-ACCURACY-SMOKE / TEST-ACCURACY-SMOKE
PYTHON-3-10  /  ...  /  TEST-ACCURACY-FULL
PYTHON-3-10 / TEST-ACCURACY-FULL / TEST-ACCURACY-FULL
PYTHON-3-11  /  ...  /  BENCHMARK
5m 45s
PYTHON-3-11 / BENCHMARK / BENCHMARK
PYTHON-3-11  /  ...  /  TEST
4h 38m
PYTHON-3-11 / TEST-SOLO / TEST
PYTHON-3-11  /  ...  /  TEST
0s
PYTHON-3-11 / TEST-MULTI / TEST
PYTHON-3-11  /  ...  /  TEST-ACCURACY-SMOKE
10m 47s
PYTHON-3-11 / TEST-ACCURACY-SMOKE / TEST-ACCURACY-SMOKE
PYTHON-3-11  /  ...  /  TEST-ACCURACY-FULL
PYTHON-3-11 / TEST-ACCURACY-FULL / TEST-ACCURACY-FULL
PYTHON-3-8  /  ...  /  BENCHMARK
5h 14m
PYTHON-3-8 / BENCHMARK / BENCHMARK
PYTHON-3-8  /  ...  /  TEST
4h 29m
PYTHON-3-8 / TEST-SOLO / TEST
PYTHON-3-8  /  ...  /  TEST
0s
PYTHON-3-8 / TEST-MULTI / TEST
PYTHON-3-8  /  ...  /  TEST-ACCURACY-SMOKE
9m 17s
PYTHON-3-8 / TEST-ACCURACY-SMOKE / TEST-ACCURACY-SMOKE
PYTHON-3-8  /  ...  /  TEST-ACCURACY-FULL
PYTHON-3-8 / TEST-ACCURACY-FULL / TEST-ACCURACY-FULL
PYTHON-3-9  /  ...  /  BENCHMARK
5h 14m
PYTHON-3-9 / BENCHMARK / BENCHMARK
PYTHON-3-9  /  ...  /  TEST
4h 34m
PYTHON-3-9 / TEST-SOLO / TEST
PYTHON-3-9  /  ...  /  TEST
0s
PYTHON-3-9 / TEST-MULTI / TEST
PYTHON-3-9  /  ...  /  TEST-ACCURACY-SMOKE
9m 32s
PYTHON-3-9 / TEST-ACCURACY-SMOKE / TEST-ACCURACY-SMOKE
PYTHON-3-9  /  ...  /  TEST-ACCURACY-FULL
PYTHON-3-9 / TEST-ACCURACY-FULL / TEST-ACCURACY-FULL
PYTHON-3-10  /  ...  /  BENCHMARK_REPORT
40s
PYTHON-3-10 / BENCHMARK / BENCHMARK_REPORT
PYTHON-3-10  /  ...  /  PUBLISH
46s
PYTHON-3-10 / PUBLISH / PUBLISH
PYTHON-3-11  /  ...  /  BENCHMARK_REPORT
0s
PYTHON-3-11 / BENCHMARK / BENCHMARK_REPORT
PYTHON-3-11  /  ...  /  PUBLISH
PYTHON-3-11 / PUBLISH / PUBLISH
PYTHON-3-8  /  ...  /  BENCHMARK_REPORT
49s
PYTHON-3-8 / BENCHMARK / BENCHMARK_REPORT
PYTHON-3-8  /  ...  /  PUBLISH
PYTHON-3-8 / PUBLISH / PUBLISH
PYTHON-3-9  /  ...  /  BENCHMARK_REPORT
45s
PYTHON-3-9 / BENCHMARK / BENCHMARK_REPORT
PYTHON-3-9  /  ...  /  PUBLISH
PYTHON-3-9 / PUBLISH / PUBLISH
Fit to window
Zoom out
Zoom in

Annotations

10 errors and 1 warning
PYTHON-3-10 / TEST-ACCURACY-SMOKE / TEST-ACCURACY-SMOKE
Process completed with exit code 1.
PYTHON-3-11 / BENCHMARK / BENCHMARK
Process completed with exit code 1.
PYTHON-3-11 / TEST-ACCURACY-SMOKE / TEST-ACCURACY-SMOKE
Process completed with exit code 1.
PYTHON-3-8 / TEST-ACCURACY-SMOKE / TEST-ACCURACY-SMOKE
Process completed with exit code 1.
PYTHON-3-9 / TEST-ACCURACY-SMOKE / TEST-ACCURACY-SMOKE
Process completed with exit code 1.
PYTHON-3-9 / TEST-SOLO / TEST
Process completed with exit code 1.
PYTHON-3-9 / BENCHMARK / BENCHMARK_REPORT
# :warning: **Performance Alert** :warning: Possible performance regression was detected for benchmark **'smaller_is_better'**. Benchmark result of this commit is worse than the previous benchmark result exceeding threshold `1.10`. | Benchmark suite | Current: 87571b8be8105738d6da87df053d5a32e7fa001e | Previous: 367c5ee80cc75f5d5b6af72de5e1e5e463e386f7 | Ratio | |-|-|-|-| | `{"name": "mean_ttft_ms", "description": "VLLM Serving - 2:4 Sparse\nmodel - neuralmagic/OpenHermes-2.5-Mistral-7B-pruned2.4\nmax-model-len - 4096\nsparsity - semi_structured_sparse_w16a16\nbenchmark_serving {\n \"nr-qps-pair_\": \"1500,5\",\n \"dataset\": \"sharegpt\"\n}", "gpu_description": "NVIDIA A10G x 1", "vllm_version": "0.5.0", "python_version": "3.9.17 (main, Jun 7 2023, 12:29:40) \n[GCC 9.4.0]", "torch_version": "2.3.0+cu121"}` | `1503.0183703219845` ms | `1292.66038330267` ms | `1.16` | This comment was automatically generated by [workflow](https://github.com/neuralmagic/nm-vllm/actions?query=workflow%3ANightly) using [github-action-benchmark](https://github.com/marketplace/actions/continuous-benchmark). Comment was generated at https://github.com/neuralmagic/nm-vllm/commit/87571b8be8105738d6da87df053d5a32e7fa001e#commitcomment-142837486
PYTHON-3-9 / TEST-MULTI / TEST
This request was automatically failed because there were no enabled runners online to process the request for more than 1 days.
PYTHON-3-8 / TEST-MULTI / TEST
This request was automatically failed because there were no enabled runners online to process the request for more than 1 days.
PYTHON-3-11 / TEST-MULTI / TEST
This request was automatically failed because there were no enabled runners online to process the request for more than 1 days.
PYTHON-3-9 / BENCHMARK / BENCHMARK_REPORT
Performance alert! Previous value was 1292.66038330267 and current value is 1503.0183703219845. It is 1.1627326014910913x worse than previous exceeding a ratio threshold 1.1

Artifacts

Produced during runtime
Name Size
3.10.12-nm-vllm-nightly-0.5.0.20240607.tar.gz Expired
578 KB
3.11.4-nm-vllm-nightly-0.5.0.20240607.tar.gz Expired
578 KB
3.8.17-nm-vllm-nightly-0.5.0.20240607.tar.gz Expired
578 KB
3.9.17-nm-vllm-nightly-0.5.0.20240607.tar.gz Expired
578 KB
9410196878-aws-test-a10g-24G-3.10.12 Expired
124 KB
9410196878-aws-test-a10g-24G-3.8.17 Expired
124 KB
9410196878-aws-test-a10g-24G-3.9.17 Expired
124 KB
cc-vllm-html-aws-test-4-a10g-96G-3.10.12 Expired
2.59 MB
cc-vllm-html-aws-test-a10g-24G-3.10.12 Expired
2.59 MB
cc-vllm-html-aws-test-a10g-24G-3.11.4 Expired
2.59 MB
cc-vllm-html-aws-test-a10g-24G-3.8.17 Expired
2.59 MB
cc-vllm-html-aws-test-a10g-24G-3.9.17 Expired
2.59 MB
gh_action_benchmark_jsons-9410196878-aws-test-a10g-24G-3.10.12 Expired
28.6 KB
gh_action_benchmark_jsons-9410196878-aws-test-a10g-24G-3.8.17 Expired
28.4 KB
gh_action_benchmark_jsons-9410196878-aws-test-a10g-24G-3.9.17 Expired
28.3 KB
nm_vllm_nightly-0.5.0.20240607-cp310-cp310-manylinux_2_17_x86_64.whl Expired
130 MB
nm_vllm_nightly-0.5.0.20240607-cp311-cp311-manylinux_2_17_x86_64.whl Expired
130 MB
nm_vllm_nightly-0.5.0.20240607-cp38-cp38-manylinux_2_17_x86_64.whl Expired
130 MB
nm_vllm_nightly-0.5.0.20240607-cp39-cp39-manylinux_2_17_x86_64.whl Expired
130 MB