Skip to content

Commit

Permalink
Fix usage command
Browse files Browse the repository at this point in the history
Add MODEL_PATH description to displayed output
  • Loading branch information
rahul-tuli committed Feb 20, 2024
1 parent a284a0a commit 08f0e67
Showing 1 changed file with 9 additions and 1 deletion.
10 changes: 9 additions & 1 deletion src/deepsparse/evaluation/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@
##########
Example command for evaluating a quantized MPT model from SparseZoo using the Deepsparse Engine.
The evaluation will be run using `lm-evaluation-harness` on `hellaswag` and `gsm8k` datasets:
deepsparse.eval zoo:mpt-7b-mpt_pretrain-base_quantized \
deepsparse.evaluate zoo:mpt-7b-mpt_pretrain-base_quantized \
--dataset hellaswag \
--dataset gsm8k \
--integration lm-evaluation-harness \
Expand Down Expand Up @@ -173,6 +173,14 @@ def main(
metrics,
integration_args,
):
"""
Evaluate MODEL_PATH on the various evaluation integrations
MODEL_PATH can be path to an ONNX model, local directory
containing ONNX model (including all the auxiliary files)
or a SparseZoo stub
"""
# join datasets to a list if multiple datasets are passed
datasets = list(dataset) if not isinstance(dataset, str) else dataset
# format kwargs to a dict
Expand Down

0 comments on commit 08f0e67

Please sign in to comment.