Skip to content

Commit

Permalink
show img in readme (#178)
Browse files Browse the repository at this point in the history
* feat:  add way to verify

* center img

* center img

* center img

* center img
  • Loading branch information
YannDubs authored Nov 27, 2023
1 parent 5d66c75 commit 94cd8b6
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 4 deletions.
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1059,15 +1059,17 @@ git push
<details>
<summary><h3 tabindex="-1" dir="auto">Getting your model verified</h3></summary>
<img alt="verified.png" src="..%2F..%2Fverified.png" width="100"/>
<p align="center">
<img align="center" alt="verified.png" src="figures/verified.png" width="500"/>
</p>
A verified result in AlpacaEval indicates that a core maintainer has decoded the outputs from the model and performed the evaluation. Unfortunately, we, the AlpacaEval maintainers, lack the resources (time/GPU/money) to verify all models. We apologize for any inconvenience this may cause and appreciate your understanding. To have your model verified, please follow the steps below:
1. Contact @yann or @rtaori on Discord, or email us if you have our email, providing a brief rationale for why your model should be verified.
1. Contact `@yann` or `@rtaori` on Discord, or email us if you have our email, providing a brief rationale for why your model should be verified.
2. Await our response and approval before proceeding.
3. Prepare a script to decode from your model that does not require a GPU, typically the same script used for your model contribution. It should run using alpaca_eval evaluate_from_model --model_configs '<your_model_name>' without GPU support.
3. Prepare a script to decode from your model that does not require a GPU, typically the same script used for your model contribution. It should run using `alpaca_eval evaluate_from_model --model_configs '<your_model_name>'` without requiring a local GPU.
4. Generate temporary API keys for running the script and share them with us. Specifically, we need the keys for both decoding your model and for evaluation (e.g., OpenAI or Anthropic key).
5. We will execute the alpaca_eval evaluate_from_model --model_configs '<your_model_name>', update the results, and inform you so you can revoke the temporary keys.
5. We will execute the alpaca_eval `evaluate_from_model --model_configs '<your_model_name>'`, update the results, and inform you so you can revoke the temporary keys.
Note that we will not re-evaluate the same model. Due to sampling variance, the results might slightly differ from your initial ones. We will replace your previous community results with the verified ones.
Expand Down
Binary file added figures/verified.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 94cd8b6

Please sign in to comment.