You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To ensure that new benchmarks and models actually run through, we should add our own tests that provide a dummy model or dummy benchmark and simulate an entire benchmark or model run respectively. Otherwise we might merge new plugins that don't actually run properly.
We might need to do this separately for each domain, i.e. dummy model+benchmark for language, and the same for vision (and all future domains).
The text was updated successfully, but these errors were encountered:
To ensure that new benchmarks and models actually run through, we should add our own tests that provide a dummy model or dummy benchmark and simulate an entire benchmark or model run respectively. Otherwise we might merge new plugins that don't actually run properly.
We might need to do this separately for each domain, i.e. dummy model+benchmark for language, and the same for vision (and all future domains).
The text was updated successfully, but these errors were encountered: