Skip to content

Issues: UKGovernmentBEIS/inspect_ai

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

49 Open 269 Closed
49 Open 269 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

The "Value" type in score is overly restrictive for dictonaries
#1333 opened Feb 17, 2025 by max-kaufmann
1
Side by side comparison of two models
#1327 opened Feb 16, 2025 by yassersouri
@dragonstyle
5
Scorers which don't return a numeric Score cause crashes in the log viewer
#1324 opened Feb 15, 2025 by max-kaufmann
2
Include Gemini Reasoning Traces
#1312 opened Feb 13, 2025 by lennart-finke
2
Reasoning Traces for OpenRouter
#1310 opened Feb 13, 2025 by lennart-finke
1
Include sample id in docker project name
#1304 opened Feb 12, 2025 by AlexLloyd0
2
web_browser: strip base64 encoded images from accessibility trees web browser
#1303 opened Feb 12, 2025 by jjallaire-aisi
1
1
Non-deterministic error in proto package when using Google models
#1299 opened Feb 12, 2025 by tadamcz
15
[feature request] progress of forked solvers / subtasks are not visible in terminal viewer
#1296 opened Feb 12, 2025 by skinnerjc
2
[feature request] set default cache policy
#1292 opened Feb 11, 2025 by kaifronsdal
1
MATH dataset used for mathematics tutorial no longer accessible
#1283 opened Feb 10, 2025 by ChiWilliams
1
improved retry behavior
#1277 opened Feb 10, 2025 by jjallaire
1
Custom Displays
#1267 opened Feb 7, 2025 by Baidicoot
1
Run cleanup and finalizers
#1265 opened Feb 7, 2025 by sjawhar
Unique ID for each run of each sample
#1262 opened Feb 7, 2025 by sjawhar
Log viewer: scroll bar resets when using the scorer drop-down
#1220 opened Jan 30, 2025 by tadamcz
@dragonstyle
[Feature request] Support for LiteLLM and LiteLLM proxy
#1213 opened Jan 30, 2025 by danrr
Execute --runslow tests in CI
#1212 opened Jan 30, 2025 by jjallaire-aisi
@art-dsit
1
Tutorial: Mathematics dataset not available in HF anymore
#1183 opened Jan 24, 2025 by guiem
Broader notion of retryable exception
#1174 opened Jan 22, 2025 by tadamcz
10
Feature Request: Grouped output metrics & exportable result files for benchmarks
#1172 opened Jan 22, 2025 by owenparsons
@dragonstyle
3
Enable sandboxes to connect to existing Docker or k8s instances
#1153 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
Enforcing cost limits
#1152 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
2
Broader notions of "model" for evals
#1149 opened Jan 19, 2025 by jjallaire
@jjallaire
Bridge to convert from compose.yaml to HELM values
#1148 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
1
ProTip! Add no:assignee to see everything that’s not assigned.