You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Using OpenML tasks ensures reproducibility, but at the same time it can be limiting, since you can't share runs post-hoc, just after you discovered something nice, the way most experiments are.
Maybe we should allow uploading runs post-hoc, when models are trained even if they didn't start from an OpenML task.
So, say you have a sklearn or PyTorch model trained, allow uploading it by not linking it to an existing task (because you can't), but perhaps with an ad-hoc task described manually, or by extracting all the info we can from the ML environment.
It would give people a way to easily share experiments and models with each other with still 'pretty acceptable reproducibility'.
People can always filter those out if they don't trust them enough.
Thoughts?
The text was updated successfully, but these errors were encountered:
Using OpenML tasks ensures reproducibility, but at the same time it can be limiting, since you can't share runs post-hoc, just after you discovered something nice, the way most experiments are.
Maybe we should allow uploading runs post-hoc, when models are trained even if they didn't start from an OpenML task.
So, say you have a sklearn or PyTorch model trained, allow uploading it by not linking it to an existing task (because you can't), but perhaps with an ad-hoc task described manually, or by extracting all the info we can from the ML environment.
It would give people a way to easily share experiments and models with each other with still 'pretty acceptable reproducibility'.
People can always filter those out if they don't trust them enough.
Thoughts?
The text was updated successfully, but these errors were encountered: