-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ifeval: Dowload punkt_tab
on rank 0
#2267
base: main
Are you sure you want to change the base?
Conversation
nltk.download("punkt_tab") | ||
print("Downloaded punkt_tab") | ||
else: | ||
time.sleep(5) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This isn't really necessary as the code runs at the beginning (before generations). But couldn't hurt.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
suggestion: if sleep is not needed we should not add it.
Sharing a slightly more verbose version.
|
suggestion: would rethink the current download on import behaviour. |
@al093 . Thanks very much for your suggestion. I'm hesitant in add a |
Works for me. This is actually what i did finally to make it work for my case as well. We cannot synchronise multiple worker/processes if we dont know what framework was used to launch the multiple processes. So I think it has to handled by user code or manual intervention as you suggest. |
closes #2266. Also removed the
pkg_resources
dependency as that's depreciated.