Language translation example added (#1131) #1240
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for pytorch-examples-preview canceled.
|
msaroufim
approved these changes
Mar 31, 2024
|
Looks like the test failed could you please take a look? |
|
Hi! I think the issue was that the model (with default settings) is quite large. I'm thinking the test server was potentially running out of VRAM and crashed. In any case, I have passed the appropriate flags so that it is a lot smaller. |
|
Thanks @msaroufim. I was going to fix that tonight after work. In the future, how can I make sure my environment matches the testing environment so we don't have to troubleshoot here? The tests were working on my machine but seems like my environment was different. |
|
TBH I test in CI XD - but on your end you can turn https://github.com/pytorch/examples/blob/main/.github/workflows/main_python.yml into a shell script and use that, should be mostly fine |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello all, this PR seeks to add language translation to the repo as requested in issue #1131. It utilizes transformers and closely follows the Attention is All You Need paper.
I understand there is a desire to reduce dependencies for the sake of simplicity, however I had to use the spacy library. Currently, torchtext does not offer a lot of language tokenizers, just basic English. As such, we need to lean on other tokenizers, which spacy provides. The word from torchtext devs is that spacy is an "optional dependency" (source: pytorch/text#178) so I hope you will accept it's necessity here in this example. The alternative would be to write tokenizers for every language we want to support, which I feel is antithetical to simplicity in these examples.
Other than that, this example tries to rely heavily on the tools provided in torchtext to better showcase the library as well as get users familiar with transformers and how they work!
Please let me know if there is anything you'd like to see changed.