Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[paraformer] add paraformer tokenizer #2219

Merged
merged 9 commits into from
Dec 12, 2023
Merged

[paraformer] add paraformer tokenizer #2219

merged 9 commits into from
Dec 12, 2023

Conversation

Mddct
Copy link
Collaborator

@Mddct Mddct commented Dec 12, 2023 •
edited
Loading

follow this PR： #2165

  • paraformer tokenizer
  • unit test
  • fix lint
  • unit resource 不新增

xingchensong
xingchensong reviewed Dec 12, 2023
View reviewed changes
test/wenet/text/test_paraformer_tokenizer.py Outdated Show resolved Hide resolved
wenet/utils/init_tokenizer.py Outdated Show resolved Hide resolved
wenet/text/paraformer_tokenizer.py Outdated Show resolved Hide resolved
@xingchensong xingchensong merged commit 5887279 into main Dec 12, 2023
6 checks passed
@xingchensong xingchensong deleted the Mddct-tokenizer-paraformer branch December 12, 2023 10:11
@Mddct Mddct mentioned this pull request Dec 12, 2023
Merged
15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xingchensong xingchensong xingchensong approved these changes

@robin1001 robin1001 Awaiting requested review from robin1001

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Mddct @xingchensong