Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make preprocessing fully differentiable with torch API #4

Closed
HudsonHuang opened this issue Dec 16, 2020 · 2 comments
Closed

Make preprocessing fully differentiable with torch API #4

HudsonHuang opened this issue Dec 16, 2020 · 2 comments

Comments

@HudsonHuang
Copy link

HudsonHuang commented Dec 16, 2020

I appreciate your efforts, nice work.
But your audio_toolkit was implement in librosa and numpy, which was not differentiable.
It might limited the application. Eg. If I have an TTS model to generated Mel spectrogram, and if your dvector if fully differentiable, we can use this like a discriminator, to force the TTS model output exactly as expected person.
From waveform to Melspectrogram, you can make preprocessing fully differentiable with torchaudio, and it seems it can keep consitency with librosa

@yistLin
Copy link
Owner

yistLin commented Dec 16, 2020

Hi, thanks for your suggestion. I'm actually considering ditching librosa for torchaudio especially after I chose to do silence trimming with sox instead of webrtcvad.

Since I'd like to make the preprocessing modules as simple as possible (import less packages as possible), I probably need some time to study the usage of sox effects in the most recent version of torchaudio.

@yistLin
Copy link
Owner

yistLin commented Jan 24, 2021

I've developed completely new preprocessing toolkits which use torchaudio, can be compiled with TorchScript and be used anywhere without any dependencies.

@yistLin yistLin closed this as completed Jan 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants