Training model with nvidia a100

[JRMeyer]

>>> betim
[February 17, 2021, 3:44pm]

Hi,

Is there anyone who managed to run Deepspeech Model training on NVIDIA
A100 GPUs?

We got a server with 8 NVIDIA A100 40GB CoWoS HBM2 PCIe 4.0 slash -- Passive
Cooling, but couldn't manage to run training with GPU yet.

[This is an archived TTS discussion thread from discourse.mozilla.org/t/training-model-with-nvidia-a100]

[JRMeyer]

>>> lazyguy
[February 17, 2021, 4:18pm]

Hi Betim,

I tried training it on A100 sometime back but A100 does not support CUDA
10.0, I guess(I could be wrong, it's been a while last I checked :p). If
you can get CUDA 10.0, then it should work.

[Archived Post]

[JRMeyer]

>>> lissyx
[February 17, 2021, 4:27pm]

Maybe NVIDIA provides tensorflow r1.15 for those GPUs ?

[Archived Post]

[JRMeyer]

>>> betim
[February 17, 2021, 9:00pm]

So we have to uninstall DS tf and install NVIDIA r1.15?

[Archived Post]

[JRMeyer]

>>> lissyx
[February 17, 2021, 9:04pm]

> DS tf

There is no such thing as DeepSpeech TensorFlow. We rely on upstream
TensorFlow. I read somewhere that NVIDIA is providing a TensorFlow r1.15
package for RTX 3xxx, so maybe this could apply to your case as well.

[Archived Post]

[JRMeyer]

>>> betim
[February 17, 2021, 9:18pm]

I see that description match what we're talking on this gh link:
https://github.com/NVIDIA/tensorflow.

can you just check it to give
me a hint if we're pointing to the same direction.

[Archived Post]

[JRMeyer]

>>> lissyx
[February 17, 2021, 9:20pm]

I'm sorry but if you don't ask me a clear question, I don't have time to
dig into nvidia's repos, and I can't speak for them nor provide support
for their work.

[Archived Post]

[JRMeyer]

>>> lissyx
[February 17, 2021, 9:21pm]

And the start of their readme is
pretty clear to me, it seems to be exactly this usecase that is
addressed.

[Archived Post]

[JRMeyer]

>>> carlfm01
[February 19, 2021, 1:08am]

> I see that description match what we're talking on this gh link:
> https://github.com/NVIDIA/tensorflow .

I confirm I can train a model
using 3000 series with Nvidia's version, I guess it also works with the
a100 since they are ampere based gpus.

[Archived Post]

[JRMeyer]

>>> NanoNabla
[February 22, 2021, 12:07pm]

NVIDIA does also offer docker container with their TensorFlow build slash
https://ngc.nvidia.com/catalog/containers/nvidia:tensorflow slash
Maybe, you want to give it a try instead of building TensorFlow on you
own. slash
I also can not test A100 GPUs at the moment.

[Archived Post]