Best model for isolating dialogues in a film?

[kaswardy]

I am using VR-Architecture 1_HP_UVR and it does a good job, but I was wondering if there is another model that is better suited for isolating the vocal dialogues from a film which may have music and sound effects.

I have a RTX 4080, so I can run a heavier model if needed.

[codebespawler]

I'm not sure if it's the best, but I've been using this ensemble mode reccomended by @sabaasa:

MDX-Net: Kim Vocal 1, UVR-MDX-NET inst 3 & UVR-MDX-NET inst main
Demucs: v4: htdemucs_ft

It's done a pretty good job when I've wanted to isolate dialogue from films.

Full thread with more infomation here https://github.com/Anjok07/ultimatevocalremovergui/discussions/444#discussioncomment-5313230

[marshalleq]

I'm finding / hearing that MDX-Net models are not GPU accelerated / very slow. Out of interest is this your experience too? This is such a fantastic tool, only just discovered it. I would never have dreamed something could do what this can do, it's like magic.

[codebespawler]

For me a 4:07 song takes around 2:30 to process. It appears to be using my NVIDIA 3050 Ti, but I use Demucs in my ensemble mode, so not sure if it's using the GPU just for that model and not the MDX-Net ones.

[RyLeo154]

I'm finding / hearing that MDX-Net models are not GPU accelerated / very slow. Out of interest is this your experience too? This is such a fantastic tool, only just discovered it. I would never have dreamed something could do what this can do, it's like magic.

@marshalleq GPU acceleration on MDX-NET models seem to work just fine on my home desktop's RTX 3060 Ti on all other MDX-Net models I've tested so far like Kim Vocal 2, Inst HQ 1, Inst HQ 2 and Inst HQ 3 seem to properly utilize the GPU although utilization was sometimes misreported in Task Manager; I tried inferencing a sample file (3m 17s, 128kbps mp3) with MDX-Net Inst HQ 2 with my laptop's GTX 1060 Max-Q and here's my results:

^ results of GPU-accelerated inferencing, 1m 15s to process the sample file on a GTX 1060 Max-Q
(additional footnote: my 3060 Ti processed 3-minute 320kbps mp3 files in under a minute, I'll update this post later with results when I'm able to use my desktop to process the same sample file later)

^ results of CPU-only inferencing, 9m 26s to process the same sample file on a 4-core i7-7700HQ (took this screenshot after a small delay; the last 50% of the CPU histogram represented CPU utilization during inference)

1m 15s (GPU-accelerated) vs 9m 26s (CPU-only) is a pretty stark difference in processing time, so GPU acceleration is working fine on my end right out of the box, no additional tweaks needed

[kdcyberdude]

@codebespawler @marshalleq MDX-Net models aren't utilising the GPU? Do you have any idea why that might be happening?

[PodRED]

As a complete aside if you have access to the original 5.1 / 7.1 audio you can basically always grab just the voice by extracting the centre channel track on it's own as nothing else generally gets put in centre channel.

[COOLak]

As a complete aside if you have access to the original 5.1 / 7.1 audio you can basically always grab just the voice by extracting the centre channel track on it's own as nothing else generally gets put in centre channel.

There's a LOT of other stuff in the center channel. Believe me.