How to implement an audio codec like libopus in order to save bandwidth

[ludzeller]

Is there any guidance, hook, or template for how to implement an audio codec like libopus in order to save bandwidth? I am using univoice in combination with Unity Relay and the voice transmission quickly creates hundreds of MB per hour per session.

https://github.com/xiph/opus

Implementing libopus as external native library for Unity should be straightforward, but I am specifically asking where in the univoice system one would add the compression and decompression. Any thoughts on that?

[adrenak]

Hi @ludzeller sorry for the super late reply.

Univoice is getting Opus out of the box in version 4 soon. In the meantime you can check out this fork of UnityOpus I made recently: https://github.com/adrenak/UnityOpus

Here is the scripting reference: https://www.vatsalambastha.com/UnityOpus/

There's some more info here: https://www.reddit.com/r/Unity3D/comments/1hgqq5g/opensource_unityopus_encode_your_audio_data_using/

As far as where you'd put this, you can add it to ChatroomAgent where you encode and decode the audio using opus.

[ludzeller]

Hi @adrenak, thanks for getting back on this and sharing your developments! That is super exciting and I am still looking for a solution in this regard, resp. I didn't have time to implement it myself.

Are you also planning to share your specific implementation of your UnityOpus fork into univoice? If I see it correctly, that is not yet available in the code that you shared, right?

I am currently using UniMic and univoice-audiosource-output. Are you also working on implementing libopus for this combination or will it be a more generic, interface-based implementation that you are working on?

By the way, I think I discovered a [memory and performance leak] in univoice-audiosource-output(adrenak/univoice-audiosource-output#5).

[adrenak]

Hi @ludzeller I've release UniVoice v4, much awaited updates.

The main effort has been to move all the things that UniVoice supports out of the box into this single repositories to avoid confusion, make it easier to use and make changes in a single place.l

New features include:

• There's support for incoming and outgoing audio filters.
  • Opus encode and decode filters are provided out of the box.
  • There's also audio gaussian smoothing for some primitive noise removal.
• repositories that were separate packages have been made a direct dependency this includes:
  • univoice-unimic-input is now in the main repository as UniMicInput.cs that uses the Mic class from UniMic
  • univoice-audiosource-output is now in the main repository as StreamedAudioSourceOutput that uses the StreamedAudioSource class from UniMic
• ChatroomAgent class is now ClientSession
• Another key difference. the IChatroomNetwork interface has been split into IAudioServer and IAudioClient the project has the Mirror implementations out of the box in classes MirrorServer.cs and MirrorClient.cs

There's a comprehensive sample, to use it add Mirror to your project and add the UNIVOICE_MIRROR_NETWORK compilation symbol. The sample code should explain just about every feature in detail.

Currently only Mirror network implementation is provided out of the box. I'll be moving the Telepathy, Airpeer and PUN2 implementations soon after which they would be available for activation using compilation symbols like Mirror is now.

But if you need those network implementations now, you can find them in my repositories list, adapt them to the new IAudioServer and IAudioClient interfaces and use them.

Lots of videos coming soon too

[ludzeller]

That's amazing! Will there some update strategy for people who already integrated v3 in their app?

[adrenak]

Yess I'll write an upgrade guide