Skip to content
This repository has been archived by the owner on Aug 23, 2024. It is now read-only.

True gapless playback #316

Open
stefan1983 opened this issue May 5, 2022 · 24 comments
Open

True gapless playback #316

stefan1983 opened this issue May 5, 2022 · 24 comments
Labels
1.0.0-rewrite enhancement New feature or request

Comments

@stefan1983
Copy link

stefan1983 commented May 5, 2022 •
edited
Loading

Hey there, is it planned to make Sonixd able to support true gapless playback? Most of my albums are Mix-CDs, Live Albums etc. and the short but still significantly hearable gap is a bit annoying between each new track.

For iOS I use amperfy and the engineer is looking into the same topic currently. Maybe a look here helps:
(BLeeEZ/amperfy#96 (comment))

Thanks a lot and keep it up.
@stefan1983 stefan1983 added the enhancement New feature or request label May 5, 2022
@jeffvli
Copy link
Owner

jeffvli commented May 5, 2022

Having looked at other options, I don't think I'll be able to change the overall player implementation.

Can you name me some albums where you're noticing gaps? I can do some adjustments on my side to see if I can make the transition between tracks more accurate.

@stefan1983
Copy link
Author

Sure, I could provide some tracks to you if that helps.

@stefan1983
Copy link
Author

By the way, does the current implementation make a difference if the source is MP3 or FLAC?

@jeffvli
Copy link
Owner

jeffvli commented May 5, 2022

The source does not matter as long as it's playable by the browser/server.

@jeffvli
Copy link
Owner

jeffvli commented May 6, 2022

I did some testing with the albums you provided me, and is indeed noticeable gaps on certain songs in the albums.

I'm not able to get it perfect with the current player... but I'll see if I can explore some other solutions for this.

@stefan1983
Copy link
Author

Thanks for looking into it, hope you find something ;)

@jeffvli
Copy link
Owner

jeffvli commented Jun 6, 2022 •
edited
Loading

sonixd-1.0.0-alpha-player-demo.mp4

Adding a new player may be possible. I've been building out a custom player for the 1.0.0 version using the following:

  • PyAudio
  • FFmpeg

So far I've only implemented regular/gapless playback, but I'll eventually experiment with crossfade as well if I can get it stable enough. I've attached a demo with one of the albums you provided.

There are a few issues such as not being able to dynamically seek and change the volume without restarting the stream (causes a slight pause when changing the volume), but overall the implementation seems to work. It will be controlled using HTTP websockets from a flask webserver which is running the player.

If someone more experienced with Python/C or any other languages can help build out a simple player that I can integrate I'd be welcome to that as well.

@stefan1983
Copy link
Author

stefan1983 commented Jun 6, 2022 via email

@dweymouth
Copy link

Have you looked into using libmpv for the player backend? It should support true gapless playback and the ability to natively play all major audio formats without needing to invoke ffmpeg commands. You can see how the sublime-music client is using libmpv with Python here

@jeffvli
Copy link
Owner

jeffvli commented Jun 29, 2022

@dweymouth I actually did take a look at sublime music when trying to decide how to implement the new player.

The requirements at the time were:

  • Needs to be able to stream from an html stream (this includes m3u8 audio streams that are returned by Jellyfin's transcodes)
  • Needs to support gapless playback (pre-loading the next X number of tracks)

I'm not sure if MPV supports streaming from html, but if so that would be an option.
I've been doing a lot of thinking on whether or not I want to adopt a full offline approach for the desktop app -- where songs would need to be fully downloaded before playback starts (I believe Sublime follows this approach?). This way, users wouldn't need to redownload the song every time they wanted to play it.

I'll do a bit of testing with MPV though, since I do also see some node packages that may be usable to avoid having to include a python dependency.

@dweymouth
Copy link

dweymouth commented Jun 30, 2022 •
edited
Loading

Another possibility to avoid a Python dependency would be to write the backend player in C or C++, and use libffmpeg directly (or keep the approach of invoking the ffmpeg binary as a subprocess) and Miniaudio (other cross-platform C++ audio libraries exist, but I can attest that miniaudio is super easy to integrate and works out-of-the-box on Mac and Linux - don't currently have a Windows box).

You could use sockets to make calls to the C++ player like with the Python prototype, or probably a Node native module. I'm not a frontend or Node dev so can't offer too much advice there. If you are able to set up a c++ player stub that can be invoked from the FE and integrates into the project build system I may be willing to give the implementation a go. Can't give any promises on my time availability though.

But do give the node packages a try too - if any of them support true gapless playback all this would become moot!

@jeffvli
Copy link
Owner

jeffvli commented Jun 30, 2022

I don't have any knowledge of C/C++ so that's why I ended up using Python. Most of the Python modules are just bindings for C++ libraries anyways, which did include Miniaudio, PortAudio, etc. I was also looking at Irrklang as a possible solution as well, but I found the documentation to be lacking.

I'll try out an mpv implementation this weekend and update this thread on how it goes. If it's not feasible, I'll see what I can do in terms of setting up that player stub for you. It will most likely be fairly simple, with the player running as a separate process from the client (similar to how the mpv implementation will work as well I'm assuming).

From my testing, I found sockets to be usable but also a bit clunky as there's lag between the player controls and the audio player. I was also thinking of having the player use keyboard shortcuts to manage the controls to eliminate the need for sockets for the player controls (though I would need to do a bit of testing to see if this is possible from the frontend client).

@dweymouth
Copy link

dweymouth commented Jul 1, 2022 •
edited
Loading

OK, a totally new idea - you might be able to stick entirely with Javascript if you use the WebAudio API with the AudioBuffer source, and use one of the various Node packages that wraps ffmpeg or any other Node audio codec libraries to decode data from the stream into raw samples to fill the Buffer. Then for gapless, you just start decoding the next stream before your buffer is empty and you have a seamless transition between the tracks. Again I'm not a web dev so I have no idea if this would work but I don't see why it wouldn't.

Edit: this might be worth exploring too: https://www.npmjs.com/package/node-mpv

@jeffvli
Copy link
Owner

jeffvli commented Jul 4, 2022

I've done a bit of testing with the Webaudio API + ffmpeg, but wasn't satisfied with the workability of it. Python seemed to more easily be able to handle all the requirements I was looking for.

I started playing around with the node-mpv package you linked, and I think it will actually work perfectly here. This will most likely be the player approach moving forwards, since it will save a lot of headache from having to build player from scratch. Hoping to have a working prototype in the next couple of days so that I can go into full gear with the rest of the backend/frontend development.

@dweymouth
Copy link

dweymouth commented Jul 5, 2022 •
edited
Loading

Does node-mpv play gaplessly? (I haven't tested but it should, as long as the next track is pre-loaded in the playlist b/c it's just using mpv under the hood.) Seems like the only potential headache here would be figuring out how to bundle mpv(.exe) into the installation package for all the OSes so the software can work out-of-the-box vs. requiring the user to download mpv and point Sonixd to it. IANAL, but since both Sonixd and mpv are GPL I think you should be good to go, license-wise, with bundling.

Edit: looks like mpv gapless playback is disabled by default unless you pass --gapless-audio=yes or --gapless-audio=weak (which plays gapless as long as the next track has the same sample format as the previous) to the mpv instance.

@jeffvli
Copy link
Owner

jeffvli commented Jul 5, 2022

If allowed by the license, packaging the mpv executable shouldn't be too hard, since electron builder handles most of the work there with the extraResources configuration.

From my initial testing, gapless was working out of the box with node-mpv, but custom parameters can also be added onto the mpv instance so there shouldn't be too many issues there. I may also try implementing a dual-instance crossfade similar to what I was doing with the html audio players but I'll have to see how mpv handles the volume first.

@stefan1983
Copy link
Author

Hey Jeff, any idea when a first test-build will be available with gapless playback?

Thanks
Stefan

@jeffvli
Copy link
Owner

jeffvli commented Oct 12, 2022

Hey Jeff, any idea when a first test-build will be available with gapless playback?

Thanks Stefan

Sorry for the delayed response. The project was on hold for a while due to conflicting prorities but I've now resumed work on the rewrite. I don't have a for-sure timeline but keep an eye on the rewrite repository for any changes.

#332

@jeffvli
Copy link
Owner

jeffvli commented Nov 21, 2022

The first alpha release of Feishin has been released which uses MPV for gapless playback.

@stefan1983
Copy link
Author

stefan1983 commented Nov 22, 2022 via email

@stefan1983
Copy link
Author

stefan1983 commented Nov 22, 2022 via email

@jeffvli
Copy link
Owner

jeffvli commented Nov 24, 2022

@stefan1983 Yes that's correct. Feishin uses its own server.

I'll be attempting to add mpv player into Sonixd as well so hang tight on that.

@stefan1983
Copy link
Author

stefan1983 commented Nov 27, 2022 •
edited
Loading

I see... 😁

image

So shall I wait for the update or point it manually (e.g. via homebrew)?

@stefan1983
Copy link
Author

Gapless playback works in Feishin, I tested it successfully today with MP3 as well as FLAC files (via Navidrome).

Great work! Thanks

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
1.0.0-rewrite enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dweymouth @stefan1983 @jeffvli