Yoshimi crashes when there's too many (open) files in Ardour #201
Comments
|
On Sun, 10 Mar 2024 01:52:29 -0800 Lorenzo Miniero ***@***.***> wrote:
This is an old issue I bumped into some time ago, and it seems more a problem in libfltk than in Yoshimi itself, but I thought I'd open an issue anyway, just in case you were not aware of it.
When a session that has too many open files crashed, you won't be able to open the session again either, since it will crash in the same point: the only fix is to manually edit the Ardour session file, remove the Yoshimi LV2 instances, start again, and then clean unused sources to reduce the number of open files before re-adding Yoshimi again. I don't know how frequent having too many open files is, but my Ardour projects have a ton of tracks, and being a sloppy player I do many takes of my parts. Audio captures that were removed from Ardour still live there, to allow for undo's (more context [here](https://tracker.ardour.org/view.php?id=8655), where I originally reported the problem a few years ago).
This is what Ardour gives me when crashing at startup (I snipped all the Ardour calls before, since they're not relevant):
```
*** bit out of range 0 - FD_SETSIZE on fd_set ***: terminated
Thread 1 "ArdourGUI" received signal SIGABRT, Aborted.
__pthread_kill_implementation (threadid=<optimized out>, ***@***.***=6,
***@***.***=0) at pthread_kill.c:44
44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0;
(gdb) bt
c#0 __pthread_kill_implementation
(threadid=<optimized out>, ***@***.***=6, ***@***.***=0)
at pthread_kill.c:44
#1 0x00007ffff47cf8f3 in __pthread_kill_internal
(signo=6, threadid=<optimized out>) at pthread_kill.c:78
#2 0x00007ffff477eafe in __GI_raise ***@***.***=6)
at ../sysdeps/posix/raise.c:26
#3 0x00007ffff476787f in __GI_abort () at abort.c:79
#4 0x00007ffff476860f in __libc_message
***@***.***=0x7ffff48e22e6 "*** %s ***: terminated\n")
at ../sysdeps/posix/libc_fatal.c:150
#5 0x00007ffff4863a79 in __GI___fortify_fail
***@***.***=0x7ffff48e7398 "bit out of range 0 - FD_SETSIZE on fd_set")
at fortify_fail.c:24
#6 0x00007ffff48639c5 in __fdelt_chk (d=<optimized out>) at fdelt_chk.c:26
#7 0x00007ffe8eb77d60 in Fl::remove_fd ***@***.***=1149, ***@***.***=1)
at /usr/src/debug/fltk-1.3.8-6.fc38.x86_64/src/Fl_x.cxx:187
#8 0x00007ffe8eb77e15 in Fl::add_fd
(n=1149, ***@***.***=1, ***@***.***=0x7ffe8eb77b80 <thread_awake_cb(int, void*)>, ***@***.***=0x0)
at /usr/src/debug/fltk-1.3.8-6.fc38.x86_64/src/Fl_x.cxx:115
#9 0x00007ffe8eb77fb6 in Fl::lock ()
at /usr/src/debug/fltk-1.3.8-6.fc38.x86_64/src/Fl_lock.cxx:351
--Type <RET> for more, q to quit, c to continue without paging--c
#10 0x00007ffe4768e329 in YoshimiLV2Plugin::instantiate
(desc=0x7ffe47aec460 <yoshimi_lv2_desc>, sample_rate=48000, bundle_path=0x555582f3a7d0 "/usr/lib64/lv2/yoshimi.lv2/", features=0x555582f3a750)
at /usr/src/debug/yoshimi-2.2.3-1.fc38.x86_64/src/LV2_Plugin/YoshimiLV2Plugin.cpp:427
#11 0x00007ffff59bcc56 in lilv_plugin_instantiate () at /lib64/liblilv-0.so.0
...
```
The problem is that libfltk is using FD_SET internally, but that's known to be severely limited, since due to the mask it uses, it's limited to 1024 descriptors. When too many files are opened (more than that), the call will have undefined behaviour and break. This is why all network applications (or applications using FD_SET for file descriptors) use poll instead.
Again, this is an error in libfltk, not Yoshimi directly, but since Yoshimi is using that library, it's impacted by this problem, which in turn brings everything down.
I've not come across anything like this before, but then I don't use Ardour
apart from basic testing. I doubt there is much -if anything- we can do about
it. However, what version of FLTK are you using? I'm on V1.3.8 and I know
V1.3.9 is available, so it may have been addressed there. Also the FLTK
development branch is V1.4.0 and may have resolved this - but you would have to
build and install that yourself (removing V1.3.x).
Meanwhile I'll ask around the other devs.
…--
Will J Godfrey {apparently now an 'elderly'}
https://willgodfrey.bandcamp.com/
http://yoshimi.github.io
Say you have a poem and I have a tune.
Exchange them and we can both have a poem, a tune, and a song.
|
|
Thanks for the quick answer! I'm on 1.3.8 too, but looking at their github repo, the latest branch still uses FD_SET, and none of the existing pull requests seem to address a change on that. I think the fact I'm using Ardour is only partly relevent: actually, it's only "accelerating" the issue due to my large projects with a ton of tracks, which causes a lot of files to be opened, thus more easily causing file descriptors to go beyond |
|
On Sun, 10 Mar 2024 05:43:34 -0700 Lorenzo Miniero ***@***.***> wrote:
Thanks for the quick answer! I'm on 1.3.8 too, but looking at their github repo, the latest branch still uses FD_SET, and none of the existing pull requests seem to address a change on that.
I think the fact I'm using Ardour is only partly relevent: actually, it's only "accelerating" the issue due to my large projects with a ton of tracks, which causes a lot of files to be opened, thus more easily causing file descriptors to go beyond `1024` in Yoshimi/fltk. In smaller projects (and with better players that don't have to record their part dozens of time :smile: ) it's unlikely to happen.
With further thought this may be a Yoshimi issue :(
If Ardour is holding on to the (currently) unused instance IDs, Yoshimi will
create new IDs. However, we never thought anyone would want more than 31
instances, simply because no processor for the foreseeable future could
actually run that many!
We use the bits of an integer to mark which are in use, so we could change to a
64 bit storage, but that probably only delays the issue.
I'll try artificially reducing the limit and see *exactly* what Yoshimi does
when trying to create another - it should simply refuse to start the new one.
…--
Will J Godfrey {apparently now an 'elderly'}
https://willgodfrey.bandcamp.com/
http://yoshimi.github.io
Say you have a poem and I have a tune.
Exchange them and we can both have a poem, a tune, and a song.
|
|
Managing Yoshimi instances was revised recently, so it might be worth checking this again, to see if it was a Yoshimi problem. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is an old issue I bumped into some time ago, and it seems more a problem in libfltk than in Yoshimi itself, but I thought I'd open an issue anyway, just in case you were not aware of it.
When a session that has too many open files crashed, you won't be able to open the session again either, since it will crash in the same point: the only fix is to manually edit the Ardour session file, remove the Yoshimi LV2 instances, start again, and then clean unused sources to reduce the number of open files before re-adding Yoshimi again. I don't know how frequent having too many open files is, but my Ardour projects have a ton of tracks, and being a sloppy player I do many takes of my parts. Audio captures that were removed from Ardour still live there, to allow for undo's (more context here, where I originally reported the problem a few years ago).
This is what Ardour gives me when crashing at startup (I snipped all the Ardour calls before, since they're not relevant):
The problem is that libfltk is using FD_SET internally, but that's known to be severely limited, since due to the mask it uses, it's limited to 1024 descriptors. When too many files are opened (more than that), the call will have undefined behaviour and break. This is why all network applications (or applications using FD_SET for file descriptors) use poll instead.
Again, this is an error in libfltk, not Yoshimi directly, but since Yoshimi is using that library, it's impacted by this problem, which in turn brings everything down.
The text was updated successfully, but these errors were encountered: