matlab.engine and multiprocessing.Process don't play nicely together

[alexhroom]

see the following example: using PoolExecutors for simplicity. first with threads:

import matlab.engine
from concurrent.futures import ThreadPoolExecutor

def matlab_func(i):
    eng = matlab.engine.start_matlab()
    return eng.sqrt(i)

with ThreadPoolExecutor() as executor:
    future = executor.submit(matlab_func, 4.0)
    print(future.result())

vs the same with Processes:

import matlab.engine
from concurrent.futures import ProcessPoolExecutor

def matlab_func(i):
    eng = matlab.engine.start_matlab()
    return eng.sqrt(i)

with ProcessPoolExecutor() as executor:
    future = executor.submit(matlab_func, 4.0)
    print(future.result())

i find that the thread example works, and the process example hangs forever. if other people can reproduce this i will file an issue with the MATLAB engine package. In any case, this means that MATLAB custom models currently don't run. If I change the runner to use threads instead of processes, it works correctly, but the big downside is that threads can't be interrupted!

This is with matlabengine version 24.2.1, MATLAB R2024b. (edit: still happens with 24.2.2 and MATLAB R2024b update 3)

[StephenNneji]

This worked on my windows machine. This is on matlabengine v9.13.9 which is for 2022b, also tested on 2024b win, and 2021a linux all work

import matlab.engine
import multiprocessing
from concurrent.futures import ProcessPoolExecutor

def matlab_func(i):
    eng = matlab.engine.start_matlab()
    return eng.sqrt(i)

if __name__ == '__main__':
    multiprocessing.freeze_support()
    with ProcessPoolExecutor() as executor:
        future = executor.submit(matlab_func, 4.0)
        print(future.result())

[alexhroom]

@StephenNneji are you able to try 2024b on Linux? that's really the operative setup as it'll be what's running on IDAaaS (2024b is the first release to support matlab engine on python 3.12). multiprocessing.freeze_support() does nothing at all on Linux so your script and mine shouldn't be doing anything different on Linux?

or alternatively would you be able to try the original place I found the issue: import this RasCAL-1 project and try to run it (you'll need this custom model file in the folder that you're running RasCAL-2 from). for me, this logs that it's started RAT, and then it freezes indefinitely for all procedures (but works if i change the RAT runner to use Threads, which is what I had to do for the demo yesterday)

[StephenNneji]

@alexhroom I can confirm it hangs when I run on Linux with 2024b

[alexhroom]

created issue: mathworks/matlab-engine-for-python#50