-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gen_to_unit incompatible with multiprocessing #85
Comments
I pip installed I don't know if this is the right solution for this project though. |
@pperanich , minimum repro (except needs #90 to show working example): import asyncio
import ezmsg.core as ez
import numpy as np
from dataclasses import replace
from typing import Any, Generator
from ezmsg.util.messages.axisarray import AxisArray
from ezmsg.util.debuglog import DebugLog
from ezmsg.util.gen_to_unit import gen_to_unit
from ezmsg.util.generator import consumer
@consumer
def pow(n: float) -> Generator[AxisArray, AxisArray, None]:
axis_arr_in = AxisArray(np.array([]), dims=[""])
axis_arr_out = AxisArray(np.array([]), dims=[""])
while True:
axis_arr_in = yield axis_arr_out
axis_arr_out = replace(axis_arr_in, data=axis_arr_in.data**n)
AutoPowSettings, AutoPow = gen_to_unit(pow)
class MessageSender(ez.Unit):
OUTPUT = ez.OutputStream(Any)
@ez.publisher(OUTPUT)
async def send_data(self):
yield self.OUTPUT, AxisArray(np.arange(100), dims=["data"])
await asyncio.sleep(1)
raise ez.NormalTermination
if __name__ == "__main__":
comps = {
"SEND": MessageSender(),
"POW": AutoPow(3),
"LOG": DebugLog()
}
conns = (
(comps["SEND"].OUTPUT, comps["POW"].INPUT_SIGNAL),
(comps["POW"].OUTPUT_SIGNAL, comps["LOG"].INPUT)
)
ez.run(
components=comps,
connections=conns,
process_components=(comps["POW"],) # Comment me to work.
) |
Can verify this is still an issue. Maybe best path forward is to add |
But it works fine if I fully define
DummySettings(ez.Settings)
andDummy(Gen)
.I tried providing the
__module__
on the dynamic type defined at the bottom of thegen_to_unit
method...This gets past the first error and might be a good idea anyway, but a different error remains:
Apparently we also (or only) need to provide
__reduce__
. Relevant SO answer.The text was updated successfully, but these errors were encountered: