Automatic Serialization

[LarsKue]

The goals of this PR are:

• Remove the need to implement from_config and get_config methods for most objects
• Remove the need to save a config object in the constructor of most objects
• Unify the serialization of types, functions, and other objects
• Improve the serialization of functions, especially with respect to numpy

To achieve the above goals, this PR introduces:

• A custom serialize and deserialize function that wrap around the keras utilities
• A custom Serializable class that automatically implements from_config and get_config

[stefanradev93]

That's a super nice utility! Which networks are already tested through?

[LarsKue]

It's still in progress, so not functional for all networks yet. But I made changes such that most networks should already be tested.

[vpratz]

This looks really nice! Please take care when removing get_config functions, some attributes might accumulate state that is not automatically serialized by Keras, so already serializing them when __init__ is called might not be sufficient. I'm not totally sure, but I think this might apply for example for ContinuousApproximator.adapter.

[LarsKue]

@vpratz Thank you for raising the issue. After investigating further, I opted to go with a semi-automatic and more classical OOP approach of using a Serializable base class that implements from_config and get_config and allows semi-automatic saving of stateless and stateful variables with a single self.initialize_config() call. Could you take a brief look?

[vpratz]

Thanks for taking this into account. I currently don't have a clear opinion, so I'll just share some thoughts:

• can we somehow get rid of the self.initialize_config() without arguments, e.g. by moving it to __init__subclass? Instead I would opt for a register_stateful_variables function that appends to self.config['statefule_fields'].
• if we make the creation of config implicit, do we want to make it private/protected, i.e., rename it to self._config?
• currently, I think in most places we follow the convention that everything that is not Keras weights but needed to construct an object is passed through __init__. With those changes we do away with this convention, which is okay in my opinion, but it comes at the cost that we can no longer point at how Keras does it to explain how we do it.
• I liked that the decorator was somewhat explicit. The sub-sub-classing might make it hard to track what is going on, so we need to document it prominently for devs.

[LarsKue]

Thanks for sharing. To address your points:

• No, it is not possible to safely get rid of the call entirely. There is no way to inspect the actual values that are passed to the constructor from the outside without major hacking (e.g. replacing the constructor function entirely) or additional user code (e.g. another decorator on the __init__)
• __init_subclass__ is called when the class is created, not the instance.
• We could make a register_stateful_fields function, but I would opt to make it in addition to the initialize_config

make self.config protected

• I have thought about this, but I don't mind either solution. I think it is ok to have the config attribute public, since it is only created semi-automatically.

everything is passed through init

• This is incorrect. Any class that internally changes state violates this property. We still mostly follow the keras idiom in that we remember the constructor arguments and by default assume no such internal state changes.

• I agree that the decorator is more explicit, but also not particularly typical for OOP and too limiting for the automation of this process. In particular, if we create the from_config and get_config methods in a decorator, static type checkers and auto completion will have a hard time figuring out what is going on. Allowing to override the methods is then also complicated. Essentially, we would emulate OOP by non-OOP methods, which is just not idiomatic.

[vpratz]

Thanks a lot for the detailed response. With this additional information on the constraints, I'm happy with the proposed solution.