Improve inference performance with loaded TransformerChain ML.NET model #371
Conversation
|
@ganik should I add a small ML.NET model file to the project and write a test for inferencing with a loaded ML.NET model? |
|
yes, that would be great In reply to: 556551197 [](ancestors = 556551197) |
| all_nodes = [] | ||
| if is_transformer_chain: | ||
| if (hasattr(self, '_is_transformer_chain') and | ||
| self._is_transformer_chain): | ||
| inputs = dict([('data', ''), ('transform_model', self.model)]) |
There was a problem hiding this comment.
How much is perf gain? If its not much I would like to leave this as it is. Reason is that we actually should move to new ML.NET format so then this will be broken with your new fix
There was a problem hiding this comment.
6% gain, with .predict called on 100 row UCI Adult test data, in a 100 rep for loop, repeated 5 times.
I think we should take the change. (1) If we leave it as it is, it will still be broken when NimbusML moves to new ML.NET format, and how the graph is constructed will need to change anyway. (2) Moving to new ML.NET format will likely take a long time, as it requires non-trivial changes to how model loading is handled in the entrypoints infrastructure on ML.NET side i.e. new model implementation in addition to PredictorModel and TransformModel. (3) The changes directly below this address an issue where PredictedLabel column is only converted to int32 from bool if it exists (i.e. it is not regression or ranking) and if the dtype is bool (i.e. if it is Binary classification).
| @@ -148,6 +150,12 @@ def test_model_datastream(self): | |||
|
|
|||
| os.remove(model_filename) | |||
|
|
|||
| def test_mlnet_model_can_be_scored(self): | |||
| data = FileDataStream.read_csv(test_file, sep=',', numeric_dtype=np.float32) | |||
There was a problem hiding this comment.
ML.NET model is trained on a file with label, and it expects label to be present in the schema of the data being passed to predict
Fix #370
PR #230 introduced ability to load and score ML.NET models trained in the new ML.NET
TransformerChainserialization format. This is done by checking whether "TransformerChain" exists in the archive members. Currently, this is done every timetest,predict,predict_proba, anddecision_functionmethods call_predict. ThisPR improves the performance by checking for "TransformerChain" only once when the model is loaded.