You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
From my analysis (see #10) it looks like the main reason for our massive perofrmance degradation is that we use HashMap<String, Object> for the transport of values. But we have too many operations for each record which adds up and leads to several Millions of operations for a mediocre sized batch of messages (140k messages lead to 80 Mio hash calculations on String).
So I suggest to rewrite the value handling internally (not externally) and this should lead to a massive performance gain (I expect x10 - x100 speedup).
There are different ideas I have and I'll try to document it.
But I wanted to open an Issue here to track work on this topic.
The text was updated successfully, but these errors were encountered:
From my analysis (see #10) it looks like the main reason for our massive perofrmance degradation is that we use HashMap<String, Object> for the transport of values. But we have too many operations for each record which adds up and leads to several Millions of operations for a mediocre sized batch of messages (140k messages lead to 80 Mio hash calculations on String).
So I suggest to rewrite the value handling internally (not externally) and this should lead to a massive performance gain (I expect x10 - x100 speedup).
There are different ideas I have and I'll try to document it.
But I wanted to open an Issue here to track work on this topic.
The text was updated successfully, but these errors were encountered: