Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Major Performance Improvement #11

Open
JulianFeinauer opened this issue Jan 18, 2019 · 0 comments
Open

Major Performance Improvement #11

JulianFeinauer opened this issue Jan 18, 2019 · 0 comments
Assignees

Comments

@JulianFeinauer
Copy link
Collaborator

From my analysis (see #10) it looks like the main reason for our massive perofrmance degradation is that we use HashMap<String, Object> for the transport of values. But we have too many operations for each record which adds up and leads to several Millions of operations for a mediocre sized batch of messages (140k messages lead to 80 Mio hash calculations on String).

So I suggest to rewrite the value handling internally (not externally) and this should lead to a massive performance gain (I expect x10 - x100 speedup).
There are different ideas I have and I'll try to document it.

But I wanted to open an Issue here to track work on this topic.

@JulianFeinauer JulianFeinauer self-assigned this Jan 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant