-
Notifications
You must be signed in to change notification settings - Fork 228
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
After adding 10000 centroids and merging I am get NAn Value #165
Comments
Hey there. I can't tell exactly what you are doing. Normally what would be done would be to add samples (not centroids) to a t-digest for some short window (1 or 5 minutes is common). These digests could be serialized and sent to another location. This other location could deserialize the digests and combine or augment them. The things that you say that confuse me are
Can you clarify exactly what you mean? Could you also say which version of t-digest you are using? If you have sample code, it would help enormously. |
HI , 1.We have one application which produce key and value pair data through kafka, here value(protobuf object) has mean and weight. So here problem is on second point the published data has some NAN mean values not sure what causing the mean values to become Nan and when consume as for 3rd point we are getting this below error error :java.lang.IllegalArgumentException: Cannot add NaN to t-digest ----we are using t-diges3.2 version on second point code tDigestWrapper---- has the actual t-digest object which collected mean and weight //decay t-digest before publishing // custom DTO object to publish data
|
Hmm... there are going to be some serious problems here.
It is usually possible to avoid the temptation to decay the weights in a t-digest by using digests devoted to individual time windows. Generally, you can combine unweighted digests transparently. If you seriously want to emphasize some windows at the expense of others, you should consider doing the weighting in your downstream statistical processes rather than inside the t-digest itself. Can you say why you aren't adding the original raw observations to a t-digest? Why are you adding means with weights? And why you aren't just using windows? Where does the requirement for decaying a t-digest come from? |
I haven't heard back on this in over 6 months and will likely close this issue soon if I don't hear more. |
I run into the same error
I added 87087.0 double numbers by calling com.tdunning.math.stats.AbstractTDigest#add(double) from a MergingDigest instance with compassion of 10000. At some point, some of the mean[] elements have Nan values. Here is the hex format of the binary of the MergingDigest |
Hi Team, I am adding stream of centroids global t-digest and after some interval of time, apply decay and I am sending these centriod t-digest ( global t-digest to another system through Kafka , when I consume and trying add to another t-digest, I am get NaN for values , what could be the resaon ,please guide if am doing wrong ??
The text was updated successfully, but these errors were encountered: