Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

/supervoxel-splits response is missing some events #316

Open
stuarteberg opened this issue May 16, 2019 · 3 comments
Open

/supervoxel-splits response is missing some events #316

stuarteberg opened this issue May 16, 2019 · 3 comments
Labels

Comments

@stuarteberg
Copy link
Member

stuarteberg commented May 16, 2019

The /supervoxel-splits endpoint is supposed to mention all** of the split and split-supervoxel mutations that can be found in the kafka log, but I have a few examples from the kafka log that are missing from the /supervoxel-splits output.

The following 7 split events are missing from the /supervoxel-splits output:

In [34]: msgs_df.query('mutid in @missing_muts').drop(columns=['msg'])
Out[34]:
                     timestamp                              uuid       mutid            action  target_body   target_sv
1076   2018-10-16 14:02:59.627  ef1da1a01c4941b6876ab77502843d0c  1002217029             split   5812981673           0
1079   2018-10-16 14:03:00.628  ef1da1a01c4941b6876ab77502843d0c  1002217029    split-complete   5812981673           0
1082   2018-10-16 14:04:50.683  ef1da1a01c4941b6876ab77502843d0c  1002217032             split   5812981673           0
1083   2018-10-16 14:04:51.403  ef1da1a01c4941b6876ab77502843d0c  1002217032    split-complete   5812981673           0
1214   2018-10-16 14:22:18.843  ef1da1a01c4941b6876ab77502843d0c  1002217098             split    394768379           0
1225   2018-10-16 14:22:19.984  ef1da1a01c4941b6876ab77502843d0c  1002217098    split-complete    394768379           0
1440   2018-10-16 14:25:18.639  ef1da1a01c4941b6876ab77502843d0c  1002217211             split    394768379           0
1441   2018-10-16 14:25:19.763  ef1da1a01c4941b6876ab77502843d0c  1002217211    split-complete    394768379           0
1454   2018-10-16 14:27:04.146  ef1da1a01c4941b6876ab77502843d0c  1002217218             split    394768379           0
1455   2018-10-16 14:27:05.032  ef1da1a01c4941b6876ab77502843d0c  1002217218    split-complete    394768379           0
1456   2018-10-16 14:28:40.281  ef1da1a01c4941b6876ab77502843d0c  1002217219             split    394768379           0
1457   2018-10-16 14:28:40.995  ef1da1a01c4941b6876ab77502843d0c  1002217219    split-complete    394768379           0
1488   2018-10-16 14:31:51.632  ef1da1a01c4941b6876ab77502843d0c  1002217235             split    394768379           0
1489   2018-10-16 14:31:52.726  ef1da1a01c4941b6876ab77502843d0c  1002217235    split-complete    394768379           0

**Unless the event has no corresponding split-complete message in the kafka log. But that doesn't apply to the examples above, obviously.

@DocSavage DocSavage added this to the v1.0 milestone Nov 1, 2019
@DocSavage
Copy link
Member

@stuarteberg In the case of a kafka queue overload, there could be a situation where a split-complete message is successful but the actual split is not. Given that we have the overflow logs more recently, do we still believe this is a problem on the DVID side and not a kafka queue overload issue?

@stuarteberg
Copy link
Member Author

@DocSavage

there could be a situation where a split-complete message is successful but the actual split is not

I don't understand. I guess I'll talk to you in person.

@DocSavage
Copy link
Member

The above was based on incorrect view of events missing from kafka log not from the internal representation. Just to verify, the only missing supervoxel splits were from the broad split label request and not missing supervoxel-split requests, right? (The actions are listed as "split" and not "split-supervoxel".)

@DocSavage DocSavage added the bug label Nov 2, 2019
@DocSavage DocSavage removed this from the v1.0 milestone May 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants