DM-41312: Fix blockInfo parsing for minimal data #65
Conversation
| @@ -236,6 +236,15 @@ def augmentData(self): | |||
| but is also much harder to maintain/debug, as the vectorized regexes | |||
| are hard to work with, and to know which row is causing problems. | |||
| """ | |||
| if 'lastCheckpoint' not in self.data.columns: | |||
| self.log.warning("No lastCheckpoint column in data, can't parse block data") | |||
There was a problem hiding this comment.
I would suggest 'lastCheckpoint' to make it explicit that that is the name of the column and not a description, and state that empty columns are being added. Also, is it possible to print any sort of informational message about the data that was being attempted, such as an identifier or just the number of rows?
There was a problem hiding this comment.
Done. (When this happens there are almost always warnings about how little data is being found for the TMA as well btw)
| @@ -236,6 +236,15 @@ def augmentData(self): | |||
| but is also much harder to maintain/debug, as the vectorized regexes | |||
| are hard to work with, and to know which row is causing problems. | |||
| """ | |||
| if 'lastCheckpoint' not in self.data.columns: | |||
There was a problem hiding this comment.
Do you also need this check in augmentDataSlow?
There was a problem hiding this comment.
I actually tested and didn't, but it was because I was getting zero rows of data, and thus the iterrows() there was meaning that line never got hit, so you're right, if I only got few but non-zero lines that would need be necessary! Thanks - added 👍
No description provided.