new USFM crash

[johnml1135]

For Engine 67670ddcdf523878b567e8f1...

Serval.Machine.Shared.Services.PreprocessBuildJob[0]
      Build faulted (67670e60df523878b567e904)
      System.InvalidOperationException: An error occurred while parsing the text 'EXO' in project 'ABC'. Verse: EXO :0, line: 421, character: 31, error: 'Stack empty.'
       ---> System.InvalidOperationException: Stack empty.
         at System.Collections.Generic.Stack`1.ThrowForEmptyStack()
         at SIL.Machine.Corpora.UsfmTextBase.TextRowCollector.EndVerseText(UsfmParserState state, IReadOnlyList`1 scriptureRefs)
         at SIL.Machine.Corpora.ScriptureRefUsfmParserHandlerBase.Verse(UsfmParserState state, String number, String marker, String altNumber, String pubNumber)
         at SIL.Machine.Corpora.UsfmParser.ProcessToken()

[Enkidu93]

Is this a test project or have you intentionally obscured the project name?

[johnml1135]

I'm checking it out now. I'll post more data here soon (yes, I obscured the project name because I don't know if it is sensitive or not).

[johnml1135]

Minimum USFM to reproduce:

\id MAT - Test
\c 1
1 This is the 1\v 00 verse.

[johnml1135]

@Enkidu93 - I was able to identify the issue and make it reproducable (see https://github.com/sillsdev/machine/tree/usfm_error_589). Can you bring this to completion?

[johnml1135]

@Enkidu93 - I assigned you to this - would you be able to finish this USFM bug to completion?

[Enkidu93]

@Enkidu93 - I assigned you to this - would you be able to finish this USFM bug to completion?

Yes, I can do that. I've been more-or-less focused completely on word alignment (unless something urgent comes up). I was planning on tackling some of these once I'm waiting on reviews for word alignment (which should be very soon), but if any need to be prioritized, let me know & I can pause the word alignment work.

[johnml1135]

I would like to try to cut a version of Serval in the next few days. If this could be finished up soon, that would be great.

[Enkidu93]

@johnml1135 I'm not sure quite what to do with this issue; it's a bit different than some of our other USFM bugs: What's the intended behavior? This USFM is pretty messed up 😆 . How should this be interpreted?

[johnml1135]

@Enkidu93 - This may be a good opportunity to have a try/catch around each call to Process token and if it really does crash, to walk forward until it either hits a new paragraph or a new verse. That should make it "crash proof." We should likely still record the catch as an error so we can fix the USFM bug, but the user will not see any meaningful difference.