Hang while highlighting JavaScript using Packages#v4050

[Nemo157]

When attempting to highlight some JS code we noticed that it would never return and just spin consuming a full core. We're using syntect with a custom pack built from mostly https://github.com/sublimehq/Packages/releases/tag/v4050. The code:

{
    // this is an inline comment

    "foo": "bar" // another inline comment

    /*
       This is a block comment
       that continues on another line
    */
}

This hangs while parsing the /* line. I've pushed a complete testcase, but since this requires a newer version of sublimehq/Packages than currently tested against I expect that this would need a smaller syntax file to be synthesized for a proper testcase.

[keith-hall]

4050 is already using branch points
https://github.com/sublimehq/Packages/blob/466e8bfa1166fef08282cffdbd0825e12bc8a09f/JavaScript/JavaScript.sublime-syntax#L929-L932
which syntect doesn't support yet.

[Nemo157]

Hmmm, it would be nice if that gave an error while loading. In any case it seems like an incorrect syntax file shouldn't allow infinite processing to occur, and this code snippet shouldn't be hitting the branch point.

[keith-hall]

Good point :)

I have narrowed it down to the line_ending_ahead variable in that sublime-syntax file - removing (?:/\*{{block_comment_contents}})? stops the hang from occurring.

[dandavison]

@keith-hall would you mind explaining why bat doesn't suffer from this problem? Delta uses bat's highlighting assets and is suffering from it.

[keith-hall]

I'm not convinced that bat doesn't suffer from it:

echo -e 'export async function go_to_bar(bar_no) {\n  foo = bar_no\n    /*\n    testing123\n    */\n}\n' | ~/repos/bat/target/release/bat -l JavaScript

hangs, and when I press Ctrl+C, I get the following output:

───────┬────────────────────────────────────────────────────────────────────────
       │ STDIN
───────┼────────────────────────────────────────────────────────────────────────
   1   │ export async function go_to_bar(bar_no) {
   2   │   foo = bar_no

[Nemo157]

IIRC when trying to determine that too I realised that bat was using JavaScript (Babel) for .js files, not JavaScript, using -l js or -l 'JavaScript (Babel)' in the above command runs successfully. I switched docs.rs over to using this too to avoid the issue in rust-lang/docs.rs#1944.

[dandavison]

Thanks @keith-hall and @Nemo157!