Use Gherkin parser as soon as it becomes available

[ehuelsmann]

The cucumber project started "Gherkin3", a project which allows to consistently parse Gherkin .feature files across languages (https://github.com/cucumber/gherkin).

[pjlsergeant]

Work in progress here: https://github.com/pjlsergeant/gherkin/tree/master/perl

[pjlsergeant]

This will break use of "Scenario" to mean "Scenario Outline", so I'm unsure about this.

[ehuelsmann]

The Gherkin release notes suggest that in Gherkin 6 Scenario Outline vs Scenario has been resolved:
https://github.com/cucumber/cucumber/blob/master/gherkin/CHANGELOG.md#6013---2018-09-25

[ehuelsmann]

After cucumber/common#694 gets merged the Gherkin parser has everything we need to drive cucumber tests: the generated pickles list all the scenarios and their steps that will be executed. This will also eliminate the need to parse Gherkin "ourselves".

[ehuelsmann]

I'm only inclined to go this way if we can have the protobuf library required to produce the wire protocol data be an optional component in the ecosystem.

[ehuelsmann]

An alternative would be to test our parser with the central testing corpus.

[ehuelsmann]

There's a fully updated Gherkin parser available now on CPAN!

[ehuelsmann]

BTW, changing to the new parser also means to support the Rule keyword, bit at the same time has a big impact on the rest of the program due to the fact that Gherkin has a different setup for the AST of the feature file.

For the execution, things will become simpler, because we will be handed pickles to be expanded into test cases and executed from the there. For the formatters (harnesses), things will be very different as they will need to combine AST data with test results to print scenarios and rules with their outcomes.

[ehuelsmann]

As noted in cucumber/common#768, we need the pickles to be expanded, because the steps in the pickles don't hold a keyword whereas we have always used the keyword for dispatching the correct step function.

[ehuelsmann]

With the merge of cucumber/common#1741, the data that we need is now available in the pickles! All we need is a release and some effort to replace our own parser with the one published by the Cucumber project!

[ehuelsmann]

As of Gherkin 24.0.0 (released to CPAN today) all prerequisites are there to move to the parser from the Cucumber project!

[pjlsergeant]

Great work @ehuelsmann :-D

[ehuelsmann]

First step towards using Cucumber infrastructure taken: released 0.85 today which uses Cucumber::TagExpressions tag expression parser and evaluator.