Performance problem with CachingGlue / ExpressionFactory

[simonstratmann]

👓 What did you see?

We execute cucumber scenarios with a high number of threads (> 150). We had massive problems with the tests starting slowly and needing a lot of initial CPU. Profiling the code I found the main bottleneck to be the creation of expressions in io.cucumber.cucumberexpressions.ExpressionFactory#createExpression which is called (via some other methods) by io.cucumber.core.runner.CachingGlue#prepareGlue.
For every thread all glue code classes are parsed and step definitions are created

✅ What did you expect to see?

As the glue code is the same for all scenarios I'd expect it to be parsed once beforehand. I wrote a custom implementation with copies of some of the involved classes which (very hackily) saves the created step definitions of ExpressionFactory in a static map. This reduces the startup time of the tests on our pipeline drastically.

📦 Which version are you using?

7.0.0

🔬 How could we reproduce it?

Start 150 scenarios or more in parallel. Or just two - the basic problem is always there, it's just more apparent with more threads.

🤔 Anything else?

We have more than 5000 step definitions using method annotations and use a lot of custom parameter types which makes parsing them quite heavy.

[mpkorstanje]

As the glue code is the same for all scenarios I'd expect it to be parsed once beforehand.

Unfortunately this isn't true. Due to some unfortunate design choices cucumber-java8 must define step definitions and parameter types at runtime. While in practice they're always the same, we can't quite ensure this.

This particular problem would be resolved by #2279. After that step definitions would only have to be compiled once per language and can be made immutable.

Closing as a duplicate of #2035.

[simonstratmann]

Understood. Thanks for the quick response. One question, though: Wouldn't it be possible to enable caching if cucumber-java8 is not present?

[mpkorstanje]

It would be in theory. In practice I'm not looking forward to the bugs caused by the added complexity.