outline: is there a way to resolve the node text returned from func?

[dhiller]

Hey, first of all thank you for your awesome work - the KubeVirt community is using onsi/ginkgo a lot!

As background of my question: I am working on tooling to re-run selected tests that have been changed to detect flakes.

In the KubeVirt e2e test codebase we are aiming to avoid repetition of labels and test name prefixes using wrapper functions SIGDescribe that call Describe to inject additional text and args in the call.
Since that obviously breaks the ginkgo outline mechanism (those nodes are missing as the identNames don't match any case of the internal switch 1).

I refactored the ginkgo.Describe call by replacing it with a wrapper that returns the extended text and args instead 2 . This restores the original Describe call and therefore the node reappears in the outline.

That said, our ideal goal would be to have the full ginkgo node text in the outline to avoid partial test names, which currently is not supported as the ginkgo outline construction can't resolve the func to a literal 3 .

Is there a way to achieve this somehow that I have overlooked? Would I need to go the route of resolving the text myself by following the AST? Or do you have another idea that I should look into?

[onsi]

Thanks for the kind words!

Since ginkgo outline is parsing the source it has some inherent limitations around resolving dynamically generated specs.

I see two options:

1. Use Ginkgo's JSON output to (re)build your tooling. This should have all the information you need and provides a stable interface that is intended to be both machine readable and complete.

2. Use ginkgo dry-run in verbose mode to get console output that includes every spec name. Since this compiles and runs the binary through the tree construction phase and simply stops short of executing any lead nodes it should give you the complete list of specs.

I do think #1 is worth a bit of investment. It could pay dividends for much of the k8s community as it will let you analyze slice and dice test runs much more effectively programmatically.

[dhiller]

Hey, thanks for the fast reply!

We actually are using the ginkgo outline for some of our tooling already, i.e. to map the outline to git blame lines and generate a report over selected tests 1 and (a WIP though) to determine the names of tests that were changed during the course of a PR 2 - although this is only part of a bigger effort in re-running optimization.

I have the latter largely working and maybe the partial test name problem is even not a real one, so it might work well enough already.
If it doesn't I will elaborate the tool to parse the AST myself and add the information programmatically.

Thank you!

[dhiller]

One note though: calling the ginkgo outline programmatically was giving us a hard time, since the command func was not exported - or am I just holding it wrong ?