[TKNECO-42] RnD Test framework for TKNECO

[jitendar-singh]

• Using Godog Cucumber BDD framework for Golang
• Defining test scenarios
• Adding step definition for scenarios
• Updating the Makefile
• Adding gofmt script

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jitendar-singh
Once this PR has been reviewed and has the lgtm label, please assign otaviof for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

Hi @jitendar-singh. Thanks for your PR.

I'm waiting for a openshift-pipelines member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[otaviof]

/ok-to-test

[jitendar-singh]

/retest

[vdemeester]

So I am not sure if I want us to go toward those types of tests. Mainly, this repository (and the main branch) will hold a few "generic or experimental" tasks (that we don't know where to but) and those might be easier to test in a very generic way or the way things like task-git work. I feel we are starting with a very complicated setup.

What we want to provide to users, is a way to easily write tests for their tasks, and this without any prior knowledge of Go or anyting except Tekton. The though is, if they author a Task, they most likely already try it out with TaskRun and PipelineRun (or at least they know how to write those). So allowing them to tell us : this is the yamls to run for testing this task seems ok.

In addition, we'll setup some "pull mechasims" to get Task (and other resources long-term) from external repositories (releases). Those repositories will have to follow some gates we set, and most likely, some of these will be "provide us some smoke test we can run on our infra". And those ought to be simple, yamls that we can execute to start with seems like the simplest start.

For example, there is no need for a .feature to test that the pipeline instance is READY, this is an infra issue and an "assumption" the "test framework" itself should make.

[jitendar-singh]

So I am not sure if I want us to go toward those types of tests. Mainly, this repository (and the main branch) will hold a few "generic or experimental" tasks (that we don't know where to but) and those might be easier to test in a very generic way or the way things like task-git work. I feel we are starting with a very complicated setup.

What we want to provide to users, is a way to easily write tests for their tasks, and this without any prior knowledge of Go or anyting except Tekton. The though is, if they author a Task, they most likely already try it out with TaskRun and PipelineRun (or at least they know how to write those). So allowing them to tell us : this is the yamls to run for testing this task seems ok.

In addition, we'll setup some "pull mechasims" to get Task (and other resources long-term) from external repositories (releases). Those repositories will have to follow some gates we set, and most likely, some of these will be "provide us some smoke test we can run on our infra". And those ought to be simple, yamls that we can execute to start with seems like the simplest start.

For example, there is no need for a .feature to test that the pipeline instance is READY, this is an infra issue and an "assumption" the "test framework" itself should make.

@vdemeester @otaviof
But what I think of this framework is that "This is a Behavior-driven development (or BDD) test framework . BDD is an agile software development technique that encourages collaboration between developers, QA and non-technical or business participants in a software project.

The end goal with this framework is to achieve the QA verification of any Jira Stories development takes on for the OpenShift-pipelines TEKECO, as well as any regression test suites the team wants to cultivate.

Hence, the reader should consider these tests that run in OpenShift CI prow jobs as part of merging pull requests with code changes. These provide another means of testing that are focused on the end user experience, in language and terms that end users can understand.

The feature file testing the pipeline instance is READY was just for dummy purpose, as I am not aware of what tasks we are about to test, so just for the sake of example added it.

[vdemeester]

@jitendar-singh yes, but this works well when there is a developer, and some code, in some language that is not necessarily shared between all the shareholders (dev, QA, non-technical). Here, for me, it's a bit different. The "common" language is yaml already. And, at least on this repository, what we want to ensure is that the Task we pull from elsewhere, are tested and go through all the gates we set.

[openshift-merge-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.