Adding system tests #399
Adding system tests #399
Conversation
* install matlab-batch to MATLAB_HOME/bin as workaround for node paths * remove unused getNthParent method
* Delete unnecessary tests * Move some tests into 'systemTests' folder
Update release format for install automatically
| @@ -1,136 +1,136 @@ | |||
| package unit.com.mathworks.ci.actions; | |||
There was a problem hiding this comment.
I never noticed this before but we have separate test packages. In the Bamboo plugin we use the same package. Should the package be updated or are we going to keep them in separate packages?
| @@ -0,0 +1,28 @@ | |||
| name: using mpm to install MATLAB | |||
There was a problem hiding this comment.
I like that we're using GitHub Actions. Does this mean we can remove the Azure Pipelines yaml?
There was a problem hiding this comment.
Yeah, we only run 'mvn verify' command in azure pipelines as well. So we can remove that.
.github/workflows/main.yml
Outdated
| name: using mpm to install MATLAB | ||
| on: push | ||
| jobs: | ||
| my-job: |
There was a problem hiding this comment.
Could maybe give this job a more descriptive name in case we want to add more in the future.
There was a problem hiding this comment.
Updated the job name to 'run-tests-job'. Hope this makes sense.
| FreeStyleBuild build = project.scheduleBuild2(0).get(); | ||
|
|
||
| jenkins.assertBuildStatus(Result.FAILURE, build); | ||
| jenkins.assertLogContains("Verify global tool configuration for the specified node.", build); |
There was a problem hiding this comment.
This could fail in a future Jenkins update, which is just something that we should be aware of in case it ever shows up.
There was a problem hiding this comment.
Yeah, the message could change. I added a note specifying the same.
| @Test | ||
| public void verifyMATLABscratchFileNotGenerated() throws Exception { | ||
| this.buildWrapper.setMatlabBuildWrapperContent( | ||
| new MatlabBuildWrapperContent(Message.getValue("matlab.custom.location"), Utilities.getMatlabRoot())); | ||
| project.getBuildWrappersList().add(this.buildWrapper); | ||
| scriptBuilder.setTasks(""); | ||
| project.getBuildersList().add(this.scriptBuilder); | ||
| FreeStyleBuild build = project.scheduleBuild2(0).get(); | ||
| File matlabRunner = new File(build.getWorkspace() + File.separator + "runMatlabTests.m"); | ||
| Assert.assertFalse(matlabRunner.exists()); | ||
| } |
There was a problem hiding this comment.
I think this test case is probably not necessry.
There was a problem hiding this comment.
Yeah, I will delete these tests
src/test/java/com/mathworks/ci/systemTests/RunMatlabBuildIT.java
Outdated
Show resolved
Hide resolved
src/test/java/com/mathworks/ci/systemTests/RunMatlabBuildIT.java
Outdated
Show resolved
Hide resolved
src/test/java/com/mathworks/ci/systemTests/StartupOptionsIT.java
Outdated
Show resolved
Hide resolved
src/test/java/com/mathworks/ci/systemTests/UseMATLABVersionIT.java
Outdated
Show resolved
Hide resolved
This PR has following changes:
System tests needs access to MATLAB to run successfully. The MATLAB root path is read from 'MATLAB_ROOT' env variable. These tests are configured to run only if the 'MATLAB_ROOT' variable is defined. This is to ensure that the tests pass in Jenkinsci jobs and it makes it easier to run the tests locally as well.
Matrix project needs two versions of MATLAB to run successfully. Since there is an error while using Set up MATLAB twice, these tests would be skipped(Updating the main.yml to define MATLAB_ROOT_22b would trigger the tests)
System tests follow the standard integration tests format *IT.java. These tests can be run in isolation using failsafe plugin.
mvn test - Runs only unit tests
mvn verify - Runs both unit and system tests
mvn failsafe:integration-test - Runs only system tests