Define approaches for dealing with external service startup

[starksm64]

Startup of external services like databases and mail servers was done using ant scripts in EE10. We need to document how this should be done in EE11.

[scottmarlow]

We could add bash scripts for starting local database/mail server prior to starting the TCK tests that need it. We could have separate scripts for stopping local database/mail server after running the TCK tests. For this to be robust enough, we could add a signal handler via trap in a higher level script that starts/stops the external services.

We could look at integration with https://github.com/testcontainers as well or something similar. I think that the testcontainers approach is to monitor the client process to know when to stop the database server.

[starksm64]

Junit5 supports a notion of suites that can be defined on a class via annotations:
JUnit Platform Suite Engine

There are currently no available BeforeSuite/AfterSuite lifecycle methods or annotations in a public release (it is an alpha dev feature). This can be mimicked using a Junit5 extension. The ejb32 module has an example suite class for the com.sun.ts.tests.ejb32.lite.timer.basic.xa package:

package com.sun.ts.tests.ejb32.lite.timer.basic.xa;

import org.junit.platform.suite.api.SelectClasses;
import org.junit.platform.suite.api.Suite;


/**
 * A Junit5 suite that runs all the tests in the com.sun.ts.tests.ejb32.lite.timer.basic.xa package. This should
 * be extended by a vendor to include the startup of the database backing the ejblite-pu/jta-data-source jdbc/DBTimer
 * datasource.
 */
@Suite
@SelectClasses({
        ClientEjblitejsfTest.class,
        ClientEjblitejspTest.class,
        ClientEjbliteservlet2Test.class,
        ClientEjbliteservletTest.class,
        JsfClientEjblitejsfTest.class,
        JsfClientEjblitejspTest.class,
        JsfClientEjbliteservlet2Test.class,
        JsfClientEjbliteservletTest.class
})
public class XATimerSuite {
}

This can be run from surefire by passing in the suite class name as you would a test class name.

This package needs a database to back the ejblite-pu referenced by the persistence.xml descriptor used by the tests. One way to start/stop a database for the tests is to create an extension of this suite class that si also a Junit5 extension:

package ejb32.lite.timer.basic.xa;

import com.sun.ts.tests.ejb32.lite.timer.basic.xa.XATimerSuite;
import ee.tck.javadb.JavaDBController;
import org.junit.jupiter.api.extension.BeforeAllCallback;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.platform.suite.api.Suite;

import java.nio.file.Path;
import java.nio.file.Paths;

/**
 * Extends the XATimerSuite to start and stop the JavaDB server before and after the suite.
 */
@Suite
public class JavaDBSetupExtension extends XATimerSuite implements BeforeAllCallback, ExtensionContext.Store.CloseableResource {
    @Override
    public void beforeAll(ExtensionContext extensionContext) throws Exception {
        System.out.printf("beforeAll suite");
        Path tsHome = Paths.get(System.getProperty("ts.home"));
        Path derbyHome = tsHome.resolve("../glassfish7/javadb");
        JavaDBController controller = new JavaDBController();
        System.out.println("JavaDBSetupExtension.starting JavaDB with derbyHome="+derbyHome);
        extensionContext.publishReportEntry("JavaDBSetupExtension.beforeAll", "JavaDBSetupExtension.starting JavaDB with derbyHome="+derbyHome);
        controller.startJavaDB(derbyHome);
        System.out.println("JavaDBSetupExtension.started JavaDB");
        extensionContext.publishReportEntry("JavaDBSetupExtension.beforeAll", "Starting JavaDB");
    }

    @Override
    public void close() throws Throwable {
        System.out.printf("Closing suite");
        JavaDBController controller = new JavaDBController();
        controller.stopJavaDB();
        System.out.println("StartStopTest.stopped JavaDB");
    }
}

This is from the https://github.com/jakartaredhat/wildfly-ee11-tck-runner.git repo. In the BeforeAllCallback#beforeAll method the javadb is started using a new tck libutil ee.tck.javadb.JavaDBController class. It is stopped when the ExtensionContext.Store.CloseableResource#close method is called.

[starksm64]

Another thing we may want to do is to define a new custom Junit5 TestEngine that leverages Junit5/Arquillian, but add supports for exclusions, and service startup. A couple of blogs on the topic:

https://blogs.oracle.com/javamagazine/post/junit-build-custom-test-engines-java
https://blogs.oracle.com/javamagazine/post/junit-test-custom-test-engines-java

[starksm64]

Another way a exclusion list could be implemented is to provide a https://junit.org/junit5/docs/current/api/org.junit.jupiter.api/org/junit/jupiter/api/extension/ExecutionCondition.html that references the current exclusion list.