Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automate TCK documentation #332

Merged
merged 3 commits into from
Oct 4, 2023
Merged

Automate TCK documentation #332

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ed4e7c5
Add assertion annotation
KyleAure Aug 16, 2023
f471526
Introduce challenge annotation
KyleAure Aug 23, 2023
60984e4
Auto generate TCK Distribution sections
KyleAure Aug 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 76 additions & 25 deletions tck-dist/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
/*
* Copyright (c) 2022, 2023 Contributors to the Eclipse Foundation
*
* This program and the accompanying materials are made available under the
* terms of the Eclipse Public License v. 2.0, which is available at
* http://www.eclipse.org/legal/epl-2.0.
*
* This Source Code may also be made available under the following Secondary
* Licenses when the conditions for such availability set forth in the
* Eclipse Public License v. 2.0 are satisfied: GNU General Public License,
* version 2 with the GNU Classpath Exception, which is available at
* https://www.gnu.org/software/classpath/license.html.
*
* SPDX-License-Identifier: EPL-2.0 OR GPL-2.0 WITH Classpath-exception-2.0
*/
~ Copyright (c) 2022, 2023 Contributors to the Eclipse Foundation
~
~ This program and the accompanying materials are made available under the
~ terms of the Eclipse Public License v. 2.0, which is available at
~ http://www.eclipse.org/legal/epl-2.0.
~
~ This Source Code may also be made available under the following Secondary
~ Licenses when the conditions for such availability set forth in the
~ Eclipse Public License v. 2.0 are satisfied: GNU General Public License,
~ version 2 with the GNU Classpath Exception, which is available at
~ https://www.gnu.org/software/classpath/license.html.
~
~ SPDX-License-Identifier: EPL-2.0 OR GPL-2.0 WITH Classpath-exception-2.0
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
Expand All @@ -26,8 +26,6 @@
</parent>

<artifactId>jakarta.enterprise.concurrent-tck-dist</artifactId>
<packaging>pom</packaging>

<name>Jakarta Concurrency TCK Distribution</name>
<description>Jakarta Concurrency TCK Distribution</description>

Expand All @@ -38,7 +36,8 @@
<distribution>repo</distribution>
</license>
<license>
<name>GNU General Public License, version 2 with the GNU Classpath Exception</name>
<name>GNU General Public License, version 2 with the GNU Classpath
Exception</name>
<url>https://projects.eclipse.org/license/secondary-gpl-2.0-cp</url>
<distribution>repo</distribution>
</license>
Expand All @@ -60,14 +59,65 @@
</dependency>
<dependency>
<groupId>jakarta.enterprise.concurrent</groupId>
<artifactId>jakarta.enterprise.concurrent.parent</artifactId>
<artifactId>jakarta.enterprise.concurrent-api</artifactId>
<version>${jakarta.concurrent.tck.version}</version>
<type>pom</type>
</dependency>
</dependencies>

<build>
<plugins>
<!-- Compile class files -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.11.0</version>
<configuration>
<source>17</source>
<target>17</target>
</configuration>
</plugin>

<!-- Load TCK dependency location as variable -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>3.6.0</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>properties</goal>
</goals>
</execution>
</executions>
</plugin>

<!-- Automatically generate a list of excluded tests and test counts -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<id>generate-asciidoc</id>
<phase>process-classes</phase>
<goals>
<goal>java</goal>
</goals>
</execution>
</executions>
<configuration>
<mainClass>ee.jakarta.tck.data.metadata.CollectMetaData</mainClass>
<arguments>
<argument>true</argument>
<argument>
${jakarta.enterprise.concurrent:jakarta.enterprise.concurrent-tck:jar}</argument>
<argument>
${project.basedir}/src/main/asciidoc/generated</argument>
</arguments>
</configuration>
</plugin>

<!-- Aciidoctor will create the html and pdf distributions of the user-guide -->
<plugin>
<groupId>org.asciidoctor</groupId>
Expand All @@ -83,7 +133,7 @@
<executions>
<execution>
<id>asciidoc-to-html</id>
<phase>generate-resources</phase>
<phase>prepare-package</phase>
<goals>
<goal>process-asciidoc</goal>
</goals>
Expand All @@ -96,7 +146,7 @@
</execution>
<execution>
<id>asciidoc-to-pdf</id>
<phase>generate-resources</phase>
<phase>prepare-package</phase>
<goals>
<goal>process-asciidoc</goal>
</goals>
Expand All @@ -117,7 +167,7 @@
<skip>${maven.adoc.skip}</skip>
</configuration>
</plugin>

<!-- Assembly plugin to collect everything into a single distribution -->
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
Expand All @@ -132,7 +182,8 @@
<descriptors>
<descriptor>src/main/assembly/assembly.xml</descriptor>
</descriptors>
<finalName>concurrency-tck-${jakarta.concurrent.tck.version}</finalName>
<finalName>
concurrency-tck-${jakarta.concurrent.tck.version}</finalName>
</configuration>
</execution>
</executions>
Expand Down
148 changes: 18 additions & 130 deletions tck-dist/src/main/asciidoc/concurrency-tck-reference-guide.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Jakarta Concurrency TCK Reference Guide
:author: Kyle Aure
:email: [email protected]
:revnumber: v1.0
:revnumber: v1.1
:toc:
:sectnums:

Expand All @@ -13,7 +13,6 @@
:APIEclipseSite: https://projects.eclipse.org/projects/ee4j.cu
:APIGitSite: https://github.com/jakartaee/concurrency
:TCKTestPlatform: Junit5
:RuntimeTestCount: 164
:SigPluginGAV: org.netbeans.tools:sigtest-maven-plugin:1.6

== Preface
Expand Down Expand Up @@ -182,11 +181,12 @@ Because there is flexibility regarding how a user could use Maven to configure a

=== Runtime tests

For the runtime test (JUnit) component of the TCK:
For the {TCKTestPlatform} runtime tests of the TCK,
the following table shows the number of tests that should pass based on platform:

* {RuntimeTestCount} tests must be passed to successfully execute the EE TCK suite
include::generated/runtime-tests.adoc[]

**Note:* This count includes the signature test
**Note:** Counts include signature test, but do not include disabled tests.

== Example runner

Expand Down Expand Up @@ -343,44 +343,9 @@ $ mvn clean test

=== Expected Output

Here is example output when we, as in the starter runner, run successfully:
Here is example output when the starter runner runs successfully:

[source, sh]
----
$ mvn clean test
...

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
...
...

Tests run: 162, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 410.985 sec - in TestSuite

Results :

Tests run: 162, Failures: 0, Errors: 0, Skipped: 0

16:05:47,343 [INFO]
16:05:47,344 [INFO] ------< io.openliberty.jakarta.enterprise.concurrent:tck.runner >-------
16:05:47,345 [INFO] Building Jakarta Concurrency TCK Runner 1.0-SNAPSHOT [2/2]
16:05:47,346 [INFO] --------------------------------[ pom ]---------------------------------
16:05:47,351 [INFO]
16:05:47,351 [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tck.runner ---
16:05:47,359 [INFO] ------------------------------------------------------------------------
16:05:47,380 [INFO] Reactor Summary for Jakarta Concurrency TCK Runner 1.0-SNAPSHOT:
16:05:47,381 [INFO]
16:05:47,382 [INFO] Jakarta Concurrency TCK Runner TCK Module .......... SUCCESS [06:55 min]
16:05:47,383 [INFO] Jakarta Concurrency TCK Runner ..................... SUCCESS [ 0.017 s]
16:05:47,384 [INFO] ------------------------------------------------------------------------
16:05:47,385 [INFO] BUILD SUCCESS
16:05:47,385 [INFO] ------------------------------------------------------------------------
16:05:47,386 [INFO] Total time: 06:55 min
16:05:47,387 [INFO] Finished at: 2022-03-30T16:05:47-05:00
16:05:47,387 [INFO] ------------------------------------------------------------------------
----
include::generated/expected-output.adoc[]

== Signature Tests

Expand Down Expand Up @@ -433,92 +398,7 @@ As mentioned in the prerequisite section the signature file formats across the v

The the Signature Test plugin will log output to `System.out`. Whereas, the Signature Test framework we use to set up the test will log using `java.util.logging` so you may see these logs output to two separate locations depending on your application server.

[source, txt]
----
[3/30/22, 16:01:03:250 CDT] 00000045 TestServlet I --> testSignatures
[3/30/22, 16:01:12:618 CDT] 00000045 SystemOut O ********** BEGIN PACKAGE LEVEL SIGNATURE VALIDATION **********
[3/30/22, 16:01:12:618 CDT] 00000045 SystemOut O ********** BEGIN VALIDATE PACKAGE 'jakarta.enterprise.concurrent' **********
[3/30/22, 16:01:12:618 CDT] 00000045 SystemOut O ********** VALIDATE IN STATIC MODE - TO CHECK CONSANT VALUES ****
[3/30/22, 16:01:12:619 CDT] 00000045 SystemOut O Static mode supports checks of static constants values
[3/30/22, 16:01:12:622 CDT] 00000045 SystemOut O Setting static mode flag to allow constant checking.
...
[3/30/22, 16:01:13:392 CDT] 00000045 SystemOut O ********** Status Report 'jakarta.enterprise.concurrent' **********
[3/30/22, 16:01:13:392 CDT] 00000045 SystemOut O SignatureTest report
Base version: 3.0.0-SNAPSHOT
Tested version:
Check mode: src [throws normalized]
Constant checking: on

Warning: The return type java.util.concurrent.Future<{%%0}> can't be resolved
Warning: The return type java.util.concurrent.Future<{%%0}> can't be resolved
Warning: The return type java.util.concurrent.Future<?> can't be resolved

[3/30/22, 16:01:13:392 CDT] 00000045 SystemOut O ********** Package 'jakarta.enterprise.concurrent' - PASSED (STATIC MODE) **********
[3/30/22, 16:01:13:393 CDT] 00000045 SystemOut O ********** VALIDATE IN REFLECTIVE MODE ****
[3/30/22, 16:01:13:393 CDT] 00000045 SystemOut O Reflective mode supports verification within containers (ie ejb, servlet, etc)
[3/30/22, 16:01:13:394 CDT] 00000045 SystemOut O Not Setting static mode flag to allow constant checking.
[3/30/22, 16:01:13:404 CDT] 00000045 SystemOut O Calling: com.sun.tdk.signaturetest.SignatureTest() with following args:
...
[3/30/22, 16:01:13:892 CDT] 00000045 SystemOut O ********** Status Report 'jakarta.enterprise.concurrent' **********
[3/30/22, 16:01:13:892 CDT] 00000045 SystemOut O SignatureTest report
Base version: 3.0.0-SNAPSHOT
Tested version:
Check mode: src [throws normalized]
Constant checking: on

Warning: The return type java.util.concurrent.Future<{%%0}> can't be resolved
Warning: The return type java.util.concurrent.Future<{%%0}> can't be resolved
Warning: The return type java.util.concurrent.Future<?> can't be resolved

[3/30/22, 16:01:13:893 CDT] 00000045 SystemOut O ********** Package 'jakarta.enterprise.concurrent' - PASSED (REFLECTION MODE) **********
[3/30/22, 16:01:13:893 CDT] 00000045 SystemOut O ********** END VALIDATE PACKAGE 'jakarta.enterprise.concurrent' **********
[3/30/22, 16:01:13:893 CDT] 00000045 SystemOut O ********** BEGIN VALIDATE PACKAGE 'jakarta.enterprise.concurrent.spi' **********
[3/30/22, 16:01:13:893 CDT] 00000045 SystemOut O ********** VALIDATE IN STATIC MODE - TO CHECK CONSANT VALUES ****
[3/30/22, 16:01:13:893 CDT] 00000045 SystemOut O Static mode supports checks of static constants values
[3/30/22, 16:01:13:894 CDT] 00000045 SystemOut O Setting static mode flag to allow constant checking.
[3/30/22, 16:01:13:909 CDT] 00000045 SystemOut O Calling: com.sun.tdk.signaturetest.SignatureTest() with following args:
...
[3/30/22, 16:01:14:292 CDT] 00000045 SystemOut O ********** Status Report 'jakarta.enterprise.concurrent.spi' **********
[3/30/22, 16:01:14:293 CDT] 00000045 SystemOut O SignatureTest report
Base version: 3.0.0-SNAPSHOT
Tested version:
Check mode: src [throws normalized]
Constant checking: on

Warning: The return type java.util.concurrent.Future<{%%0}> can't be resolved
Warning: The return type java.util.concurrent.Future<{%%0}> can't be resolved
Warning: The return type java.util.concurrent.Future<?> can't be resolved

[3/30/22, 16:01:14:293 CDT] 00000045 SystemOut O ********** Package 'jakarta.enterprise.concurrent.spi' - PASSED (STATIC MODE) **********
[3/30/22, 16:01:14:293 CDT] 00000045 SystemOut O ********** VALIDATE IN REFLECTIVE MODE ****
[3/30/22, 16:01:14:293 CDT] 00000045 SystemOut O Reflective mode supports verification within containers (ie ejb, servlet, etc)
[3/30/22, 16:01:14:294 CDT] 00000045 SystemOut O Not Setting static mode flag to allow constant checking.
[3/30/22, 16:01:14:301 CDT] 00000045 SystemOut O Calling: com.sun.tdk.signaturetest.SignatureTest() with following args:
...
[3/30/22, 16:01:14:616 CDT] 00000045 SystemOut O ********** Status Report 'jakarta.enterprise.concurrent.spi' **********
[3/30/22, 16:01:14:616 CDT] 00000045 SystemOut O SignatureTest report
Base version: 3.0.0-SNAPSHOT
Tested version:
Check mode: src [throws normalized]
Constant checking: on

Warning: The return type java.util.concurrent.Future<{%%0}> can't be resolved
Warning: The return type java.util.concurrent.Future<{%%0}> can't be resolved
Warning: The return type java.util.concurrent.Future<?> can't be resolved

[3/30/22, 16:01:14:617 CDT] 00000045 SystemOut O ********** Package 'jakarta.enterprise.concurrent.spi' - PASSED (REFLECTION MODE) **********
[3/30/22, 16:01:14:617 CDT] 00000045 SystemOut O ********** END VALIDATE PACKAGE 'jakarta.enterprise.concurrent.spi' **********
[3/30/22, 16:01:14:618 CDT] 00000045 ConcurrencySigTest I
******************************************************
All package signatures passed.
Passed packages listed below:
jakarta.enterprise.concurrent(static mode)
jakarta.enterprise.concurrent(reflection mode)
jakarta.enterprise.concurrent.spi(static mode)
jakarta.enterprise.concurrent.spi(reflection mode)
******************************************************
[3/30/22, 16:01:14:620 CDT] 00000045 TestServlet I <-- testSignatures
----
include::generated/expected-sig-output.adoc[]

== TCK Challenges/Appeals Process

Expand All @@ -536,7 +416,14 @@ The challenge process is defined within the [underline]#Challenges# section with

Challenges will be tracked via the {APIGitSite}/issues[issues] of the {APILongName} Specification repository.

As a shortcut through the challenge process mentioned in the *Jakarta EE TCK Process 1.1* you can click {APIGitSite}/issues/new?labels=challenge[here], though it is recommended that you read through the challenge process to understand it in detail.
The challenge process mentioned in the *Jakarta EE TCK Process 1.1* can be started by clicking {APIGitSite}/issues/new?assignees=&labels=challenge&projects=&template=tck-challenge.yml&title=TCK+Challenge[here],
it is recommended that you read through the challenge process to understand it in detail.

=== Successful Challenges

The following tests are exempt from TCK testing due to challenges:

include::generated/successful-challenges.adoc[]

== Certification of Compatibility

Expand All @@ -553,7 +440,8 @@ The certification of compatibility process is defined within the [underline]#Cer

Certifications will be tracked via the {APIGitSite}/issues[issues] of the {APILongName} Specification repository.

As a shortcut through the certification of compatibility process mentioned in the *Jakarta EE TCK Process 1.1* you can click {APIGitSite}/issues/new?labels=certification[here], though it is recommended that you read through the certification process to understand it in detail.
The certification of compatibility process mentioned in the *Jakarta EE TCK Process 1.1* can be started by clicking {APIGitSite}/issues/new?assignees=&labels=certification&projects=&template=certification.yml&title=Certification[here],
it is recommended that you read through the certification process to understand it in detail.

== Rules for {APILongName} Products

Expand Down
5 changes: 5 additions & 0 deletions tck-dist/src/main/asciidoc/generated/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
runtime-tests.adoc
successful-challenges.adoc
expected-sig-output.adoc
expected-output.adoc
test-properties.adoc
Loading