How to import XML wih paramterized tests to XRay Test Execution #58

lhk135 · 2024-10-07T10:07:28Z

lhk135
Oct 7, 2024

Hi
I have several parametrizied tests in my test class, after test execution there are several (3) with the same JIRA_KEY, but after import it in my XRay Test Execution there are only one iteration of this test. What conditions shoould be applied to see all test iterations in my test execution?
Thanks

bitcoder · 2024-10-07T11:15:11Z

bitcoder
Oct 7, 2024
Maintainer

I don't think that's supported right now.
Can you please share the Junit XML report and a screenshot of how the Test Run looks in Xray?

1 reply

lhk135 Oct 7, 2024
Author

this is part of xml:

<testcase name="verifyNoAccessForDeniedRoles" classname="users.UsersTests" time="0" started-at="2024-09-23T17:43:56.6815248" finished-at="2024-09-23T17:43:56.7009279">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:users.UsersTests]/[test-template:verifyNoAccessForDeniedRoles(ee.lorexgrupp.login.factory.common.AccessTokenFactory)]/[test-template-invocation:#1]
display-name: [1] ee.lorexgrupp.login.factory.trader.TeddTraderAccessTokenFactory@661d49d1
]]></system-out>
<properties>
<property name="test_key" value="PROJ-4628"/>
<property name="tags" value="PROJ-4628"/>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="verifyNoAccessForDeniedRoles" classname="users.UsersTests" time="0" started-at="2024-09-23T17:43:56.7014617" finished-at="2024-09-23T17:43:56.7860937">
<failure message="1 expectation failed.
Expected status code &lt;403&gt; but was &lt;200&gt;.
" type="java.lang.AssertionError"><![CDATA[java.lang.AssertionError: 1 expectation failed.
Expected status code <403> but was <200>.

	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
	at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:73)
	at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:108)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:57)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:263)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:277)
	at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:512)
	at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure$validate$1.call(Unknown Source)
	at io.restassured.internal.ResponseSpecificationImpl.validateResponseIfRequired(ResponseSpecificationImpl.groovy:696)
	at io.restassured.internal.ResponseSpecificationImpl.this$2$validateResponseIfRequired(ResponseSpecificationImpl.groovy)
	at jdk.internal.reflect.GeneratedMethodAccessor302.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:198)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:62)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:185)
	at io.restassured.internal.ResponseSpecificationImpl.statusCode(ResponseSpecificationImpl.groovy:135)
	at io.restassured.specification.ResponseSpecification$statusCode$0.callCurrent(Unknown Source)
	at io.restassured.internal.ResponseSpecificationImpl.statusCode(ResponseSpecificationImpl.groovy:143)
	at io.restassured.internal.ValidatableResponseOptionsImpl.statusCode(ValidatableResponseOptionsImpl.java:89)
	at users.UsersTests.verifyNoAccessForDeniedRoles(UsersTests.java:65)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
]]></failure>
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:users.UsersTests]/[test-template:verifyNoAccessForDeniedRoles(ee.lorexgrupp.login.factory.common.AccessTokenFactory)]/[test-template-invocation:#2]
display-name: [2] ee.lorexgrupp.login.factory.admin.AdamAdminAccessTokenFactory@5a88ecb9
]]></system-out>
<properties>
<property name="test_key" value="PROJ-4628"/>
<property name="tags" value="PROJ-4628"/>
<property name="_dummy_" value=""/>
</properties>
</testcase>
<testcase name="verifyNoAccessForDeniedRoles" classname="users.UsersTests" time="0" started-at="2024-09-23T17:43:56.7870991" finished-at="2024-09-23T17:43:56.8113695">
<system-out><![CDATA[
unique-id: [engine:junit-jupiter]/[class:users.UsersTests]/[test-template:verifyNoAccessForDeniedRoles(ee.lorexgrupp.login.factory.common.AccessTokenFactory)]/[test-template-invocation:#3]
display-name: [3] ee.lorexgrupp.login.factory.backoffice.OperatorTestovAccessTokenFactory@1046aaa4
]]></system-out>
<properties>
<property name="test_key" value="PROJ-4628"/>
<property name="tags" value="PROJ-4628"/>
<property name="_dummy_" value=""/>
</properties>
</testcase>

that is how valid execution of parametrizied test looks like:

that is how it looks like after xml is imported:

bitcoder · 2024-10-07T14:11:56Z

bitcoder
Oct 7, 2024
Maintainer

In your screenshot I see 2 iterations.. but you mentioned before "(...) but after import it in my XRay Test Execution there are only one iteration of this test.". I'm a bit confused; or is this screenshot from what you would like to have?

2 replies

lhk135 Oct 7, 2024
Author

Sorry for confusing, 1st screenshot is just a random expamle of how it should look like, screenshot I mentioned it the 2nd one

bitcoder Oct 7, 2024
Maintainer

No worries, gotcha.
So, you have a parameterized manual test in Xray and you want to report the results for it. Currently it's not possible with this, as there is no way to identify each iteration on the Junit XML report.
You need to generate a Xray JSON report instead, which allows that. However, currently this project doesn't yet support it.
For Java users, the sibling https://github.com/Xray-App/xray-testng-extensions project has support for it, but for the TestNG framework.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to import XML wih paramterized tests to XRay Test Execution #58

{{title}}

Replies: 2 comments 3 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

How to import XML wih paramterized tests to XRay Test Execution #58

lhk135 Oct 7, 2024

Replies: 2 comments · 3 replies

bitcoder Oct 7, 2024 Maintainer

lhk135 Oct 7, 2024 Author

bitcoder Oct 7, 2024 Maintainer

lhk135 Oct 7, 2024 Author

bitcoder Oct 7, 2024 Maintainer

lhk135
Oct 7, 2024

Replies: 2 comments 3 replies

bitcoder
Oct 7, 2024
Maintainer

lhk135 Oct 7, 2024
Author

bitcoder
Oct 7, 2024
Maintainer

lhk135 Oct 7, 2024
Author

bitcoder Oct 7, 2024
Maintainer