Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(Predefined-ATOM) Query examples for each dataset / Iterate Over Get Spatial Dataset URL #722

Closed
milanSch opened this issue Mar 9, 2022 · 7 comments
Closed

(Predefined-ATOM) Query examples for each dataset / Iterate Over Get Spatial Dataset URL #722

milanSch opened this issue Mar 9, 2022 · 7 comments
Labels
deployed in reference validator Solution deployed in production non-breaking change The change makes tests less restrictive and it has no impact on resources already tested
Milestone
v2023.2

Comments

@milanSch
Copy link

milanSch commented Mar 9, 2022

Hello,
it seems to be a problem during test validation of 'Download Service - Pre-defined Atom'.

I validate this download service: https://geos.sazp.sk/inspire/atom/DownloadServiceFeed.xml

Test result: https://yzyiqfakm4.execute-api.eu-west-1.amazonaws.com/validator/v2/TestRuns/EID2b1eca9b-1662-434a-ad48-2935e25fa77f.html?lang=en#EID6f60d4f2-017f-1000-abd2-7708af45ceb9

In test result path: Conformance Class: Download Service - Pre-defined Atom / OpenSearch Description / Query examples for each dataset / Iterate over Describe Spatial Dataset Url

there is a 'Test Instruction code':

.
.
.
for( node in describeSpatialDataSetUrlTemplates ) {
	String describeSpatialDataSetUrlTemplate = node.attributes.getNamedItem("template").getNodeValue()
	String mimeType = node.attributes.getNamedItem("type").getNodeValue()

	log.info("Using template URL: "+describeSpatialDataSetUrlTemplate);
	def parameters = UriUtils.getQueryParameters(describeSpatialDataSetUrlTemplate, false)
	final Map<String, String> cleanParameters = initDefaultParameters(parameters)
	final String endpoint = UriUtils.withoutQueryParameters(describeSpatialDataSetUrlTemplate)

	for(int j=0; j<codes.size(); j++) {
		replaceParameter(parameters, cleanParameters, "inspire_dls:spatial_dataset_identifier_code", codes[j] )
		replaceParameter(parameters, cleanParameters, "inspire_dls:spatial_dataset_identifier_namespace", namespaces[j] )

			for(language in languages) {
				replaceParameter(parameters, cleanParameters, "language", language )
				final String url = UriUtils.withQueryParameters(endpoint, cleanParameters)
				if(!invokedUrls.contains(url)) {
					log.info("Using URL: "+url)
					ph.setTransferProperty("describeSpatialDataSetUrl", url)
					ph.setTransferProperty("mimeType", mimeType)
					ph.runTestStep("Describe Spatial Dataset");
					ph.runTestStep("Iterate over Get Spatial Dataset URL");
					invokedUrls.add(url);
				}
			}
	}
}

the line

ph.runTestStep("Iterate over Get Spatial Dataset URL");

calls next TestStep "Iterate over Get Spatial Dataset URL" with 'Test Instruction code':

import de.interactive_instruments.xtf.*;
SOAPUI_I.init(log,context,testRunner);
import de.interactive_instruments.UriUtils;

ProjectHelper ph = new ProjectHelper();
def xmlHolder = ph.getTestStepResult("Describe Spatial Dataset")

def getSpatialDataSetUrlTemplates = xmlHolder.getDomNodes(
	"/*:OpenSearchDescription/*:Url[@rel='results' and starts-with(@template,'http') and "+
	"contains(@template,'crs') and contains(@template,'spatial_dataset_identifier_code') and "+
	"contains(@template,'spatial_dataset_identifier_namespace') and contains(@template,'language')]");
log.info("getSpatialDatasetUrls: " + getSpatialDataSetUrlTemplates);
.
.
.

The line

def xmlHolder = ph.getTestStepResult("Describe Spatial Dataset")

should be probably changed to

def xmlHolder = ph.getTestStepResult("Get OpenSearch Description")

The result of test "Describe Spatial Dataset" doesn't contain any 'Spatial Data Set Operation request URL template' needed in lines

def getSpatialDataSetUrlTemplates = xmlHolder.getDomNodes(
	"/*:OpenSearchDescription/*:Url[@rel='results' and starts-with(@template,'http') and "+
	"contains(@template,'crs') and contains(@template,'spatial_dataset_identifier_code') and "+
	"contains(@template,'spatial_dataset_identifier_namespace') and contains(@template,'language')]");
log.info("getSpatialDatasetUrls: " + getSpatialDataSetUrlTemplates);

but it contains Dataset Atom Feed file.

Log is
(in my Docker inspire validator container v2022.0 19/01/2022 : etf/logs/etf.log
https://github.com/INSPIRE-MIF/helpdesk-validator/releases/
):
2022-03-09 16:31:09.806 [Thread-166] INFO d.i.e.t.s.STestCaseRunner - running step [Iterate over Describe Spatial Dataset URL]
2022-03-09 16:31:09.860 [Thread-166] INFO g.log - describeSpatialDataSetUrls: [
]
2022-03-09 16:31:09.861 [Thread-166] INFO g.log - languages: [en]
2022-03-09 16:31:09.861 [Thread-166] INFO g.log - namespaces: [https://data.gov.sk/set/rpi/dat/00626031/]
2022-03-09 16:31:09.862 [Thread-166] INFO g.log - codes: [https://data.gov.sk/set/rpi/dat/00626031/CF3D3C625D6E2358E0530110000A920EHULI_INSPIRE_CLC2018_DTS_DID]
2022-03-09 16:31:09.862 [Thread-166] INFO g.log - Using template URL: https://geos.sazp.sk/inspire/atom/search.php?spatial_dataset_identifier_code={inspire_dls:spatial_dataset_identifier_code?}&spatial_dataset_identifier_namespace={inspire_dls:spatial_dataset_identifier_namespace?}&crs={inspire_dls:crs?}&language={language?}&q={searchTerms?}
2022-03-09 16:31:09.864 [Thread-166] INFO g.log - Using URL: https://geos.sazp.sk/inspire/atom/search.php?language=en&spatial_dataset_identifier_code=https%3A%2F%2Fdata.gov.sk%2Fset%2Frpi%2Fdat%2F00626031%2FCF3D3C625D6E2358E0530110000A920EHULI_INSPIRE_CLC2018_DTS_DID&spatial_dataset_identifier_namespace=https%3A%2F%2Fdata.gov.sk%2Fset%2Frpi%2Fdat%2F00626031%2F
2022-03-09 16:31:09.864 [Thread-166] INFO etf-sui-owsgtl - Running test step "Describe Spatial Dataset" requested by test step "Iterate over Describe Spatial Dataset URL"
2022-03-09 16:31:09.865 [Thread-166] INFO d.i.e.t.s.STestCaseRunner - running step [Describe Spatial Dataset]
2022-03-09 16:31:09.901 [Thread-166] INFO etf-sui-owsgtl - Running test step "Iterate over Get Spatial Dataset URL" requested by test step "Iterate over Describe Spatial Dataset URL"
2022-03-09 16:31:09.901 [Thread-166] INFO d.i.e.t.s.STestCaseRunner - running step [Iterate over Get Spatial Dataset URL]
2022-03-09 16:31:09.931 [Thread-166] INFO g.log - getSpatialDatasetUrls: []
2022-03-09 16:31:09.931 [Thread-166] INFO g.log - languages: []
2022-03-09 16:31:09.931 [Thread-166] INFO g.log - namespaces: []
2022-03-09 16:31:09.932 [Thread-166] INFO g.log - codes: []
2022-03-09 16:31:09.932 [Thread-166] INFO g.log - CRSs: []
2022-03-09 16:31:09.933 [Thread-166] ERROR d.i.e.t.TestResultCollector - An internal error occurred finishing result 4a9036db-eb9c-458c-b8ad-58b3ff5b82a7
java.lang.ArrayIndexOutOfBoundsException: -1

It is possible to see empty variables there:

2022-03-09 16:31:09.931 [Thread-166] INFO g.log - getSpatialDatasetUrls: []
2022-03-09 16:31:09.931 [Thread-166] INFO g.log - languages: []
2022-03-09 16:31:09.931 [Thread-166] INFO g.log - namespaces: []
2022-03-09 16:31:09.932 [Thread-166] INFO g.log - codes: []
2022-03-09 16:31:09.932 [Thread-166] INFO g.log - CRSs: []

After correction from
def xmlHolder = ph.getTestStepResult("Describe Spatial Dataset")
to
def xmlHolder = ph.getTestStepResult("Get OpenSearch Description")

Log is:
2022-03-09 16:37:12.927 [Thread-243] INFO d.i.e.t.s.STestCaseRunner - running step [Iterate over Describe Spatial Dataset URL]
2022-03-09 16:37:13.132 [Thread-243] INFO g.log - describeSpatialDataSetUrls: [
]
2022-03-09 16:37:13.132 [Thread-243] INFO g.log - languages: [en]
2022-03-09 16:37:13.133 [Thread-243] INFO g.log - namespaces: [https://data.gov.sk/set/rpi/dat/00626031/]
2022-03-09 16:37:13.133 [Thread-243] INFO g.log - codes: [https://data.gov.sk/set/rpi/dat/00626031/CF3D3C625D6E2358E0530110000A920EHULI_INSPIRE_CLC2018_DTS_DID]
2022-03-09 16:37:13.133 [Thread-243] INFO g.log - Using template URL: https://geos.sazp.sk/inspire/atom/search.php?spatial_dataset_identifier_code={inspire_dls:spatial_dataset_identifier_code?}&spatial_dataset_identifier_namespace={inspire_dls:spatial_dataset_identifier_namespace?}&crs={inspire_dls:crs?}&language={language?}&q={searchTerms?}
2022-03-09 16:37:13.135 [Thread-243] INFO g.log - Using URL: https://geos.sazp.sk/inspire/atom/search.php?language=en&spatial_dataset_identifier_code=https%3A%2F%2Fdata.gov.sk%2Fset%2Frpi%2Fdat%2F00626031%2FCF3D3C625D6E2358E0530110000A920EHULI_INSPIRE_CLC2018_DTS_DID&spatial_dataset_identifier_namespace=https%3A%2F%2Fdata.gov.sk%2Fset%2Frpi%2Fdat%2F00626031%2F
2022-03-09 16:37:13.135 [Thread-243] INFO etf-sui-owsgtl - Running test step "Describe Spatial Dataset" requested by test step "Iterate over Describe Spatial Dataset URL"
2022-03-09 16:37:13.135 [Thread-243] INFO d.i.e.t.s.STestCaseRunner - running step [Describe Spatial Dataset]
2022-03-09 16:37:13.281 [Thread-243] INFO etf-sui-owsgtl - Running test step "Iterate over Get Spatial Dataset URL" requested by test step "Iterate over Describe Spatial Dataset URL"
2022-03-09 16:37:13.281 [Thread-243] INFO d.i.e.t.s.STestCaseRunner - running step [Iterate over Get Spatial Dataset URL]
2022-03-09 16:37:13.383 [Thread-243] INFO g.log - getSpatialDatasetUrls: [
]
2022-03-09 16:37:13.383 [Thread-243] INFO g.log - languages: [en]
2022-03-09 16:37:13.384 [Thread-243] INFO g.log - namespaces: [https://data.gov.sk/set/rpi/dat/00626031/]
2022-03-09 16:37:13.392 [Thread-243] INFO g.log - codes: [https://data.gov.sk/set/rpi/dat/00626031/CF3D3C625D6E2358E0530110000A920EHULI_INSPIRE_CLC2018_DTS_DID]
2022-03-09 16:37:13.393 [Thread-243] INFO g.log - CRSs: [http://www.opengis.net/def/crs/EPSG/0/4258]
2022-03-09 16:37:13.393 [Thread-243] INFO g.log - Using template URL: https://geos.sazp.sk/inspire/atom/search.php?spatial_dataset_identifier_code={inspire_dls:spatial_dataset_identifier_code?}&spatial_dataset_identifier_namespace={inspire_dls:spatial_dataset_identifier_namespace?}&crs={inspire_dls:crs?}&language={language?}&q={searchTerms?}
2022-03-09 16:37:13.401 [Thread-243] INFO g.log - Using URL: https://geos.sazp.sk/inspire/atom/search.php?crs=http%3A%2F%2Fwww.opengis.net%2Fdef%2Fcrs%2FEPSG%2F0%2F4258&language=en&spatial_dataset_identifier_code=https%3A%2F%2Fdata.gov.sk%2Fset%2Frpi%2Fdat%2F00626031%2FCF3D3C625D6E2358E0530110000A920EHULI_INSPIRE_CLC2018_DTS_DID&spatial_dataset_identifier_namespace=https%3A%2F%2Fdata.gov.sk%2Fset%2Frpi%2Fdat%2F00626031%2F
2022-03-09 16:37:13.401 [Thread-243] INFO etf-sui-owsgtl - Running test step "Get Spatial Dataset" requested by test step "Iterate over Describe Spatial Dataset URL"
2022-03-09 16:37:13.402 [Thread-243] INFO d.i.e.t.s.STestCaseRunner - running step [Get Spatial Dataset]
2022-03-09 16:37:13.469 [Thread-243] ERROR d.i.e.t.TestResultCollector - An internal error occurred finishing result 3e55838a-6ac8-422c-84f3-a958d9956095
java.lang.ArrayIndexOutOfBoundsException: -1

and variables are assigned:

2022-03-09 16:37:13.383 [Thread-243] INFO g.log - languages: [en]
2022-03-09 16:37:13.384 [Thread-243] INFO g.log - namespaces: [https://data.gov.sk/set/rpi/dat/00626031/]
2022-03-09 16:37:13.392 [Thread-243] INFO g.log - codes: [https://data.gov.sk/set/rpi/dat/00626031/CF3D3C625D6E2358E0530110000A920EHULI_INSPIRE_CLC2018_DTS_DID]
2022-03-09 16:37:13.393 [Thread-243] INFO g.log - CRSs: [http://www.opengis.net/def/crs/EPSG/0/4258]

In both cases (before and after correction of code) test validation ended up with errors:

2022-03-09 16:37:13.469 [Thread-243] ERROR d.i.e.t.TestResultCollector - An internal error occurred finishing result 3e55838a-6ac8-422c-84f3-a958d9956095
java.lang.ArrayIndexOutOfBoundsException: -1
at de.interactive_instruments.etf.testdriver.AbstractTestCollector.setStatusAndParentStatus(AbstractTestCollector.java:210)
at de.interactive_instruments.etf.testdriver.AbstractTestCollector.releaseSubCollector(AbstractTestCollector.java:198)
at de.interactive_instruments.etf.testdriver.AbstractTestStepResultCollector.doEnd(AbstractTestStepResultCollector.java:153)
at de.interactive_instruments.etf.testdriver.AbstractTestCollector.end(AbstractTestCollector.java:119)
at de.interactive_instruments.etf.testdriver.AbstractTestStepResultCollector.doEnd(AbstractTestStepResultCollector.java:164)
at de.interactive_instruments.etf.testdriver.AbstractTestCollector.end(AbstractTestCollector.java:119)
at de.interactive_instruments.etf.testdriver.AbstractTestResultCollector.doEnd(AbstractTestResultCollector.java:233)
at de.interactive_instruments.etf.testdriver.AbstractTestCollector.end(AbstractTestCollector.java:132)
at de.interactive_instruments.etf.sel.mapping.TestRunCollector.afterStep(TestRunCollector.java:462)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.runTestStep(AbstractTestCaseRunner.java:222)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.runTestStepByName(AbstractTestCaseRunner.java:200)
at com.eviware.soapui.model.testsuite.TestCaseRunner$runTestStepByName.call(Unknown Source)
at de.interactive_instruments.xtf.ProjectHelper.runTestStep(ProjectHelper.groovy:354)
at de.interactive_instruments.xtf.ProjectHelper.runTestStep(ProjectHelper.groovy)
at de.interactive_instruments.xtf.ProjectHelper$runTestStep.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at Script1.run(Script1.groovy:80)
at com.eviware.soapui.support.scripting.groovy.SoapUIGroovyScriptEngine.run(SoapUIGroovyScriptEngine.java:92)
at com.eviware.soapui.impl.wsdl.teststeps.WsdlGroovyScriptTestStep.run(WsdlGroovyScriptTestStep.java:141)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.runTestStep(AbstractTestCaseRunner.java:211)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.runTestStepByName(AbstractTestCaseRunner.java:200)
at com.eviware.soapui.model.testsuite.TestCaseRunner$runTestStepByName.call(Unknown Source)
at de.interactive_instruments.xtf.ProjectHelper.runTestStep(ProjectHelper.groovy:354)
at de.interactive_instruments.xtf.ProjectHelper.runTestStep(ProjectHelper.groovy)
at de.interactive_instruments.xtf.ProjectHelper$runTestStep.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at Script1.run(Script1.groovy:74)
at com.eviware.soapui.support.scripting.groovy.SoapUIGroovyScriptEngine.run(SoapUIGroovyScriptEngine.java:92)
at com.eviware.soapui.impl.wsdl.teststeps.WsdlGroovyScriptTestStep.run(WsdlGroovyScriptTestStep.java:141)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.runTestStep(AbstractTestCaseRunner.java:211)
at com.eviware.soapui.impl.wsdl.testcase.WsdlTestCaseRunner.runCurrentTestStep(WsdlTestCaseRunner.java:47)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:138)
at com.eviware.soapui.impl.wsdl.support.AbstractTestCaseRunner.internalRun(AbstractTestCaseRunner.java:46)
at com.eviware.soapui.impl.wsdl.support.AbstractTestRunner.run(AbstractTestRunner.java:129)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2022-03-09 16:37:13.539 [Thread-243] ERROR d.i.e.t.TestResultCollector - An internal error occurred finishing result 3e55838a-6ac8-422c-84f3-a958d9956095
java.lang.IllegalStateException: java.lang.ArrayIndexOutOfBoundsException: -1
at de.interactive_instruments.etf.testdriver.AbstractTestStepResultCollector.doEnd(AbstractTestStepResultCollector.java:171)
.
.
.

This 'java.lang.IllegalStateException: java.lang.ArrayIndexOutOfBoundsException: -1' is probably next different problem (I can't find a reason).

My issue is related to and continues after this one #642 (comment)

Maybe I am completely wrong but definitely thank you for your time.

milan
@dperezBM dperezBM added the ready for testing Solution provided to reporter or developed & deployed in staging (or beta), waiting for testing label Mar 11, 2022
@arantzaetxebarria arantzaetxebarria removed the ready for testing Solution provided to reporter or developed & deployed in staging (or beta), waiting for testing label Mar 11, 2022
@milanSch milanSch mentioned this issue Apr 22, 2022
Closed
@milanSch
Copy link
Author

Dear,
just wanted to ask, is something new with this issue?

thank you
milan

@mwjsanders mwjsanders mentioned this issue May 11, 2022
Closed
@arantzaetxebarria
Copy link
Collaborator

Dear @milanSch

We have implemented some changes in issue 751. Some of them are related to this issue, could you please check these changes in staging environment? Thanks

@arantzaetxebarria arantzaetxebarria added ready for testing Solution provided to reporter or developed & deployed in staging (or beta), waiting for testing and removed under analysis labels May 16, 2022
@milanSch
Copy link
Author

Dear @arantzaetxebarria, thank you.

I tried to validate atom download service:
https://geos.sazp.sk/inspire/atom/DownloadServiceFeed.xml
by staging validator
http://staging-inspire-validator.eu-west-1.elasticbeanstalk.com/etf-webapp/test-selection/index.html
but I am not able get through. There is an error:

<etf:error>
TestRunDao: An object with the id "1a80a752-3f2c-4d15-837e-a7b6bca093cc" could not be found!
</etf:error>

http://staging-inspire-validator.eu-west-1.elasticbeanstalk.com/validator/v2/TestRuns/EID1a80a752-3f2c-4d15-837e-a7b6bca093cc.xml

image

When I used
https://inspire.ec.europa.eu/validator/test-selection/index.html
it worked as usual and I obtained validation report.
image

thank you
milan

@milanSch milanSch mentioned this issue May 18, 2022
Closed
@arantzaetxebarria arantzaetxebarria added under analysis and removed ready for testing Solution provided to reporter or developed & deployed in staging (or beta), waiting for testing labels May 26, 2022
@milanSch
Copy link
Author

Hello,
I put a comment here #642 (comment)

thank you
milan

@milanSch milanSch mentioned this issue Sep 16, 2022
Closed
@arantzaetxebarria
Copy link
Collaborator

Dear @milanSch

We have implemented some changes related to the issue #819, these changes seem to fix this issue, could you please verify? Many thanks

Best regards

@arantzaetxebarria arantzaetxebarria added the ready for testing Solution provided to reporter or developed & deployed in staging (or beta), waiting for testing label Jun 15, 2023
@milanSch
Copy link
Author

Dear @arantzaetxebarria,
thank you for information. Unfortunately my testing data has changed and I cannot repeat testing (verification). So, please close this issue.

Thank you
milan

@fabiovinci
Copy link
Collaborator

Dear @milanSch,

thank you for your feedback.
In any case, by validating the service the initially reported error is no longer present.
We will close the issue, please reopen the issue or open a new one if the error persists.

@fabiovinci fabiovinci added solved Solution developed and accepted, not yet deployed and removed ready for testing Solution provided to reporter or developed & deployed in staging (or beta), waiting for testing labels Jun 27, 2023
@fabiovinci fabiovinci added the deployed in reference validator Solution deployed in production label Jun 27, 2023
@fabiovinci fabiovinci added this to the v2023.2 milestone Jun 27, 2023
@fabiovinci fabiovinci added non-breaking change The change makes tests less restrictive and it has no impact on resources already tested and removed solved Solution developed and accepted, not yet deployed labels Jun 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
deployed in reference validator Solution deployed in production non-breaking change The change makes tests less restrictive and it has no impact on resources already tested
Projects
None yet
Milestone
v2023.2
Development

No branches or pull requests
4 participants
@milanSch @fabiovinci @dperezBM @arantzaetxebarria