Querying Observations returns 404 not found. #1581

mikedolx · 2023-06-07T13:02:16Z

mikedolx
Jun 7, 2023

Hi,

i'm currently analysing our FROST-Server in terms of performance. As it seems, our Observations currenlty do not return values anymore. No matter, what type of query i'm creating (select, orderby, top), the result is always 404 - not found. I am using the REST-API path /Observations. Inserts seem to work fine.

<!doctype html>
<html lang="en">

<head>
	<title>HTTP Status 404 – Not Found</title>
	<style type="text/css">
		body {
			font-family: Tahoma, Arial, sans-serif;
		}

		h1,
		h2,
		h3,
		b {
			color: white;
			background-color: #525D76;
		}

		h1 {
			font-size: 22px;
		}

		h2 {
			font-size: 16px;
		}

		h3 {
			font-size: 14px;
		}

		p {
			font-size: 12px;
		}

		a {
			color: black;
		}

		.line {
			height: 1px;
			background-color: #525D76;
			border: none;
		}
	</style>
</head>

<body>
	<h1>HTTP Status 404 – Not Found</h1>
	<hr class="line" />
	<p><b>Type</b> Status Report</p>
	<p><b>Description</b> The origin server did not find a current representation for the target resource or is not
		willing to disclose that one exists.</p>
	<hr class="line" />
	<h3>Apache Tomcat/9.0.72</h3>
</body>

</html>

Could it be, that we have reached a limit on our database? Is it possible to tweak some settings?

I've checked the row count on our PROD and DEV instance, and their both around 600Million rows. I have seen this Discussion-Thread Discussion: FROST-Server and TimescaleDB and was wondering if we might need to migrate to another DB?

hylkevds · 2023-06-07T18:00:23Z

hylkevds
Jun 7, 2023
Maintainer

That 404 is not generated by FROST. You probably have an incorrect URL that does not go to FROST at all.
/Observations is not a valid STA URL, since it lacks the version number.

0 replies

hylkevds · 2023-06-07T18:16:32Z

hylkevds
Jun 7, 2023
Maintainer

How many Observations your database can handle reliably depends on the speed of the disks in the machine, the amount of memory the database has available and which indices you added.

https://airquality-frost.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1/ has about 670e6 Observations on a very under-powered server, but with SSD disks. Visualisation: https://api4inspire.k8s.ilt-dmz.iosb.fraunhofer.de/servlet/is/127/

https://iosb-airquality.apps.bauhausmobilitylab.de/FROST-Server/v1.1 has about 930e6 Observations, with slightly better hardware.

2 replies

mikedolx Jun 9, 2023
Author

Hi @hylkevds,

thanks for the infos. I did some testing and it turns out, that as soon as i add $orderby=phenomenonTime the FROST-Server get's somehow into trouble. If i try this

$ curl --location 'https://airquality-frost.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1/Observations?%24orderby=phenomenonTime&%24top=100'

i get this response

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    86  100    86    0     0      1      0  0:01:26  0:00:50  0:00:36    19{"code":500,"type":"error","message":"Failed to execute query. See logs for details."}

First i thought, that this is some issue with Postman, but then i tried it with different tools. Hence i provided the curl-example. As soon as i remove the orderby directive, frost replies with some reponses.

hylkevds Jun 9, 2023
Maintainer

"code":500

That's not a 404 not found...

Failed to execute query. See logs for details.

We're happy to offer support contracts to debug your local installation or optimise it for your use case.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Querying Observations returns 404 not found. #1581

{{title}}

Replies: 2 comments 2 replies

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Querying Observations returns 404 not found. #1581

mikedolx Jun 7, 2023

Replies: 2 comments · 2 replies

hylkevds Jun 7, 2023 Maintainer

hylkevds Jun 7, 2023 Maintainer

mikedolx Jun 9, 2023 Author

hylkevds Jun 9, 2023 Maintainer

mikedolx
Jun 7, 2023

Replies: 2 comments 2 replies

hylkevds
Jun 7, 2023
Maintainer

hylkevds
Jun 7, 2023
Maintainer

mikedolx Jun 9, 2023
Author

hylkevds Jun 9, 2023
Maintainer