Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Not able to submit datasource of type query to the plugin. #155

Open
saketbairoliya2 opened this issue Jul 4, 2023 · 3 comments
Open

Not able to submit datasource of type query to the plugin. #155

saketbairoliya2 opened this issue Jul 4, 2023 · 3 comments

Comments

@saketbairoliya2
Copy link
Contributor

saketbairoliya2 commented Jul 4, 2023 •
edited
Loading 

{
	"queryType": "scan",
	"context": {
		"queryId": "05fc0083-e038-4b76-9b7d-615fb5eb8029"
	},
	"intervals": {
		"type": "intervals",
		"intervals": [
			"1970-01-01T00:00:00Z/2030-01-01T00:00:00Z"
		]
	},
	"legacy": true,
	"columns": [
		"timestamp",
		"631b7586-7993-4619-8193-eb5bfa816dc1",
		"aa8986b3-6153-4aa8-b9b4-82bde592a3b6",
		"3e7c6ca2-faba-4072-b269-57a5370bc17f"
	],
	"dataSource": {
		"type": "query",
		"query": {
			"dataSource": {
				"type": "table",
				"name": "multi_tenant_join_pred_label_regression"
			},
			"dimensions": [{
				"type": "default",
				"dimension": "__model_id__",
				"outputType": "STRING"
			}],
			"intervals": {
				"type": "intervals",
				"intervals": [
					"2022-01-12T10:59:51Z/2024-06-29T10:59:51Z"
				]
			},
			"aggregations": [{
					"type": "doubleSum",
					"fieldName": "sum_difference_squares_label_pred",
					"name": "sum"
				},
				{
					"type": "doubleSum",
					"fieldName": "count",
					"name": "count"
				}
			],
			"postAggregations": [{
					"type": "arithmetic",
					"name": "MSE",
					"fn": "/",
					"fields": [{
							"type": "fieldAccess",
							"fieldName": "sum"
						},
						{
							"type": "fieldAccess",
							"fieldName": "count"
						}
					]
				},
				{
					"type": "expression",
					"name": "RMSE",
					"expression": "sqrt(MSE)"
				},
				{
					"type": "expression",
					"name": "631b7586-7993-4619-8193-eb5bfa816dc1",
					"expression": "case_searched((\"__model_id__\" == '631b7586-7993-4619-8193-eb5bfa816dc1'), RMSE, 0)"
				},
				{
					"type": "expression",
					"name": "aa8986b3-6153-4aa8-b9b4-82bde592a3b6",
					"expression": "case_searched((\"__model_id__\" == 'aa8986b3-6153-4aa8-b9b4-82bde592a3b6'), RMSE, 0)"
				},
				{
					"type": "expression",
					"name": "3e7c6ca2-faba-4072-b269-57a5370bc17f",
					"expression": "case_searched((\"__model_id__\" == '3e7c6ca2-faba-4072-b269-57a5370bc17f'), RMSE, 0)"
				}
			],
			"filter": {
				"type": "and",
				"fields": [{
					"type": "and",
					"fields": [{
							"type": "selector",
							"dimension": "__tenant_id__",
							"value": "on-prem"
						},
						{
							"type": "or",
							"fields": [{
									"type": "and",
									"fields": [{
											"type": "selector",
											"dimension": "__model_id__",
											"value": "668fd344-fa26-4c4c-a90b-562cc12fd509"
										},
										{
											"type": "selector",
											"dimension": "__split_id__",
											"value": "782d67c8-2df6-4cbf-a2ff-23b0f3adf71a"
										}
									]
								},
								{
									"type": "and",
									"fields": [{
											"type": "selector",
											"dimension": "__model_id__",
											"value": "aa8986b3-6153-4aa8-b9b4-82bde592a3b6"
										},
										{
											"type": "selector",
											"dimension": "__split_id__",
											"value": "12473d19-64fc-44b6-92b8-3c8620593083"
										}
									]
								},
								{
									"type": "and",
									"fields": [{
											"type": "selector",
											"dimension": "__model_id__",
											"value": "3e7c6ca2-faba-4072-b269-57a5370bc17f"
										},
										{
											"type": "selector",
											"dimension": "__split_id__",
											"value": "5eb9341f-2d1f-416d-9b1a-ba6ac7ed7e9d"
										}
									]
								}
							]
						}
					]
				}]
			},
			"queryType": "groupBy",
			"granularity": "day"
		}
	}
}

This works in druid:
druid

But when I try the same query in the Plugin, I get error:

            "error": "POST http://localhost:8585/druid/v2 giving up after 1 attempt(s): failed to query Druid: {Error:Unknown exception ErrorMessage:Cannot construct instance of `org.apache.druid.query.QueryDataSource`, problem: 'query' must be nonnull\n at [Source: (org.eclipse.jetty.server.HttpInputOverHTTP); line: 1, column: 2101] (through reference chain: org.apache.druid.query.scan.ScanQuery[\"dataSource\"]) ErrorClass:com.fasterxml.jackson.databind.exc.ValueInstantiationException Host:}",

grafana

I see that the UI on Grafana is able to understand the query but backend seems to through the error. What is wrong here? Can someone suggest?
@ApoorvGuptaAi
Copy link
Contributor

Have a minimum repro at grafadruid/go-druid@master...ApoorvGuptaAi:go-druid:master

@saketbairoliya2
Copy link
Contributor Author

I realised that none of the nested queries are working with the plugin.

For example, query like this works:

{
	"queryType": "scan",
	"dataSource": {
		"type": "table",
		"name": "multi_tenant_join_pred_label_regression"
	},
	"intervals": {
		"type": "intervals",
		"intervals": ["-146136543-09-08T08:23:32.096Z/146140482-04-24T15:36:27.903Z"]
	},
	"resultFormat": "compactedList",
	"limit": 1001,
	"columns": ["__model_id__"],
	"legacy": false,
	"context": {
		"sqlOuterLimit": 1001,
		"sqlQueryId": "18276ca1-03e3-4ea5-a20d-6381c0b1c6c8",
		"useNativeQueryExplain": true
	},
	"granularity": {
		"type": "all"
	}
}

Where as a nested query with scan and groupBy is not working. Internal query in the nested query becomes null when parsing:

{
	"queryType": "scan",
	"context": {
		"queryId": "05fc0083-e038-4b76-9b7d-615fb5eb8029"
	},
	"intervals": {
		"type": "intervals",
		"intervals": ["1970-01-01T00:00:00Z/2030-01-01T00:00:00Z"]
	},
	"legacy": true,
	"columns": ["timestamp", "631b7586-7993-4619-8193-eb5bfa816dc1", "aa8986b3-6153-4aa8-b9b4-82bde592a3b6", "3e7c6ca2-faba-4072-b269-57a5370bc17f"],
	"dataSource": {
		"type": "query",
		"query": {
			"dataSource": {
				"type": "table",
				"name": "multi_tenant_join_pred_label_regression"
			},
			"dimensions": [{
				"type": "default",
				"dimension": "__model_id__",
				"outputType": "STRING"
			}],
			"intervals": {
				"type": "intervals",
				"intervals": ["2022-01-12T10:59:51Z/2024-06-29T10:59:51Z"]
			},
			"aggregations": [{
				"type": "doubleSum",
				"fieldName": "sum_difference_squares_label_pred",
				"name": "sum"
			}, {
				"type": "doubleSum",
				"fieldName": "count",
				"name": "count"
			}],
			"postAggregations": [{
				"type": "arithmetic",
				"name": "MSE",
				"fn": "/",
				"fields": [{
					"type": "fieldAccess",
					"fieldName": "sum"
				}, {
					"type": "fieldAccess",
					"fieldName": "count"
				}]
			}, {
				"type": "expression",
				"name": "RMSE",
				"expression": "sqrt(MSE)"
			}, {
				"type": "expression",
				"name": "631b7586-7993-4619-8193-eb5bfa816dc1",
				"expression": "case_searched((\"__model_id__\" == '631b7586-7993-4619-8193-eb5bfa816dc1'), RMSE, 0)"
			}, {
				"type": "expression",
				"name": "aa8986b3-6153-4aa8-b9b4-82bde592a3b6",
				"expression": "case_searched((\"__model_id__\" == 'aa8986b3-6153-4aa8-b9b4-82bde592a3b6'), RMSE, 0)"
			}, {
				"type": "expression",
				"name": "3e7c6ca2-faba-4072-b269-57a5370bc17f",
				"expression": "case_searched((\"__model_id__\" == '3e7c6ca2-faba-4072-b269-57a5370bc17f'), RMSE, 0)"
			}],
			"filter": {
				"type": "and",
				"fields": [{
					"type": "and",
					"fields": [{
						"type": "selector",
						"dimension": "__truera_tenant_id__",
						"value": "on-prem"
					}, {
						"type": "or",
						"fields": [{
							"type": "and",
							"fields": [{
								"type": "selector",
								"dimension": "__model_id__",
								"value": "668fd344-fa26-4c4c-a90b-562cc12fd509"
							}, {
								"type": "selector",
								"dimension": "__truera_split_id__",
								"value": "782d67c8-2df6-4cbf-a2ff-23b0f3adf71a"
							}]
						}, {
							"type": "and",
							"fields": [{
								"type": "selector",
								"dimension": "__model_id__",
								"value": "aa8986b3-6153-4aa8-b9b4-82bde592a3b6"
							}, {
								"type": "selector",
								"dimension": "__truera_split_id__",
								"value": "12473d19-64fc-44b6-92b8-3c8620593083"
							}]
						}, {
							"type": "and",
							"fields": [{
								"type": "selector",
								"dimension": "__model_id__",
								"value": "3e7c6ca2-faba-4072-b269-57a5370bc17f"
							}, {
								"type": "selector",
								"dimension": "__truera_split_id__",
								"value": "5eb9341f-2d1f-416d-9b1a-ba6ac7ed7e9d"
							}]
						}]
					}]
				}]
			},
			"queryType": "groupBy",
			"granularity": "day"
		}
	}
}

Error message:
Screenshot 2023-07-07 at 9 13 31 PM

@saketbairoliya2 saketbairoliya2 mentioned this issue Jul 7, 2023
Open
@saketbairoliya2
Copy link
Contributor Author

I've raised a PR with the fix on go-druid plugin to bypass the serialisation: grafadruid/go-druid#85

Let me know if this sounds good - I can make the changes configurable and further modify the code as suggested. @jbguerraz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@saketbairoliya2 @ApoorvGuptaAi