Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert bool filters to 0 or 1 when querying the search service #3202

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Current (in progress)

- Nothing yet
- Convert bool filters to 0 or 1 when querying the search service [#3202](https://github.com/opendatateam/udata/pull/3202)

## 10.0.2 (2024-11-19)

Expand Down
7 changes: 6 additions & 1 deletion udata/search/query.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,12 @@ def execute_search(self):
if self.sort:
url = url + f"&sort={self.sort}"
for name, value in self._filters.items():
url = url + f"&{name}={value}"
param_value = value
# HACK: use the `.__class__.__name__` to avoid having to import `BoolFilter` here, as importing it at the top would make a import loop.
if self.adapter.filters[name].__class__.__name__ == "BoolFilter":
# The search service uses 1 and 0 for booleans.
param_value = 1 if value == "true" else 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somehow on udata-search-service side, featured is not set to 0 or 1 but 1 or 4 (probably for poorly decided scoring reasons) 🥲

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if value is True and not true when it comes from the udata-front search view: /fr/datasets/

url = url + f"&{name}={param_value}"
r = requests.get(url, timeout=current_app.config["SEARCH_SERVICE_REQUEST_TIMEOUT"])
r.raise_for_status()
result = r.json()
Expand Down
4 changes: 3 additions & 1 deletion udata/tests/apiv2/test_datasets.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from datetime import datetime

from flask import url_for
from flask import current_app, url_for

import udata.core.organization.constants as org_constants
from udata.core.dataset.apiv2 import DEFAULT_PAGE_SIZE
Expand Down Expand Up @@ -53,6 +53,8 @@ def test_search_dataset(self):
_dataset_org = DatasetFactory(organization=org)
dataset_org_public_service = DatasetFactory(organization=org_public_service)

# This is needed or the test will fail if you have a search service configured in your udata.cfg file.
current_app.config["SEARCH_SERVICE_API_URL"] = ""
Comment on lines +56 to +57
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unrelated to this PR, but this test will fail if you have a search service configured, which I needed to fix this PR ;)

response = self.get(
url_for("apiv2.dataset_search", organization_badge=org_constants.PUBLIC_SERVICE)
)
Expand Down
4 changes: 4 additions & 0 deletions udata/tests/dataset/test_dataset_tasks.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import pytest
from flask import current_app

from udata.core.dataset import tasks

Expand All @@ -22,6 +23,9 @@ def test_purge_datasets():
Dataset.objects.create(title="keep me"),
]

# This is needed or the test will fail if you have a search service configured in your udata.cfg file.
current_app.config["SEARCH_SERVICE_API_URL"] = ""

Comment on lines +26 to +28
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unrelated to this PR, but this test will fail if you have a search service configured, which I needed to fix this PR ;)

topic = Topic.objects.create(name="test topic", datasets=datasets)

user = UserFactory()
Expand Down
31 changes: 31 additions & 0 deletions udata/tests/search/test_query.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
from unittest.mock import patch

from flask import current_app

from udata.core.dataset.search import DatasetSearch
from udata.search.query import DEFAULT_PAGE_SIZE, SearchQuery
from udata.tests.api import APITestCase

Expand Down Expand Up @@ -37,3 +42,29 @@ def test_search_query_to_url(self):
search_query = SearchQuery(params=query)
url = search_query.to_url()
assert "organization=534fff81a3a7292c64a77e5c&q=insee&sort=-created&page=1" in url

@patch("requests.get")
# Mock udata.search.result.SearchResult, used in SearchQuery.execute_search.
@patch("udata.search.query.SearchResult")
def test_search_query_for_search_service(self, search_result_req, mock_req):
"""When using the search service, boolean filters need to be converted to 0 or 1."""
current_app.config["SEARCH_SERVICE_API_URL"] = "http://example.com/"
query = {
"featured": "true",
"page": 1,
"page_size": 20,
}
search_query = SearchQuery(query)
search_query.adapter = DatasetSearch
search_query.execute_search()
mock_req.assert_called_with(
"http://example.com/datasets/?q=&page=1&page_size=20&featured=1", timeout=20
)

query["featured"] = "false"
search_query = SearchQuery(query)
search_query.adapter = DatasetSearch
search_query.execute_search()
mock_req.assert_called_with(
"http://example.com/datasets/?q=&page=1&page_size=20&featured=0", timeout=20
)