Skip to content

Commit

Permalink
[16.0][MIG] l10n_ue_nace PR upgrades
Browse files Browse the repository at this point in the history
  • Loading branch information
edlopen committed Apr 19, 2024
1 parent 7c8708d commit 83c94c1
Show file tree
Hide file tree
Showing 11 changed files with 88 additions and 61 deletions.
7 changes: 4 additions & 3 deletions l10n_eu_nace/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ European NACE partner categories
!! This file is generated by oca-gen-addon-readme !!
!! changes will be overwritten. !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! source digest: sha256:b852bc1153e9c44a78d4f09753769e4475293f75e4bbecb50791457e72342531
!! source digest: sha256:3c3e1ceec16a57cae1db17cb7e2c6cbcd3a1a286f71331b4c5240df789242607
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
Expand Down Expand Up @@ -60,9 +60,10 @@ uses SPARQL to retrieve the data.

If you want to import or update the data, you can do so by running the Import
NACE Wizard. This will search for all active languages in Odoo and import the
data for each one.
data for each one. If you activate a new language, remember to run this wizard
again.

1. Go to "Contacts" > "Configuration" > "Import NACE Wizard"
1. Go to "Contacts" > "Configuration" > "Import NACE industries"
2. Click on the "Import NACE" button.

This might take a minute or so depending on the number of languages and the
Expand Down
1 change: 1 addition & 0 deletions l10n_eu_nace/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
from .post_install import import_nace_industries
from . import wizard
2 changes: 1 addition & 1 deletion l10n_eu_nace/__manifest__.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
],
"maintainers": ["rafaelbn", "yajo", "edlopen"],
"depends": ["partner_industry_secondary"],
"external_dependencies": {"python": ["requests"]},
"installable": True,
"license": "AGPL-3",
"post_init_hook": "import_nace_industries",
}
10 changes: 10 additions & 0 deletions l10n_eu_nace/post_install.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Copyright 2024 Moduon
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from odoo import SUPERUSER_ID, api


# Imports the NACE industries from the ShowVoc endpoint
def import_nace_industries(cr, registry):
env = api.Environment(cr, SUPERUSER_ID, {})
wizard = env["res.partner.industry.eu.nace.wizard"].create({})
return wizard.action_partner_industry_eu_nace()
5 changes: 3 additions & 2 deletions l10n_eu_nace/readme/USAGE.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,10 @@ uses SPARQL to retrieve the data.

If you want to import or update the data, you can do so by running the Import
NACE Wizard. This will search for all active languages in Odoo and import the
data for each one.
data for each one. If you activate a new language, remember to run this wizard
again.

1. Go to "Contacts" > "Configuration" > "Import NACE Wizard"
1. Go to "Contacts" > "Configuration" > "Import NACE industries"
2. Click on the "Import NACE" button.

This might take a minute or so depending on the number of languages and the
Expand Down
7 changes: 4 additions & 3 deletions l10n_eu_nace/static/description/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -367,7 +367,7 @@ <h1 class="title">European NACE partner categories</h1>
!! This file is generated by oca-gen-addon-readme !!
!! changes will be overwritten. !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! source digest: sha256:b852bc1153e9c44a78d4f09753769e4475293f75e4bbecb50791457e72342531
!! source digest: sha256:3c3e1ceec16a57cae1db17cb7e2c6cbcd3a1a286f71331b4c5240df789242607
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -->
<p><a class="reference external image-reference" href="https://odoo-community.org/page/development-status"><img alt="Beta" src="https://img.shields.io/badge/maturity-Beta-yellow.png" /></a> <a class="reference external image-reference" href="http://www.gnu.org/licenses/agpl-3.0-standalone.html"><img alt="License: AGPL-3" src="https://img.shields.io/badge/licence-AGPL--3-blue.png" /></a> <a class="reference external image-reference" href="https://github.com/OCA/community-data-files/tree/16.0/l10n_eu_nace"><img alt="OCA/community-data-files" src="https://img.shields.io/badge/github-OCA%2Fcommunity--data--files-lightgray.png?logo=github" /></a> <a class="reference external image-reference" href="https://translation.odoo-community.org/projects/community-data-files-16-0/community-data-files-16-0-l10n_eu_nace"><img alt="Translate me on Weblate" src="https://img.shields.io/badge/weblate-Translate%20me-F47D42.png" /></a> <a class="reference external image-reference" href="https://runboat.odoo-community.org/builds?repo=OCA/community-data-files&amp;target_branch=16.0"><img alt="Try me on Runboat" src="https://img.shields.io/badge/runboat-Try%20me-875A7B.png" /></a></p>
<p>This module imports the NACE rev. 2 classification
Expand Down Expand Up @@ -406,9 +406,10 @@ <h1><a class="toc-backref" href="#toc-entry-2">Obtaining updated data</a></h1>
uses SPARQL to retrieve the data.</p>
<p>If you want to import or update the data, you can do so by running the Import
NACE Wizard. This will search for all active languages in Odoo and import the
data for each one.</p>
data for each one. If you activate a new language, remember to run this wizard
again.</p>
<ol class="arabic simple">
<li>Go to “Contacts” &gt; “Configuration” &gt; “Import NACE Wizard</li>
<li>Go to “Contacts” &gt; “Configuration” &gt; “Import NACE industries</li>
<li>Click on the “Import NACE” button.</li>
</ol>
<p>This might take a minute or so depending on the number of languages and the
Expand Down
24 changes: 12 additions & 12 deletions l10n_eu_nace/tests/test_en_nace_request_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,22 +67,22 @@
{
"ES": {
"type": "literal",
"value": "01 Agricultura, ganadería, caza y servicios relacionados\
"value": "Agricultura, ganadería, caza y servicios relacionados\
con las mismas",
}
},
{"ES": {"type": "literal", "value": "01.1 Cultivos no perennes"}},
{"ES": {"type": "literal", "value": "Cultivos no perennes"}},
{
"ES": {
"type": "literal",
"value": "01.11 Cultivo de cereales (excepto arroz), \
"value": "Cultivo de cereales (excepto arroz), \
leguminosas y semillas oleaginosas",
}
},
{
"ES": {
"type": "literal",
"value": "02 Silvicultura y explotación \
"value": "Silvicultura y explotación \
forestal",
}
},
Expand All @@ -92,41 +92,41 @@
{
"EN": {
"type": "literal",
"value": "01 Crop and animal production, hunting and related \
"value": "Crop and animal production, hunting and related \
service activities",
}
},
{"EN": {"type": "literal", "value": "01.1 Growing of non-perennial crops"}},
{"EN": {"type": "literal", "value": "Growing of non-perennial crops"}},
{
"EN": {
"type": "literal",
"value": "01.11 Growing of cereals (except rice), leguminous crops \
"value": "Growing of cereals (except rice), leguminous crops \
and oil seeds",
}
},
{"EN": {"type": "literal", "value": "02 Forestry and logging"}},
{"EN": {"type": "literal", "value": "Forestry and logging"}},
]

NACE_FR = [
{
"FR": {
"type": "literal",
"value": "01 Culture et production animale, chasse et services \
"value": "Culture et production animale, chasse et services \
annexes",
}
},
{"FR": {"type": "literal", "value": "01.1 Cultures non permanentes"}},
{"FR": {"type": "literal", "value": "Cultures non permanentes"}},
{
"FR": {
"type": "literal",
"value": "01.11 Culture de céréales (à l'exception du riz), de \
"value": "Culture de céréales (à l'exception du riz), de \
légumineuses et de graines oléagineuses",
}
},
{
"FR": {
"type": "literal",
"value": "02 Sylviculture et exploitation \
"value": "Sylviculture et exploitation \
forestière",
}
},
Expand Down
19 changes: 6 additions & 13 deletions l10n_eu_nace/tests/test_res_partner_industry_eu_nace_wizard.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# License LGPL-3.0 or later (https://www.gnu.org/licenses/lgpl).
from unittest.mock import patch

from odoo.tests.common import Form, TransactionCase, new_test_user, users
from odoo.tests.common import Form, TransactionCase, new_test_user

from .test_en_nace_request_results import (
NACE_COMMON,
Expand All @@ -21,7 +21,7 @@ class TestResPartnerIndustryEUNaceWizard(TransactionCase):
@classmethod
def setUpClass(cls):
super().setUpClass()
cls.user = new_test_user(
cls.uid = new_test_user(
cls.env,
login="test-manager",
groups="base.group_system",
Expand All @@ -36,12 +36,11 @@ def activate_langs(self, lang_list):

def wizard_eu_nace(self):
wizard = Form(
self.env["res.partner.industry.eu.nace.wizard"].with_user(self.user)
self.env["res.partner.industry.eu.nace.wizard"].with_user(self.uid)
)
import_wizard = wizard.save()
return import_wizard

@users("test-manager")
def test_get_languages(self):
nace_wizard = self.wizard_eu_nace()
self.assertEqual(nace_wizard.get_languages(), [("en_US", "EN")])
Expand All @@ -68,7 +67,6 @@ def test_get_languages(self):
expected = [("en_US", "EN"), ("es_ES", "ES"), ("my_MM", "EN")]
self.assertCountEqual(nace_wizard.get_languages(), expected)

@users("test-manager")
def test_create_query(self):
nace_wizard = self.wizard_eu_nace()
active_languages = ["base.lang_en", "base.lang_es", "base.lang_fr"]
Expand All @@ -79,25 +77,23 @@ def test_create_query(self):
self.assertIn("?ES", query)
self.assertIn("?FR", query)
self.assertIn("?FR", query)
self.assertIn("skos:prefLabel ?LabelES;", query)
self.assertIn("skos:prefLabel ?LabelEN;", query)
self.assertIn("skos:prefLabel ?LabelFR;", query)
self.assertIn("skos:altLabel ?LabelES;", query)
self.assertIn("skos:altLabel ?LabelEN;", query)
self.assertIn("skos:altLabel ?LabelFR;", query)
self.assertIn('FILTER (LANG(?LabelES) = "es")', query)
self.assertIn("BIND (STR(?LabelES) as ?ES)", query)
self.assertIn('FILTER (LANG(?LabelEN) = "en")', query)
self.assertIn("BIND (STR(?LabelEN) as ?EN)", query)
self.assertIn('FILTER (LANG(?LabelFR) = "fr")', query)
self.assertIn("BIND (STR(?LabelFR) as ?FR)", query)

@users("test-manager")
@patch(MOCK_PATH, return_value=create_response(NACE_COMMON, NACE_EN))
def test_update_partner_industry_eu_nace_english_only(self, mock_request):
self.activate_langs(["base.lang_en"])
nace_wizard = self.wizard_eu_nace()
records_created = nace_wizard.update_partner_industry_eu_nace()
self.assertEqual(len(records_created), 4)

@users("test-manager")
@patch(
MOCK_PATH, return_value=create_response(NACE_COMMON, NACE_EN, NACE_ES, NACE_FR)
)
Expand All @@ -110,7 +106,6 @@ def test_update_partner_industry_eu_nace_multiple_languages_with_english(
records_created = nace_wizard.update_partner_industry_eu_nace()
self.assertEqual(len(records_created), 4)

@users("test-manager")
@patch(
MOCK_PATH, return_value=create_response(NACE_COMMON, NACE_EN, NACE_ES, NACE_FR)
)
Expand All @@ -120,7 +115,6 @@ def test_update_partner_industry_eu_nace_no_english(self, mock_request):
records_created = nace_wizard.update_partner_industry_eu_nace()
self.assertEqual(len(records_created), 4)

@users("test-manager")
@patch(MOCK_PATH, return_value=create_response(NACE_COMMON, NACE_EN, NACE_ES))
def test_update_partner_industry_eu_nace_new_language_update(self, mock_request):
self.activate_langs(["base.lang_en", "base.lang_es"])
Expand All @@ -139,7 +133,6 @@ def test_update_partner_industry_eu_nace_new_language_update(self, mock_request)
nace_fr_names = list(map(lambda item: item.get("FR").get("value"), NACE_FR))
self.assertCountEqual(records_created_name_fr, nace_fr_names)

@users("test-manager")
@patch(MOCK_PATH, return_value=create_response(NACE_COMMON, NACE_EN, NACE_ES))
def test_update_partner_industry_eu_nace_idempotent_update(self, mock_request):
self.activate_langs(["base.lang_en"])
Expand Down
47 changes: 31 additions & 16 deletions l10n_eu_nace/wizard/res_partner_industry_eu_nace_wizard.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

import requests

from odoo import models
from odoo import _, models

ENDPOINT = "https://publications.europa.eu/webapi/rdf/sparql"
# List of languages imported in previous versions of the module.
Expand Down Expand Up @@ -75,7 +75,7 @@ def get_select_query(language):

@staticmethod
def get_skos_query(language):
return f"skos:prefLabel ?Label{language};"
return f"skos:altLabel ?Label{language};"

@staticmethod
def get_filter_query(language):
Expand Down Expand Up @@ -133,7 +133,8 @@ def _create_query(self, language_list):
BIND (STRAFTER(?NACELEVEL, "/nace2/") AS ?LEVEL)
?Member skos:prefLabel ?MemberLabel .
FILTER (LANG(?MemberLabel) = "en")
}} ORDER BY ?code
BIND (xsd:integer(REPLACE(?code, "\\D", "")) AS ?code_formatted)
}} ORDER BY ?code_formatted
"""
return query

Expand All @@ -143,31 +144,33 @@ def _create_nace_industry(self, nace_data, languages):
nace_ids = self.env["res.partner.industry"]
nace_json = nace_data.json()
bindings = nace_json.get("results", {}).get("bindings", {})
all_naces = nace_ids.search_read([], ["full_name"])
nace_map = {
nace.get("full_name").split(" - ")[0]: nace.get("id") for nace in all_naces
}
for binding in bindings:
nace_code = binding.get("code", {}).get("value", "")
nace_id = self.env["res.partner.industry"].search(
[("full_name", "=like", nace_code + "%")], limit=1
)
nace_id = nace_map.get(nace_code, False)
nace_parent_code = binding.get("parentCode", {}).get("value", "")
parent_id = self.env["res.partner.industry"].search(
[("full_name", "=like", nace_parent_code + "%")], limit=1
)
parent_id = nace_map.get(nace_parent_code, False)
nace_name = binding.get("EN", {}).get("value")
if not nace_id:
nace_id = self.env["res.partner.industry"].create(
nace = self.env["res.partner.industry"].create(
{
"name": nace_name,
"full_name": nace_name,
"parent_id": parent_id and parent_id.id,
"full_name": f"{nace_code} - {nace_name}",
"parent_id": parent_id,
}
)
nace_ids |= nace_id
nace_map[nace_code] = nace.id
nace_ids |= nace
for lang, lang_code in languages:
nace_translated = binding.get(lang_code, {}).get("value")
nace_id.with_context(lang=lang).write(
nace = nace_ids.browse(nace_map[nace_code])
nace.with_context(lang=lang).write(
{
"name": nace_translated,
"full_name": nace_translated,
"full_name": f"{nace_code} - {nace_translated}",
}
)
return nace_ids
Expand All @@ -176,8 +179,20 @@ def update_partner_industry_eu_nace(self):
languages = self.get_languages()
query = self._create_query(languages)
result = requests.get(
ENDPOINT, params={"format": "json", "query": query}, timeout=60
ENDPOINT, params={"format": "json", "query": query}, timeout=120
)
result.raise_for_status()
nace_ids = self._create_nace_industry(result, languages)
return nace_ids

def action_partner_industry_eu_nace(self):
self.update_partner_industry_eu_nace()
tree_view_id = self.env.ref("base.res_partner_industry_view_tree").id
return {
"name": _("Partner Industries by EU NACE"),
"view_mode": "tree",
"res_model": "res.partner.industry",
"view_id": tree_view_id,
"type": "ir.actions.act_window",
"domain": [],
}
24 changes: 15 additions & 9 deletions l10n_eu_nace/wizard/res_partner_industry_eu_nace_wizard_view.xml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8" ?>
<!-- Copyright 2022 Moduon
<!-- Copyright 2024 Moduon
License LGPL-3.0 or later (https://www.gnu.org/licenses/lgpl.html). -->
<odoo>

Expand All @@ -9,15 +9,21 @@
<field name="arch" type="xml">
<form string="Import NACE industries">
<sheet>
<group>
<button
type="object"
name="update_partner_industry_eu_nace"
string="Import NACE industries"
class="btn-primary"
/>
</group>
<p>
Import NACE industries from ShowVoc API. This will
create new industries or update existing ones with their
translations for all active languages in the system.
</p>
</sheet>
<footer>
<button
type="object"
name="action_partner_industry_eu_nace"
string="Import NACE industries"
class="btn-primary"
/>
<button string="Cancel" class="btn-secondary" special="cancel" />
</footer>
</form>
</field>
</record>
Expand Down
3 changes: 1 addition & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
# generated from manifests external_dependencies
pycountry
requests
schwifty
schwifty

0 comments on commit 83c94c1

Please sign in to comment.