An example for importing properties with constraints

[shigapov]

Hi Dennis, I managed to run import_one.py and import_list.py and to import entities with multilingual labels and external IDs. Now I would like to add the constraints to properties, how can I make it?

[D063520]

Hi! So the properties are there, no? or do you mean to make the plugin work?

[shigapov]

The properties contain only labels and "Wikidata PID". I thought that the local equivalent of P2302 is also created for all imported properties, no?

The extension WikibaseQualityConstraints is installed.

[D063520]

When you import an entity then all related properties and statements are imported. The outgoing entities and properties are only imported as leaves. If you want all properties of an outgoing object or a property you need to run the import one again. BTW, can we have a chat once?
(otherwise we would impirt recursivly all Wikidata ; ))

[shigapov]

Hm, I started with a clean Wikibase instance, and wanted to create just one property like this:

python3 import_one.py P31
WARNING: /.local/lib/python3.10/site-packages/rdflib_jsonld/__init__.py:9: DeprecationWarning: The rdflib-jsonld package has been integrated into rdflib as of rdflib==6.0.1.  Please remove rdflib-jsonld from your project's dependencies.
  warnings.warn(
Traceback (most recent call last):
  File "/WikibaseSync/import_one.py", line 31, in <module>
    wikibase_importer.change_property(wikidata_property, wikibase_repo, True)
  File "/WikibaseSync/util/util.py", line 1110, in change_property
    self.changeClaims(wikidata_item, wikibase_item)
  File "/WikibaseSync/util/util.py", line 865, in changeClaims
    for wikibase_claims in wikibase_item.claims:
AttributeError: 'NoneType' object has no attribute 'claims'
CRITICAL: Exiting due to uncaught exception <class 'AttributeError'>

My use case is just to create all properties with constraints. In principle, for that I need to create a list with all Wikidata PIDs in the list-file and then to run python import_list.py, right?

P.S. Sure, let's agree on time via LI.

[D063520]

can you run the script a second time?

[shigapov]

On a clean instance after the second run of import_one.py I have:

python3 import_one.py P31
WARNING: /.local/lib/python3.10/site-packages/rdflib_jsonld/__init__.py:9: DeprecationWarning: The rdflib-jsonld package has been integrated into rdflib as of rdflib==6.0.1.  Please remove rdflib-jsonld from your project's dependencies.
  warnings.warn(

WARNING: API error modification-failed: Property [[Property:P1|P1]] already has label "Wikidata PID" associated with language code en.
Importing P31
Change Property P31
Import Property P31 from Wikidata
WARNING: API error modification-failed: Property [[Property:P2|P2]] already has label "instance of" associated with language code en.
Could not set description of  -1
Edit to page [[my:Property:-1]] failed:
modification-failed: Property [[Property:P2|P2]] already has label "instance of" associated with language code en.
[messages: [{'name': 'wikibase-validator-label-conflict', 'parameters': ['instance of', 'en', '[[Property:P2|P2]]'], 'html': {'*': 'Property <a href="/wiki/Property:P2" title="Property:P2">P2</a> already has label "instance of" associated with language code en.'}}];
 help: See http://localhost/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes.]
This should not happen 6
Error probably property or item already existing  Edit to page [[my:Property:-1]] failed:
modification-failed: Property [[Property:P2|P2]] already has label "instance of" associated with language code en.
[messages: [{'name': 'wikibase-validator-label-conflict', 'parameters': ['instance of', 'en', '[[Property:P2|P2]]'], 'html': {'*': 'Property <a href="/wiki/Property:P2" title="Property:P2">P2</a> already has label "instance of" associated with language code en.'}}];
 help: See http://localhost/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes.]
Traceback (most recent call last):
  File "/WikibaseSync/import_one.py", line 31, in <module>
    wikibase_importer.change_property(wikidata_property, wikibase_repo, True)
  File "/WikibaseSync/util/util.py", line 1110, in change_property
    self.changeClaims(wikidata_item, wikibase_item)
  File "/WikibaseSync/util/util.py", line 865, in changeClaims
    for wikibase_claims in wikibase_item.claims:
AttributeError: 'NoneType' object has no attribute 'claims'
CRITICAL: Exiting due to uncaught exception <class 'AttributeError'>

[D063520]

I'm trying to reproduce it ... did you use https://www.mediawiki.org/wiki/Wikibase/Docker?

[shigapov]

Yes, the latest release https://github.com/wmde/wikibase-release-pipeline/tree/wmde.2 within https://github.com/UB-Mannheim/RaiseWikibase.

[D063520]

As you saw I run into this:

wmde/wikibase-release-pipeline#265

I will wait a bit to see if someone has an idea on how to fix it ... passed my morning trying to make it work. Had never a problem with the old version ....

[shigapov]

Yes, a weird problem.

[phucty]

Regarding the warning of "Property [[Property:P2|P2]] already has label "instance of" associated with language code en."
--> It is better to search property names instead of getting error while adding a new property.

WikibaseSync/util/PropertyWikidataIdentifier.py

Line 23 in 6d04d6a

wikibase_item.editEntity(data, summary=u'Insert a property to have a wikidata identifier')

--> This will create a new property ID and remove it. At the end, the property ID number will be very big. like this one

Regarding:
Traceback (most recent call last): File "/WikibaseSync/import_one.py", line 31, in <module> wikibase_importer.change_property(wikidata_property, wikibase_repo, True) File "/WikibaseSync/util/util.py", line 1110, in change_property self.changeClaims(wikidata_item, wikibase_item) File "/WikibaseSync/util/util.py", line 865, in changeClaims for wikibase_claims in wikibase_item.claims: AttributeError: 'NoneType' object has no attribute 'claims' CRITICAL: Exiting due to uncaught exception <class 'AttributeError'>

It seems that there was an imported problem of wikibase_item in this function:

WikibaseSync/util/util.py

Line 1057 in 6d04d6a

def change_item(self, wikidata_item, wikibase_repo, statements):

Maybe you have to debug to see what was happened.

I also got this problem, then, I found that the error is in my config/application.config.ini.
If we use the default setting of Wikibase Docker, it might be wikibase.svc as the prefix of entityUri and propertyUri

[D063520]

@shigapov : could you check, I pushed a fix. Also the config files are more aligned to the latest release

[shigapov]

I ran python3 import_one.py P31 and it created the property with everything in it, very nice, thank you!

But then I wanted to create yet another property and it has failed:

python3 import_one.py P281
WARNING: /.local/lib/python3.10/site-packages/rdflib_jsonld/__init__.py:9: DeprecationWarning: The rdflib-jsonld package has been integrated into rdflib as of rdflib==6.0.1.  Please remove rdflib-jsonld from your project's dependencies.
  warnings.warn(

WARNING: API error modification-failed: Property [[Property:P1|P1]] already has label "Wikidata PID" associated with language code en.
Importing P281
Change Property P281
Import Property P281 from Wikidata
claimsToRemove  []
P1855
False
Import Property P1855 from Wikidata
WARNING: API error modification-failed: Property [[Property:P4|P4]] already has label "Wikidata property example" associated with language code en.
Could not set description of  -1
Edit to page [[my:Property:-1]] failed:
modification-failed: Property [[Property:P4|P4]] already has label "Wikidata property example" associated with language code en.
[messages: [{'name': 'wikibase-validator-label-conflict', 'parameters': ['Wikidata property example', 'en', '[[Property:P4|P4]]'], 'html': {'*': 'Property <a href="/wiki/Property:P4" title="Property:P4">P4</a> already has label "Wikidata property example" associated with language code en.'}}];
 help: See http://localhost/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes.]
This should not happen 6
Error probably property or item already existing  Edit to page [[my:Property:-1]] failed:
modification-failed: Property [[Property:P4|P4]] already has label "Wikidata property example" associated with language code en.
[messages: [{'name': 'wikibase-validator-label-conflict', 'parameters': ['Wikidata property example', 'en', '[[Property:P4|P4]]'], 'html': {'*': 'Property <a href="/wiki/Property:P4" title="Property:P4">P4</a> already has label "Wikidata property example" associated with language code en.'}}];
 help: See http://localhost/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes.]
Import Entity Q70 from Wikidata
Traceback (most recent call last):
  File "/WikibaseSync/import_one.py", line 31, in <module>
    wikibase_importer.change_property(wikidata_property, wikibase_repo, True)
  File "/WikibaseSync/util/util.py", line 1112, in change_property
    self.changeClaims(wikidata_item, wikibase_item)
  File "/WikibaseSync/util/util.py", line 992, in changeClaims
    claim = self.translateClaim(wikidata_claim.get('mainsnak'))
  File "/WikibaseSync/util/util.py", line 551, in translateClaim
    claim = pywikibot.Claim(self.wikibase_repo, self.id.get_id(wikidata_propertyId), datatype='wikibase-item')
  File "/WikibaseSync/util/IdSparql.py", line 52, in get_id
    return self.mapProperty[id]
KeyError: 'P1855'
CRITICAL: Exiting due to uncaught exception <class 'KeyError'>

[D063520]

@shigapov new fix .... this should not have happened ... the problem is that the version that we have in production is not fully aligned to this one. I will try to move this one to production so that we can contribute better to the repo.

Did you execute the imports above in a test instance?

[D063520]

@shigapov did you find time to test it?

[shigapov]

Hi @D063520! I've just repeated the experiment on a fresh Wikibase instance 1.36 with your updates from 20.12.2021. I've got the same error message as explained in #4 (comment). So I ran python3 import_one.py P31 and then python3 import_one.py P281. The first command worked excellent, and the second one has failed.

[mezez]

I ran python3 import_one.py P31 and it created the property with everything in it, very nice, thank you!

But then I wanted to create yet another property and it has failed:

python3 import_one.py P281
WARNING: /.local/lib/python3.10/site-packages/rdflib_jsonld/__init__.py:9: DeprecationWarning: The rdflib-jsonld package has been integrated into rdflib as of rdflib==6.0.1.  Please remove rdflib-jsonld from your project's dependencies.
  warnings.warn(

WARNING: API error modification-failed: Property [[Property:P1|P1]] already has label "Wikidata PID" associated with language code en.
Importing P281
Change Property P281
Import Property P281 from Wikidata
claimsToRemove  []
P1855
False
Import Property P1855 from Wikidata
WARNING: API error modification-failed: Property [[Property:P4|P4]] already has label "Wikidata property example" associated with language code en.
Could not set description of  -1
Edit to page [[my:Property:-1]] failed:
modification-failed: Property [[Property:P4|P4]] already has label "Wikidata property example" associated with language code en.
[messages: [{'name': 'wikibase-validator-label-conflict', 'parameters': ['Wikidata property example', 'en', '[[Property:P4|P4]]'], 'html': {'*': 'Property <a href="/wiki/Property:P4" title="Property:P4">P4</a> already has label "Wikidata property example" associated with language code en.'}}];
 help: See http://localhost/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes.]
This should not happen 6
Error probably property or item already existing  Edit to page [[my:Property:-1]] failed:
modification-failed: Property [[Property:P4|P4]] already has label "Wikidata property example" associated with language code en.
[messages: [{'name': 'wikibase-validator-label-conflict', 'parameters': ['Wikidata property example', 'en', '[[Property:P4|P4]]'], 'html': {'*': 'Property <a href="/wiki/Property:P4" title="Property:P4">P4</a> already has label "Wikidata property example" associated with language code en.'}}];
 help: See http://localhost/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes.]
Import Entity Q70 from Wikidata
Traceback (most recent call last):
  File "/WikibaseSync/import_one.py", line 31, in <module>
    wikibase_importer.change_property(wikidata_property, wikibase_repo, True)
  File "/WikibaseSync/util/util.py", line 1112, in change_property
    self.changeClaims(wikidata_item, wikibase_item)
  File "/WikibaseSync/util/util.py", line 992, in changeClaims
    claim = self.translateClaim(wikidata_claim.get('mainsnak'))
  File "/WikibaseSync/util/util.py", line 551, in translateClaim
    claim = pywikibot.Claim(self.wikibase_repo, self.id.get_id(wikidata_propertyId), datatype='wikibase-item')
  File "/WikibaseSync/util/IdSparql.py", line 52, in get_id
    return self.mapProperty[id]
KeyError: 'P1855'
CRITICAL: Exiting due to uncaught exception <class 'KeyError'>

On a fresh install, I tried to reproduce this,

First with python import_one.py P31

Second with python import_one.py P281

Both imports worked successfully

[D063520]

@shigapov: sorry, could you try again and check if it is exactly the same error? Can it be you run into this issue #6?

[D063520]

It could be related also to this: #15