Remove SPARQLWrapper dependency #744

gromgull · 2017-05-18T08:24:10Z

Replace it with "SPARQLWrapper-Lite", which is based on the requests library.

This gains us lots of nice modern HTTP library features, such as keepalive, connection pooling, and by exposing the kwargs to be sent to requests, we can do http basic auth etc. easily.

SPARQLWrapper also has lots of legacy code from supporting various old RDF stores from the days before the SPARQL Protocol was a thing. The new code only supports SPARQL 1.1 Protocol standard, and can therefore be much shorter!

This is not quite done - PR for testing + feedback! (but I run this in production, so it's also not completely pre-beta :) )

gromgull · 2017-11-03T08:40:06Z

I have used this in production since May - it's pretty good. Lets merge? @joernhees ?

joernhees · 2017-11-03T08:52:32Z

sure, but shouldn't this just go upstream and replace the SPARQLWrapper default? i somehow fear that we're creating yet another WTF for the future ;) especially with colliding names won't we confuse everyone and in future talk about SPARQLWrapper's SPARQLWrapper and RDFLib's SPARQLWrapper ?

gromgull · 2017-11-03T10:04:27Z

No - it's only 120 lines of code, and it removes the SPARQLWrapper dependency all together. But it doesn't try to replace everything SPARQLWrapper does - people also use that for getting stuff out of a SPARQL endpoint, without wrapping it as a RDFLib store. This code does nothing for them.

scossu · 2018-01-03T21:02:09Z

@gromgull @joernhees Any decision on merging this PR?
Also, since it is based on requests, does this allow for connection pooling without depending on the keepalive package?

gromgull · 2018-01-03T21:16:19Z

I use it in production - so I am happy to have it merged :)

And yes, it should support connection pool without keepalive.

scossu · 2018-01-13T15:28:16Z

@gromgull I merged this on a separate repo for testing but I get an error:

>>> from rdflib.plugins.stores.sparqlstore import SPARQLUpdateStore
RDFLib Version: 5.0.0-dev
>>> from rdflib.term import URIRef
>>> store = SPARQLUpdateStore(queryEndpoint='http://localhost:3030/lsup-dev/query', update_endpoint='http://localhost:3030/lsup-dev/update')
>>> fds = Dataset(store)
>>> fgr = fds.graph(URIRef('urn:bogus:1'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/scossu/code/lakesuperior/virtualenv/lib/python3.5/site-packages/rdflib/graph.py", line 1623, in graph
    self.store.add_graph(g)
  File "/home/scossu/code/lakesuperior/virtualenv/lib/python3.5/site-packages/rdflib/plugins/stores/sparqlstore.py", line 721, in add_graph
    "CREATE GRAPH %s" % self.node_to_sparql(graph.identifier))
  File "/home/scossu/code/lakesuperior/virtualenv/lib/python3.5/site-packages/rdflib/plugins/stores/sparqlstore.py", line 638, in update
    self.setNamespaceBindings(initNs)
AttributeError: 'SPARQLUpdateStore' object has no attribute 'setNamespaceBindings'

Did the API change?

and a single parser for graph-based results

Replace with a very thin wrapper for modern RDF stores speaking SPARQL 1.1 protocol. This is based on `requests`.

all passing any kwargs to requests

gromgull · 2018-05-15T08:57:31Z

@scossu yes - that method is gone. I wonder why the tests didn't catch that the first time around. Will fix!

joernhees

main concern still is the name that will lead to confusion... left some inline comments.

joernhees · 2018-05-15T19:12:22Z

setup.py

@@ -6,9 +6,9 @@

 kwargs = {}
 kwargs['install_requires'] = [ 'six', 'isodate', 'pyparsing']


requests is a new dependency, no?

joernhees · 2018-05-15T19:17:47Z

rdflib/plugins/sparql/results/xmlresults.py

@@ -65,18 +58,11 @@ def __init__(self, source):
        elif results is not None:
            type_ = 'SELECT'
        else:
-            g = Graph()
-            try:
-                g.parse(data=xmlstring)


what happens to CONSTRUCT queries now?

joernhees · 2018-05-15T19:22:12Z

rdflib/plugins/stores/sparqlstore.py

-    raise Exception(
-        "SPARQLWrapper not found! SPARQL Store will not work." +
-        "Install with 'pip install SPARQLWrapper'")
+from .sparqlwrapper import SPARQLWrapper


can we somehow avoid using SPARQLWrapper as a name (any caps)? Maybe SPARQLConnector? Or not expose it in its own module and internalize it here as _SPARQLWrapper?

I really would like to avoid the confusion

joernhees · 2018-05-15T19:44:37Z

rdflib/query.py

@@ -195,10 +195,11 @@ def _set_bindings(self, b):
        _get_bindings, _set_bindings, doc="a list of variable bindings as dicts")

    @staticmethod
-    def parse(source, format='xml', **kwargs):
+    def parse(source=None, format=None, content_type=None, **kwargs):


hmm, what happens if source is None?

i'd somehow find the following slightly clearer (if it's what is intended)

def parse(source, format='xml', content_type=None, **kwargs): parser = plugin.get(content_type or format, ResultParser() return parser.parse(source, content_type=content_type, **kwargs)

joernhees · 2018-05-15T20:05:45Z

re #391

wikier · 2018-10-20T13:36:21Z

In addition, if you consider the underline implementation could be useful standalone, happy to make it part of SPARQLWrapper 2.x ;-)

jpmccu · 2018-10-30T17:09:45Z

This broke our ability to configure POST requests against SPARQL endpoints, and it looks like the setReturnFormat() function went missing. How should I address switching over?

gromgull · 2018-11-01T08:43:10Z

@jimmccusker this builds on the requests and you can configure requests as much as requests lets you, any additional kwargs you pass to the store constructor will be passed to requests, i.e. for basic auth:

import rdflib
from rdflib.plugins.stores.sparqlstore import SPARQLStore

store = SPARQLStore('http://localhost:8000', auth=('user', 'pass'))
g = rdflib.Graph(store)
list(g[rdflib.RDFS.Class])

Return format is also set for the constructor - i.e. pass returnFormat

I will update the doc strings in #872 to make this clear!

This was removed in 5.0.0 (see RDFLib/rdflib#744), but it may be useful for people wanting backwards compatebility, or who relied on particular SPARQLWrapper features. In retrospect, this code should always have been in the SPARQLWrapper project, avoiding the awkward circular rdflib <-> SPARQLWrapper dependency :) Now you can install rdflib + sparqlwrapper and do: ```py >>> import rdflib >>> g = rdflib.Graph('SPARQLWrapper', identifier='http://dbpedia.org') >>> g.open('http://dbpedia.org/sparql') >>> g.value(rdflib.URIRef('http://dbpedia.org/resource/Berlin'), rdflib.RDFS.label) rdflib.term.Literal(u'Berlin', lang='en') ```

gromgull · 2018-11-01T11:03:16Z

@jimmccusker For max backward-compatibility, hopefully this will be merged and you can use your original code: RDFLib/sparqlwrapper#126

gromgull added cleanup store Related to a store. labels May 18, 2017

This was referenced Nov 3, 2017

SPARQLStore does not support non-XML results #785

Closed

CONSTRUCT query with empty result on persistent dataset raises an error #775

Closed

Using Auth credentials with SPARQLStore VS SPARQLUpdateStore #755

Closed

joernhees added the SPARQL label Nov 3, 2017

gromgull added 5 commits May 15, 2018 10:52

also register sparql result parsers by content-type

654bc78

and a single parser for graph-based results

Remove the SPARQLWrapper dependency

7255e6d

Replace with a very thin wrapper for modern RDF stores speaking SPARQL 1.1 protocol. This is based on `requests`.

clean up sparqlwrapper-lite

6f63508

all passing any kwargs to requests

content-type kwarg for all sparql result parsers

87e61a7

swallow commit_pending_transaction kwarg for SPARQLStore

433dc16

gromgull force-pushed the master branch from 67f8d12 to 433dc16 Compare May 15, 2018 08:52

forgot missing setNamespaceBindings for updates

7c90f64

joernhees reviewed May 15, 2018

View reviewed changes

joernhees added this to the rdflib 5.0.0 milestone May 15, 2018

gromgull merged commit c2c0187 into RDFLib:master Oct 27, 2018

This was referenced Oct 27, 2018

rdflib.plugins.stores.sparqlstore.SPARQLStore remove returnFormat=XML #819

Closed

Empty SPARQL query results from Fuseki #759

Closed

gromgull added a commit that referenced this pull request Oct 27, 2018

actually remove the SPARQLWrapper dependency, re: #744

b0e5595

gromgull mentioned this pull request Nov 1, 2018

rename new SPARQLWrapper to SPARQLConnector #872

Merged

gromgull mentioned this pull request Nov 1, 2018

Restore SPARQLWrapper backed SPARQLStore from RDFLib 4.2.2 RDFLib/sparqlwrapper#126

Open

stevemn mentioned this pull request Jun 18, 2019

Issuing POST request with credentials via SPARQLStore #921

Closed

joernhees mentioned this pull request Oct 30, 2019

make SPARQLConnector work with DBpedia #941

Merged

gneissone mentioned this pull request May 10, 2020

Errors when using rdflib 5.0.0 lawlesst/vivo-rdflib-sparqlstore#4

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove SPARQLWrapper dependency #744

Remove SPARQLWrapper dependency #744

gromgull commented May 18, 2017

gromgull commented Nov 3, 2017

joernhees commented Nov 3, 2017

gromgull commented Nov 3, 2017

scossu commented Jan 3, 2018

gromgull commented Jan 3, 2018

scossu commented Jan 13, 2018

gromgull commented May 15, 2018

joernhees left a comment

joernhees May 15, 2018

joernhees May 15, 2018

joernhees May 15, 2018

joernhees May 15, 2018

joernhees commented May 15, 2018

wikier commented Oct 20, 2018

jpmccu commented Oct 30, 2018

gromgull commented Nov 1, 2018

gromgull commented Nov 1, 2018 •

edited

Loading

		@@ -6,9 +6,9 @@

		kwargs = {}
		kwargs['install_requires'] = [ 'six', 'isodate', 'pyparsing']

Remove SPARQLWrapper dependency #744

Remove SPARQLWrapper dependency #744

Conversation

gromgull commented May 18, 2017

gromgull commented Nov 3, 2017

joernhees commented Nov 3, 2017

gromgull commented Nov 3, 2017

scossu commented Jan 3, 2018

gromgull commented Jan 3, 2018

scossu commented Jan 13, 2018

gromgull commented May 15, 2018

joernhees left a comment

Choose a reason for hiding this comment

joernhees May 15, 2018

Choose a reason for hiding this comment

joernhees May 15, 2018

Choose a reason for hiding this comment

joernhees May 15, 2018

Choose a reason for hiding this comment

joernhees May 15, 2018

Choose a reason for hiding this comment

joernhees commented May 15, 2018

wikier commented Oct 20, 2018

jpmccu commented Oct 30, 2018

gromgull commented Nov 1, 2018

gromgull commented Nov 1, 2018 • edited Loading

gromgull commented Nov 1, 2018 •

edited

Loading