Merge pull request #453 from CLARIAH/dev

Preparing for release 1.3.9

.dockerignore

@@ -0,0 +1,55 @@
+# SPDX-FileCopyrightText: 2022 Albert Meroño, Rinke Hoekstra, Carlos Martínez
+#
+# SPDX-License-Identifier: MIT
+
+*~
+\#*\#
+bin/
+!bin/grlc-server
+build/
+config.ini
+db-cache.json
+db.json
+DispatchSparqlQuery.ipynb
+docker-compose.yml
+Dockerfile2
+*/.DS_Store
+.eggs/
+FileLoaders.ipynb
+Get_Parameters.ipynb
+GetYamlDecorators.ipynb
+.git/
+.*!.gitignore
+grlc.egg-info/
+.idea
+include/
+ink_ext_XXXXXX_img0.png
+.ipynb_checkpoints/
+lib/
+local/
+*.log
+node_modules
+pip-selfcheck.json
+Process_sparql_query_text.ipynb
+*.pyc
+.pytest_cache
+.Python
+ReleaseProcedure.md
+response_1700168662326.html
+Rewrite_Query.ipynb
+share/
+src/config.ini
+src/FileLoaderTesting.ipynb
+ssl-certificates/
+!static/swagger-ui/dist/lib
+*.swp
+testQueries/
+TODOs.md
+TwitterAPIKeys.md
+venv3.10/
+venv3.11/
+venv3.8/
+venv3.9/
+venv-dev/
+venv-tests.sh
+.vscode/

.flake8

@@ -0,0 +1,5 @@
+[flake8]
+per-file-ignores = 
+    src/__init__.py:F401
+    src/prov.py:E203
+    tests/test_grlc.py:F401

.github/workflows/linting.yml

@@ -0,0 +1,23 @@
+name: Lint
+
+on:
+  push:
+    paths:
+      - 'src/*.py'
+      - 'tests/*.py'
+jobs:
+  linter:
+    runs-on: ubuntu-latest
+    name: Lint
+    steps:
+      - name: Check out source repository
+        uses: actions/checkout@v3
+      - name: Set up Python environment
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.11"
+      - name: flake8 Lint
+        uses: py-actions/flake8@v2
+        with:
+          max-line-length: "127"
+          path: "src"

.github/workflows/publish_pypi.yml

@@ -12,7 +12,7 @@ jobs:
     - name: Set up Python
       uses: actions/setup-python@v1
       with:
-        python-version: 3.7
+        python-version: 3.11
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip

.github/workflows/testing.yml

@@ -6,7 +6,7 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-latest, macos-latest, windows-latest]
-        python-version: [3.7, 3.8, 3.9]
+        python-version: [3.8, 3.9, 3.10.x, 3.11]
     steps:
     - uses: actions/checkout@master
     - name: Set up Python ${{ matrix.python-version }}

.gitignore

@@ -39,3 +39,4 @@ TODOs.md
 TwitterAPIKeys.md
 config.ini
 ink_ext_XXXXXX_img0.png
+build/

CITATION.cff

@@ -16,7 +16,7 @@ authors:
     given-names: Carlos
     orcid: "https://orcid.org/0000-0001-5565-7577"
 cff-version: "1.0.3"
-date-released: 2023-07-30
+date-released: 2024-02-17
 doi: 10.5281/zenodo.1064391
 license: MIT
 message: "If you use this software, please cite it as below."
@@ -29,4 +29,4 @@ keywords:
   - "linked-data"
   - "semantic-web"
   - "linked-data-api"
-version: "1.3.8"
+version: "1.3.9"

CONTRIBUTING.md

@@ -58,6 +58,7 @@ services:
      - USERMAP_GID=1000
      - USERMAP_UID=1000
      - GRLC_GITHUB_ACCESS_TOKEN=xxx
+     - GRLC_GITLAB_ACCESS_TOKEN=yyy
      - GRLC_SERVER_NAME=grlc.io
 ```
 

CONTRIBUTORS.md

@@ -23,3 +23,4 @@ This is a list of all people who have contributed to grlc. Big thanks to everyon
 [GenEars](https://github.com/GenEars)
 [nichtich](https://github.com/nichtich)
 [jblom](https://github.com/jblom)
+[abelsiqueira](https://github.com/abelsiqueira)

Dockerfile

@@ -2,15 +2,17 @@
 #
 # SPDX-License-Identifier: MIT
 
-FROM python:3.8.16
+FROM python:3.11-slim
 MAINTAINER [email protected]
 
 # Default values for env variables
 ARG GRLC_GITHUB_ACCESS_TOKEN=
+ARG GRLC_GITLAB_ACCESS_TOKEN=
 ARG GRLC_SERVER_NAME=grlc.io
 ARG GRLC_SPARQL_ENDPOINT=http://dbpedia.org/sparql
 
 ENV GRLC_GITHUB_ACCESS_TOKEN=$GRLC_GITHUB_ACCESS_TOKEN \
+    GRLC_GITLAB_ACCESS_TOKEN=$GRLC_GITLAB_ACCESS_TOKEN \
     GRLC_SERVER_NAME=$GRLC_SERVER_NAME \
     GRLC_SPARQL_ENDPOINT=$GRLC_SPARQL_ENDPOINT
 

README.md

@@ -30,7 +30,7 @@ If you use grlc in your work, please cite it as:
 ```
 
 ## What is grlc?
-grlc is a lightweight server that takes SPARQL queries (stored in a GitHub repository, in your local filesystem, or listed in a URL), and translates them to Linked Data Web APIs. This enables universal access to Linked Data. Users are not required to know SPARQL to query their data, but instead can access a web API.
+grlc is a lightweight server that takes SPARQL queries (stored in a GitHub or GitLab repository, in your local filesystem, or listed in a URL), and translates them to Linked Data Web APIs. This enables universal access to Linked Data. Users are not required to know SPARQL to query their data, but instead can access a web API.
 
 ## Quick tutorial
 For a quick usage tutorial check out our wiki [walkthrough](https://github.com/CLARIAH/grlc/wiki/Quick-tutorial) and [list of features](https://github.com/CLARIAH/grlc/wiki/Features).
@@ -43,7 +43,7 @@ Your queries can add API parameters to each operation by using the [parameter ma
 Your queries can include special [decorators](#decorator-syntax) to add extra functionality to your API.
 
 ### Query location
-grlc can load your query collection from different locations: from a GitHub repository (`api-git`), from local storage (`api-local`), and from a specification file (`api-url`). Each type of location has specific features and is accessible via different paths. However all location types produce the same beautiful APIs.
+grlc can load your query collection from different locations: from a GitHub repository (`api-git`), from a GitLab repository (`api-gitlab`), from local storage (`api-local`), and from a specification file (`api-url`). Each type of location has specific features and is accessible via different paths. However all location types produce the same beautiful APIs.
 
 #### From a GitHub repository
 > API path:
@@ -58,6 +58,19 @@ grlc can make use of git's version control mechanism to generate an API based on
 
 grlc can also use a subdirectory inside your Github repo. This can be done by including a subdirectory in the URL path (`http://grlc-server/api-git/<user>/<repo>/subdir/<subdir>`).
 
+#### From a GitLab repository
+> API path:
+`http://grlc-server/api-gitlab/<user>/<repo>`
+
+grlc can build an API from any GitLab repository, specified by the GitLab user name of the owner (`<user>`) and repository name (`<repo>`).
+
+For example, assuming your queries are stored on a GitLab repo: `https://gitlab.com/c-martinez/grlc-queries`, point your browser to the following location
+`http://grlc.io/api-gitlab/c-martinez/grlc-queries/`
+
+grlc can make use of git's version control mechanism to generate an API based on a specific version of queries in the repository. This can be done by including the name of a branch in the URL path (`http://grlc-server/api-gitlab/<user>/<repo>/branch/<branch>`), for example: `http://grlc.io/api-gitlab/c-martinez/grlc-queries/branch/master`
+
+grlc can also use a subdirectory inside your GitLab repo. This can be done by including a subdirectory in the URL path (`http://grlc-server/api-gitlab/<user>/<repo>/subdir/<subdir>`), for example: `http://grlc-server/api-gitlab/c-martinez/grlc-queries/subdir/subdir`.
+
 #### From local storage
 > API path:
 `http://grlc-server/api-local/`
@@ -251,10 +264,21 @@ Syntax:
 
 Example [query](https://github.com/CLARIAH/grlc-queries/blob/master/transform.rq) and the equivalent [API operation](http://grlc.io/api-git/CLARIAH/grlc-queries/#/default/get_transform).
 
+### `endpoint-method`
+Allows the query to be sent from the grlc server to the SPARQL endpoint using either `GET` or `POST` http method. (Default: `POST`)
+
+Syntax:
+```
+#+ endpoint-method: GET
+```
+
+Example [query](https://github.com/CLARIAH/grlc-queries/blob/master/endpoint-method.rq) and the equivalent [API operation](http://grlc.io/api-git/CLARIAH/grlc-queries/#/default/get_endpoint_method).
+
 ### Example APIs
 
 Check these out:
 - http://grlc.io/api-git/CLARIAH/grlc-queries
+- http://grlc.io/api-gitlab/c-martinez/grlc-queries
 - http://grlc.io/api-url?specUrl=https://raw.githubusercontent.com/CLARIAH/grlc-queries/master/urls.yml
 - http://grlc.io/api-git/CLARIAH/wp4-queries-hisco
 - http://grlc.io/api-git/albertmeronyo/lodapi
@@ -282,9 +306,9 @@ To run grlc via [docker](https://www.docker.com/), you'll need a working install
 docker run -it --rm -p 8088:80 clariah/grlc
 ```
 
-The docker image allows you to setup several environment variable such as `GRLC_SERVER_NAME` `GRLC_GITHUB_ACCESS_TOKEN` and `GRLC_SPARQL_ENDPOINT`:
+The docker image allows you to setup several environment variable such as `GRLC_SERVER_NAME` `GRLC_GITHUB_ACCESS_TOKEN`,`GRLC_GITLAB_ACCESS_TOKEN` and `GRLC_SPARQL_ENDPOINT`:
 ```bash
-docker run -it --rm -p 8088:80 -e GRLC_SERVER_NAME=grlc.io -e GRLC_GITHUB_ACCESS_TOKEN=xxx -e GRLC_SPARQL_ENDPOINT=http://dbpedia.org/sparql -e DEBUG=true clariah/grlc
+docker run -it --rm -p 8088:80 -e GRLC_SERVER_NAME=grlc.io -e GRLC_GITHUB_ACCESS_TOKEN=xxx -e GRLC_GITLAB_ACCESS_TOKEN=yyy -e GRLC_SPARQL_ENDPOINT=http://dbpedia.org/sparql -e DEBUG=true clariah/grlc
 ```
 
 ### Pip
@@ -346,19 +370,21 @@ You can use grlc as a library directly from your own python script. See the [usa
 Regardless of how you are running your grlc server, you will need to configure it using the `config.ini` file. Have a look at the [example config file](./config.default.ini) to see how it this file is structured.
 
 The configuration file contains the following variables:
- - `github_access_token` [access token](#github-access-token) to communicate with Github API.
+ - `github_access_token` [access token](#gitaccess-token) to communicate with Github API.
+ - `gitlab_access_token` [access token](#git-access-token) to communicate with GitLab API.
  - `local_sparql_dir` local storage directory where [local queries](#from-local-storage) are located.
  - `server_name` name of the server (e.g. grlc.io)
  - `sparql_endpoint` default SPARQL endpoint
  - `user` and `password` SPARQL endpoint default authentication (if required, specify `'none'` if not required)
  - `debug` enable debug level logging.
+ - `gitlab_url` to specify the base url of your GitLab instance.
 
-##### GitHub access token
-In order for grlc to communicate with GitHub, you'll need to tell grlc what your access token is:
+##### Git access token
+In order for grlc to communicate with GitHub and/or GitLab, you'll need to tell grlc what your access token is:
 
-1. Get a GitHub personal access token. In your GitHub's profile page, go to _Settings_, then _Developer settings_, _Personal access tokens_, and _Generate new token_
-2. You'll get an access token string, copy it and save it somewhere safe (GitHub won't let you see it again!)
-3. Edit your `config.ini` or `docker-compose.yml` as value of the environment variable `GRLC_GITHUB_ACCESS_TOKEN`.
+1. Get a [GitHub personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/about-authentication-to-github#authenticating-to-the-api-with-a-personal-access-token) or [GitLab personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#create-a-personal-access-token).
+2. You'll get an access token string, copy it and save it somewhere safe.
+3. Edit your `config.ini` (`github_access_token` and `gitlab_access_token` respectively) and/or `docker-compose.yml` (`GRLC_GITHUB_ACCESS_TOKEN` and `GRLC_GITLAB_ACCESS_TOKEN` environment variables).
 
 # Contribute!
 grlc needs **you** to continue bringing Semantic Web content to developers, applications and users. No matter if you are just a curious user, a developer, or a researcher; there are many ways in which you can contribute:

bin/grlc-server

@@ -25,17 +25,17 @@ def runViaWaitress(port=8088):
 
 def runViaGunicorn(port=8088):
     from gunicorn.app.base import BaseApplication
-    from gunicorn.six import iteritems
+
     class StandaloneApplication(BaseApplication):
         def __init__(self, app, options=None):
             self.options = options or {}
             self.application = app
             super(StandaloneApplication, self).__init__()
 
         def load_config(self):
-            config = dict([(key, value) for key, value in iteritems(self.options)
+            config = dict([(key, value) for key, value in self.options.items()
                            if key in self.cfg.settings and value is not None])
-            for key, value in iteritems(config):
+            for key, value in config.items():
                 self.cfg.set(key.lower(), value)
 
         def load(self):

config.default.ini

@@ -4,6 +4,7 @@
 
 [auth]
 github_access_token = xxx
+gitlab_access_token = yyy
 
 [local]
 local_sparql_dir = /home/grlc/queries/
@@ -12,9 +13,12 @@ local_sparql_dir = /home/grlc/queries/
 # Default endpoint, if none specified elsewhere
 sparql_endpoint = http://dbpedia.org/sparql
 server_name = grlc.io
+
 # endpoint default authentication
 user = none
 password = none
+# sparql_access_token = SPARQL endpoint HTTP authorization token
+
 # Logging level
 debug = True